Web Scraping for Lead Generation: Build a List from Public Business Directories

What Web Scraping for Lead Generation Actually Looks Like

Sales prospecting often starts the same way: you know the type of company that needs your product, but you don't have a list of real people to contact. Buying a list feels risky, and manually copying names, phone numbers, and email addresses from a business directory takes hours you don't have.

A no-code web scraper changes this equation. Instead of hunting through pages of a public directory and pasting data into a spreadsheet, you point the tool at the listings you want. It captures the structured contact data—company names, descriptions, phone numbers, website links—and gives you a clean Excel or CSV file ready for your CRM.

This guide shows you how to do exactly that. You'll build a targeted lead list from a public business directory without writing code, managing servers, or wrestling with selectors. You'll also learn the legal guardrails that keep your prospecting compliant and the data-quality steps that make your list worth using.

When This Workflow Works and When It Doesn't

This workflow fits a specific scenario: you want to extract visible, structured business listings from a public directory page and turn them into a usable lead list. It excels when a directory presents contact details in a consistent format across multiple pages.

It works well when:

• The directory lists businesses with their contact details in a table or repeating card layout.
• You can view the listings without paying for access or solving CAPTCHAs.
• The site uses standard pagination—Next Page links, Load More buttons, or infinite scroll.
• You need a list for your own outreach, not for resale.

It's not the right tool when:

• The directory requires you to log into an account that you don't have or aren't authorized to use.
• The site actively blocks automated browsing or uses aggressive anti-bot measures.
• You need real-time, continuously updated data rather than a one-time snapshot.
• The contact data you want is behind individual profile pages, not on the listing page itself.

In those cases, a browser-based scraper won't solve the problem, and you're better off with a dedicated data provider or manual research. For the sweet spot—public, paginated listings that display business contact info—this no-code approach saves real time.

Legal and Data Quality Constraints You Need to Know

Before you extract a single row of data, understand two things: your legal obligations and the quality of the contacts you're collecting.

Legal boundaries when scraping public directories

Scraping publicly accessible data sits in a gray area that depends on where you operate and where the data subjects reside. There is no universal exemption that makes scraping business contact data automatically legal.

Under the GDPR, you need a lawful basis to collect and process personal data—even from a public directory. Legitimate interest is often cited, but it is not a blanket permission. The CNIL and the European Data Protection Board both clarify that legitimate interest requires a documented balancing test and specific safeguards to protect individuals' rights. CNIL EDPB

In the US, the CCPA formerly had a B2B exemption that partially shielded business contact data. That exemption expired on January 1, 2023. California business contacts now have the same rights as individual consumers, and a sole proprietor's business contact counts as a consumer under CCPA. UnifyGTM

Before you scrape, check the directory's terms of service, respect robots.txt restrictions, and consider whether you can rely on legitimate interest—with a documented assessment—for your specific use case.

Data quality: what you actually get

Scraped data mirrors what the directory publishes. If a listing has an outdated phone number or a generic info@ email address, that's what you'll extract. Build a verification step into your workflow: spot-check a sample of contacts, validate email formats, and cross-reference key details against company websites. A scraped lead list is a starting point, not a finished asset.

Your Step-by-Step No-Code Workflow

The following steps use Newise, a no-code Chrome extension, to extract business contacts from a public directory. The same principles apply to any similar tool, but the Newise workflow requires no setup beyond adding the extension to Chrome.

Step 1: Define Your Ideal Customer Profile and Select a Directory

Start with clarity on who you're targeting. Write down the criteria that matter for your outreach:

• Industry: e.g., commercial real estate, dental practices, marketing agencies.
• Location: city, state, or country.
• Company size: employee count or revenue range if the directory includes it.
• Contact role: owner, office manager, or head of sales if role data is present.

Then choose a public directory where these businesses list themselves. Some popular examples include:

• Clutch: B2B service providers by category and location.
• Yelp for Business: local businesses with phone numbers and website links.
• Manta: small business directory with owner contact details.
• Brownbook: global business listings.
• Chamber of Commerce directories: local business members with verified contact info.

Pick one directory that aligns with your ICP, navigate to a category or search results page, and confirm that the listings show the contact fields you want before you start extraction.

Step 2: Install Newise and Navigate to Your Directory Page

Newise is a free Chrome extension that works inside your browser session. No account is required—install it and you're ready.

1. Add Newise to Chrome.
2. Go to your chosen directory's listing page. Make sure you can see the business cards or table rows you want to extract.
3. If the directory requires a login that you hold, log in first. Because Newise runs in your browser, it works on pages you're already authenticated to see.

Step 3: Select the Listing or Table to Extract

Open the Newise extension from your browser toolbar and choose "Select a list or table." Hover over the directory listings. Newise will highlight the detected data region. Click to confirm.

Newise captures structured fields it recognizes—business name, phone, website URL, address—and presents them as a spreadsheet-like preview. You don't write CSS selectors or XPath. If the directory uses a consistent structure, the extraction takes one click.

Step 4: Handle Pagination to Collect All Results

Most directories spread listings across multiple pages. Newise offers built-in pagination handling for three common patterns:

• Next Page: click the Next Page button once, and Newise detects the pagination link and offers to auto-scroll through all pages.
• Load More: click the Load More button, and Newise will repeat the click until no new results appear.
• Infinite Scroll: scroll to the bottom, and Newise triggers additional loads until the full list is captured.

Run the pagination until Newise reports that all pages are collected. The result is a single, merged dataset from the entire directory section.

Step 5: Clean, Sort, and Rename Your Extracted Data

Raw scraped data often needs light cleanup. Inside the Newise preview table, you can:

• Delete irrelevant columns: remove fields you don't need, like rating stars or review counts.
• Edit cell values: fix formatting errors or remove stray characters.
• Sort columns: group rows by location or company name.
• Rename headers with AI: select a column, choose "Rename with AI," and Newise will suggest a human-readable header like "Phone Number" instead of "field_3."

Step 6: Export Your Lead List to Excel or CSV

When the data looks right, export it. Click the export button in Newise and choose your format:

• Excel (.xlsx): best for direct use in most CRMs and outreach tools.
• CSV: universally compatible for importing into Salesforce, HubSpot, or Google Sheets.
• JSON: useful if your CRM or automation platform prefers structured data.
• Markdown: handy for documenting your lead sources.

Save the file, and your lead list is ready for the next step.

Verifying and Delivering Your Lead List

Before you load your list into an outreach sequence, verify what you have.

Spot-check a sample. Pick 10–15 rows at random. Check the phone number by calling or via a quick online search. Visit the website if one was extracted. Confirm the business matches your ICP criteria.

Validate email formats. A scraper won't catch typos. Scan the email column for missing @ symbols, duplicates, or role-based addresses that may bounce (sales@, admin@).

Enrich where needed. If your directory only gave you a business name and phone, you might add a manual step to find a direct email or LinkedIn profile for the decision-maker.

Segment before importing. Group your list by priority—by location, industry, or company size—so your CRM can route leads to the right rep or sequence.

Once verified, import the list into your CRM. Use a dedicated "source" tag like "directory-scrape-feb-2025" so you can track performance later.

A scraped lead list turns a multi-hour manual task into a repeatable, minutes-long process. Paired with a solid ICP definition and a verification step, it's a practical way to build your pipeline from publicly available business data without touching code.

Frequently asked questions

Is it legal to scrape business contact data from public directories?

It depends on the jurisdiction and your lawful basis. Under GDPR, scraping personal data requires a lawful basis such as legitimate interest, which demands a documented balancing test and safeguards. The CCPA's B2B exemption expired in 2023, so California business contacts now have the same protections as consumer data. Always review the directory's terms of service and consult legal counsel for your specific use case.

Do I need coding skills to scrape a business directory?

No. No-code Chrome extensions like Newise let you extract structured data from directory listings by clicking on the list or table you want to capture. There is no need to write selectors, manage servers, or run scripts.

Can I scrape a directory that requires a login?

Yes, if you hold a valid account. Because Newise runs inside your browser session, it works on pages you're already authenticated to see. If you don't have an account, a browser-based scraper won't grant access.

How do I handle multi-page directory listings?

Newise provides built-in pagination handling for Next Page links, Load More buttons, and infinite scroll. You perform the action once—clicking the Next Page button or scrolling—and Newise detects the pattern and continues collecting until all pages are merged into one dataset.