Company Directory Extractor

Extract structured company data from business directory listing pages.

Company Directory Extractor is an Apify Actor built with TypeScript and Crawlee CheerioCrawler. Point it at directory search or listing URLs — Yelp-style results, Yellow Pages, Hotfrog, BBB, chamber-of-commerce listings, or any generic business directory — and get clean company records with optional detail-page enrichment.

Features

• Generic listing-card heuristics — detects business cards via schema.org, common CSS patterns, and heading structures
• Detail-page follow-up — enriches missing website, phone, industry, location, and LinkedIn fields
• Pagination support — follows rel=next, .pagination, and common pager links
• JSON-LD enrichment — reads structured Organization / LocalBusiness data on detail pages
• Retries & rate limiting — configurable retries, concurrency, and polite delays
• Proxy-ready — Apify Proxy support for production scraping
• Graceful abort handling — stops cleanly when the run is aborted or maxResults is reached

Input

Field	Type	Required	Default	Description
startUrls	object[]	Yes	—	Directory listing or search result URLs
maxResults	integer	No	100	Max company records to extract
followDetailPages	boolean	No	true	Visit company profile pages for enrichment
maxDetailPagesPerListing	integer	No	25	Detail pages to enqueue per listing page
maxRequestsPerCrawl	integer	No	200	Total HTTP request budget
minDelayMs	integer	No	1000	Minimum delay between requests (ms)
maxConcurrency	integer	No	10	Parallel request limit
maxRequestRetries	integer	No	3	Retry count for failed requests
proxyConfiguration	object	No	Apify Proxy on	Proxy settings

Example input

{
  "startUrls": [
    { "url": "https://www.hotfrog.com/search/new-york-ny/restaurants" },
    { "url": "https://www.yellowpages.com/search?search_terms=plumber&geo_location_terms=Los+Angeles%2C+CA" }
  ],
  "maxResults": 50,
  "followDetailPages": true,
  "maxDetailPagesPerListing": 10,
  "maxRequestsPerCrawl": 100,
  "minDelayMs": 1500,
  "maxConcurrency": 5,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}

See also: examples/input.json

Output

Each dataset item represents one company:

Field	Type	Description
company	string	Company or business name
website	string	Public website URL (if available)
industry	string	Category or industry
city	string	City
country	string	Country
linkedin	string	Public LinkedIn URL (if available)
phone	string	Public phone number (if available)
sourceUrl	string	Listing page where the company was found
detailUrl	string	Detail/profile page URL (if visited)
scrapedAt	string	ISO extraction timestamp

Example output

{
  "company": "Acme Plumbing Co.",
  "website": "https://www.acmeplumbing.example",
  "industry": "Plumbers",
  "city": "Los Angeles",
  "country": "US",
  "linkedin": "https://www.linkedin.com/company/acme-plumbing",
  "phone": "(213) 555-0199",
  "sourceUrl": "https://www.yellowpages.com/search?search_terms=plumber&geo_location_terms=Los+Angeles%2C+CA",
  "detailUrl": "https://www.yellowpages.com/los-angeles-ca/mip/acme-plumbing-123456",
  "scrapedAt": "2026-07-05T12:00:00.000Z"
}

See also: examples/output.json

How it works

1. Listing extraction — Each start URL is treated as a directory listing page. The Actor scans for business cards using semantic HTML, schema.org microdata, and common directory CSS patterns.
2. Detail enrichment — When key fields are missing and a detail link is found, the Actor enqueues the profile page and merges richer data (including JSON-LD).
3. Pagination — Next-page links are discovered and crawled until maxResults or maxRequestsPerCrawl is reached.
4. Output — Valid company records are pushed to the default dataset via Actor.pushData().

Pricing

Recommended Apify Store pricing: $0.003 per result using the synthetic apify-default-dataset-item pay-per-event model. Configure this in Apify Console under Publication → Monetization.

Run locally

cd company-directory-extractor
npm install
apify run --purge

Set test input in storage/key_value_stores/default/INPUT.json or use the provided example.

Deploy to Apify

apify login
apify push

After pushing, set monetization to Pay per event with apify-default-dataset-item priced at $0.003.

Resources

• Apify SDK for JavaScript
• Crawlee CheerioCrawler
• Pay per event monetization