HomeGuide Scraper — Contractor Lead Generation From HomeGuide By ZIP Code

An Apify Actor built to scrape HomeGuide for contractor lead generation: it turns any HomeGuide search URL into a 25-field provider row — company, founded year, intro, profile URL, photo, city/state/ZIP, lat/lng, reviews, hires, topPro, highDemand, online, verifiedBadge. Give it https://homeguide.com/search?zipcode=73301&midtail=ZQgrYdjH, it reads the ZIP and service from the query string and queries HomeGuide's own search gateway — no browser, no HTML crawl, no login — reaching 100 unique rows in about 6 calls.

Main Use Cases

• Build a home services lead list for contractor lead generation: one deduplicated row per provider, ready for CSV or CRM import.
• Collect homeguide leads ZIP by ZIP — a plumber contact list USA, HVAC company leads, or a whole contractor database by zip code from a batch of search URLs.
• Answer who are the top plumbers in [city] or best home service companies [state] with real review counts, hires and badges.
• Rank "top rated contractors near me" shortlists by filtering verifiedBadge, highDemand, online and topPro.
• Run home services market research and home services competitive analysis across trades and states, including niche categories such as well drilling company leads.
• Monitor trade services leads in your target ZIPs — new providers, review and badge changes — on a daily or weekly schedule.

How The HomeGuide Scraper Works

The Actor never loads the search page. homeguide.com sits behind a Cloudflare managed challenge and its robots.txt disallows /search, so HTML crawling returns nothing. The search page is a JavaScript app that asks HomeGuide's own GraphQL gateway for results, and that gateway is what the Actor calls:

POST https://liaison-homeguide-prod.stellate.sh/

That host's robots.txt allows exactly /, the only path the Actor requests; no browser starts and respectRobotsTxtFile stays on. The Actor parses zipcode plus service (or midtail) from each start URL, seeds one POST per URL, maps every provider to a dataset row, and re-asks that search for its next window until maxItems or three empty windows. Requests dedupe via Crawlee's request queue, rows via providerId.

Input Parameters

Field	Type	Required	Default	Notes
startUrls	array	Yes	["https://homeguide.com/search?zipcode=73301&midtail=ZQgrYdjH"]	Search URLs needing ?zipcode=... plus ?service=... or ?midtail=.... The page itself is never fetched.
maxItems	integer	No	100	Provider rows to collect, 1–100000; windows are pulled until it is reached or the search runs dry.
maxConcurrency	integer	No	10	Start URLs queried in parallel, 1–50; one URL is queried one call at a time.
delayMillis	integer	No	250	Courtesy delay between two calls to the same search, 0–10000.
resume	boolean	No	false	Skip start URLs an earlier run already explored and providers it already collected.
proxy	object	No	{ "useApifyProxy": false }	Standard Apify proxy configuration.

maxItems counts providers, not requests; a hard budget of maxItems / 5 plus 20 per start URL, at least 50, is a safety net the chains never reach. A URL missing zipcode or service/midtail is warned and skipped, and if nothing usable is left the run fails loudly instead of finishing empty.

Choosing URLs

Service keys are the midtail (or service) query parameter of any HomeGuide search URL:

{
    "startUrls": [
        { "url": "https://homeguide.com/search?zipcode=73301&midtail=ZQgrYdjH" },
        { "url": "https://homeguide.com/search?zipcode=78701&service=RK7RdSfw" }
    ]
}

Feed one search URL per ZIP × service to build a local contractor database; maxConcurrency controls how many run side by side.

Resuming A Previous Run

Each platform run gets a fresh dataset, so resume continues the work, not the storage. It reads a named key-value store (resume-store), skips start URLs a previous run explored to exhaustion, seeds the dedupe set so an already collected provider is never written twice, and is written back on normal completion and inside the aborting handler.

A start URL counts as explored only after three consecutive empty windows. A chain stopped because maxItems was reached never completes its URL, so a resumed run picks it up instead of dropping unwritten rows — it emits only the new rows from where the last one stopped. Merge datasets for a combined export, or leave resume off for a clean run.

Output Fields

One row per provider, 25 fields in four groups:

Group	Fields	Contents
Identity & profile	providerId, companyName, companyFounded, intro, url, imageUrl	Dedupe ID, trading name, founding year when reported, self-written intro, profile and photo URLs.
Business location	city, state, zip, lat, lng	The provider's registered business address, down to coordinates.
Reputation & badges	reviewsCount, reviewsAvg, hires, topPro, highDemand, online, verifiedBadge	Review count, average score out of 5, recorded jobs, and HomeGuide's Top Pro, high demand, online and verified marks.
Search provenance	searchCity, searchState, serviceName, sourceUrl, zipcode, serviceKey, crawledAt	Start URL, ZIP and service key passed in, the ZIP's resolved city/state, service name, ISO 8601 timestamp.

Schema-required: providerId, companyName, url, sourceUrl, zipcode, serviceKey, crawledAt; the rest are nullable. city/state/zip/lat/lng is the business address, searchCity/searchState the searched ZIP's resolution. Reviews are counted and averaged, not fetched individually.

Saved Console views: Overview, Reputation (reviews, rating, hires, badges, founded) and Location (address, lat/lng, searched city/state/ZIP). The dataset is the only output — export CSV/JSONL from the Dataset page or GET /v2/datasets/<id>/items?format=csv.

Example Input And Output For A Home Services Lead List

Example input

{
    "startUrls": [
        { "url": "https://homeguide.com/search?zipcode=73301&midtail=ZQgrYdjH" },
        { "url": "https://homeguide.com/search?zipcode=78701&service=RK7RdSfw" }
    ],
    "maxItems": 100,
    "resume": false
}

Example output

A row captured from a real run:

{
    "providerId": "5ec28d434f55d70aa5890777",
    "companyName": "KUTSCHER DRILLING, LTD.",
    "companyFounded": 1925,
    "intro": "We are a full service water well drilling and service company. We are family owned and operated since 1925.",
    "url": "https://homeguide.com/tx/austin/well-drilling/kutscher-drilling-ltd",
    "imageUrl": "https://res.cloudinary.com/liaison-inc/image/upload/v1/.../profile.jpg",
    "city": "San Marcos",
    "state": "TX",
    "zip": "78666",
    "lat": 29.8942825,
    "lng": -97.9810845,
    "reviewsCount": 0,
    "reviewsAvg": 0,
    "hires": 2,
    "topPro": false,
    "highDemand": false,
    "online": false,
    "verifiedBadge": false,
    "searchCity": "Austin",
    "searchState": "TX",
    "serviceName": "Plumbing",
    "sourceUrl": "https://homeguide.com/search?zipcode=73301&midtail=ZQgrYdjH",
    "zipcode": "73301",
    "serviceKey": "ZQgrYdjH",
    "crawledAt": "2026-10-04T11:35:59.000Z"
}

Build A Local Contractor Database Without The HomeGuide API

No public HomeGuide API exists, so the Actor talks to the gateway the site's own search page uses. Three properties were found the hard way and are load bearing:

1. The operation must be named Search. The gateway resolves its SearchResponse union from the operation name: an anonymous operation fails with Cannot read properties of undefined (reading 'value'), any other name fails with SearchResponse not existing.
2. anonymousId must always be sent. Without it response comes back null with the same crash.
3. There is no pagination that works. searchByMap is broken server side and search answers with one rotating window of about 30 providers out of a larger pool; the same search called again returns a different window, so the Actor keeps asking for more windows and deduplicating.

Deduplication And The maxItems Quota

One POST per start URL, then the next window, each request with its own queue key. Every new provider claims a maxItems slot before pushData, so the dataset never overshoots; requests dedupe via the request queue, rows by providerId. Results are tallied in-process rather than read back, because Dataset.getInfo() lags on the platform.

Stopping Behaviour

Quota is claimed before each pushData, so the dataset never exceeds maxItems by one. A quota-stopped search is never marked explored, so nothing is dropped; three empty windows do mark it explored and later runs skip it. On every exit — normal or aborting — a STATE record (itemsScraped, maxItems, source URLs, timestamp) goes to the default key-value store, then the resume record to resume-store; aborting calls Actor.exit(), so a stopped run ends quickly and cheaply.

How The Code Is Organised

src/main.ts reads input, seeds the window chain, owns the quota and handles aborting; src/homeguide.ts holds the GraphQL document, URL parsing and row mapping; src/routes.ts is the SEARCH handler; src/quota.ts and src/dedup.ts hold the ItemQuota counter and SeenKeys set; src/state.ts persists the resume record. The crawler is HttpCrawler from @crawlee/cheerio — session pool, retries, proxy support and responseType handling, no selector and no browser.

Testing

npm test
npm run lint
npx tsc --noEmit
apify validate-schema

Offline and fixture-driven: test/fixtures/search-response.json records a gateway response, test/homeguide.test.ts asserts URL parsing, the request and every dataset field against it, and test/main.test.ts replays it in rotating windows through a real HttpCrawler to cover quota, dedupe, chaining and "never mark a search explored early". Gateway changes then fail in CI, not as empty runs in production.

Failure Handling

failedRequestHandler logs the endpoint URL, error message and HTTP status code; GraphQL errors become exceptions so they surface in the run log instead of a silently short dataset.

Limitations

• Targets one site. Point it elsewhere only at sources that permit automated access.
• One search returns a rotating window of about 30 providers and HomeGuide exposes no working pagination, so covering a pool needs several calls and a finite pool caps how many rows one start URL can ever give you.
• Provider fields are whatever the gateway returns; adding one means changing SEARCH_QUERY in src/homeguide.ts and .actor/dataset_schema.json.
• Reviews are counted and averaged, not fetched individually.

HomeGuide Leads: How Many Rows One Search Returns

One call returns about 30 providers from a rotating window; while a window still brings back someone new the Actor asks for the next one, stopping at maxItems or after three empty windows. Measured on zipcode=73301&midtail=ZQgrYdjH (plumbing) with maxItems set out of reach:

Calls	1	3	4	5	6	12
Rows	30	67	78	95	106	143

Calls overlap, so the first few are cheap and the last slow: about 6 calls reach 100 rows. A single search has a finite pool — fewer rows than maxItems means it ran dry, and the fix is more start URLs (other zipcodes or services), not a higher cap.

Run homeguide-scraper From The Apify API, Console Or Schedule

Apify Console

Start URL:      https://homeguide.com/search?zipcode=73301&midtail=ZQgrYdjH
Maximum items:  100

API

curl -X POST "https://api.apify.com/v2/acts/HOMEGUIDE-SCRAPER/runs?token=<token>" \
  -H "Content-Type: application/json" \
  -d '{"startUrls":[{"url":"https://homeguide.com/search?zipcode=73301&midtail=ZQgrYdjH"}],"maxItems":50}'

Local

Put the input in storage/key_value_stores/default/INPUT.json and run apify run; results land in storage/datasets/default/. Local storage/ never syncs to the Console — only a platform run proves real output.

Scheduling

0 6 * * *   # 06:00 UTC daily

Cost

No browser starts, so runs are cheap and scale with requests: one call at a time per start URL until maxItems or the pool runs dry, each returning about 30 providers. Pay-per-event: a result at $0.00095 on the FREE plan (down to $0.0008 at GOLD+) plus Actor Start at $0.00005 per run. Proxy usage is billed separately if enabled — the default proxy is { "useApifyProxy": false }.

HomeGuide Vs HomeAdvisor: Comparing Contractor Directory Data

HomeGuide via this Actor gives topPro, highDemand, verifiedBadge, reviewsCount, reviewsAvg, hires, companyFounded, lat/lng and profile URLs, plus the provenance fields showing which ZIP and service surfaced each provider. Angi/HomeAdvisor data comes from homeadvisor-scraper ("$3/1K | Contractor Leads | Elite", Angi internal API, 24 trade slugs, 15 metro ZIP maps) and angi-leads-scraper ($0.008 per pro with lead score); Houzz and Networx options are under Related Actors. Badge and hire data live on HomeGuide, email and license data elsewhere; competitor figures are Store claims, accuracy unverified.

HomeGuide Cost Guide Data: What This Scraper Reads

HomeGuide's cost guide pages are separate from the provider search results this Actor reads and are out of scope: a run makes no page request at all, only the gateway POST to /, so cost guide pages, category pages and profiles are never fetched. serviceKey and serviceName record the trade you searched (for example ZQgrYdjH → Plumbing), so exported rows can be joined to cost guide or pricing material you collect separately.

Integrations & Automation

• Apify API — start runs with your input, read results from GET /v2/datasets/<id>/items?format=csv.
• Webhooks — fire on ACTOR.RUN.SUCCEEDED to push each batch of rows into a spreadsheet, database or CRM.
• Zapier / Make — trigger on new runs or dataset items and append providers to Google Sheets, Airtable or your sales tool.
• Recurring schedule — daily at 0 6 * * * (06:00 UTC) to monitor target ZIPs for new or re-badged providers, weekly to refresh a home services lead list; with resume: true, scheduled runs walk a long URL list across executions instead of re-exploring exhausted searches.

Related Actors

• homeguide-scraper — same-site HomeGuide scraper with byUrls/byCategory/byLocation modes.
• homeadvisor-scraper — Angi/HomeAdvisor leads, 24 trade slugs, 15 metro ZIP maps.
• angi-leads-scraper — Angi pros with lead score and monitoring.
• contractor-lead-scraper — Google Maps HVAC, plumbing and roofing leads with verified emails.
• houzz-professionals-scraper — Houzz professionals with emails and licenses.

FAQ

Is there an official HomeGuide API?

No. HomeGuide has no public API; this Actor calls the gateway its own search page uses (POST https://liaison-homeguide-prod.stellate.sh/) with the operation named Search and an anonymousId.

What are the alternatives to homeguide-scraper?

crawlerbros/homeguide-scraper targets the same site (claims 50 states plus DC and 80+ categories, byUrls/byCategory/byLocation modes), Related Actors covers the Angi, Houzz and Networx options, and DIY means rediscovering the Search name, anonymousId and rotating windows.

What's the best way to get contractor leads without paying per lead?

Export a directory instead of buying leads one at a time: a result costs $0.00095 on the FREE plan, so you pay only for rows received across any number of ZIP codes. Per-lead marketplaces charge as leads arrive — homeadvisor-scraper advertises "$3/1K | Contractor Leads | Elite".

How can I scrape HomeGuide data?

Pass search URLs like https://homeguide.com/search?zipcode=<ZIP>&midtail=<service-key>, set maxItems, run. The page is never fetched: the query string is parsed, the gateway is queried, and one row per provider is written.

Can I get a list of plumbers by ZIP code in the US?

Yes — HomeGuide is US-only and ZIP-keyed. ?zipcode=73301&midtail=ZQgrYdjH (plumbing, Austin TX) yields about 30 per call, roughly 100 rows after 6 calls and 143 after 12, each with address, lat/lng, reviewsCount, reviewsAvg, hires and badges.

Is HomeGuide data public and legal to use?

The profiles are public — no login, no account, no authentication bypass, and homeguide.com pages are never fetched. But homeguide.com's robots.txt disallows /search, and HomeGuide's terms prohibit copying, scraping, aggregating or redistributing content by automated tools except limited search engine indexing; compliance is your responsibility as operator.

How do I build a local contractor database for market research?

One search URL per ZIP × service, run in bulk with maxConcurrency, export CSV to your spreadsheet or CRM, resume: true to walk a long ZIP list across runs, and a weekly schedule to keep it current.

SEO Keywords

homeguide scraper, scrape homeguide, homeguide leads, homeguide api, homeguide data, homeguide alternative, homeguide cost guide, homeguide cost guide scraper, homeguide vs homeadvisor, homeguide pro directory, contractor lead generation, home services lead list, contractor database by zip code, local contractor database, contractor directory data, plumber contact list usa, hvac company leads, top rated contractors near me, home services market research, home services competitive analysis, trade services leads, angie's list contractor data, well drilling company leads, best home service companies [state]

For AI Agents & LLM Apps

Purpose: extract US contractor / home service provider records (25 fields each) from HomeGuide search URLs via HomeGuide's GraphQL search gateway — no browser, no page fetch.

Minimal input:

{ "startUrls": [{ "url": "https://homeguide.com/search?zipcode=73301&midtail=ZQgrYdjH" }] }

Output fields: providerId, companyName, companyFounded, intro, url, imageUrl, city, state, zip, lat, lng, reviewsCount, reviewsAvg, hires, topPro, highDemand, online, verifiedBadge, searchCity, searchState, serviceName, sourceUrl, zipcode, serviceKey, crawledAt (schema-required: providerId, companyName, url, sourceUrl, zipcode, serviceKey, crawledAt).

Behaviors an agent should know:

• Start URLs need ?zipcode= plus ?service= or ?midtail=; malformed URLs are skipped and a run left with none usable fails rather than returning an empty dataset.
• maxItems caps provider rows, not requests — a slot is claimed before every pushData; safety budget maxItems / 5 + 20 per start URL, minimum 50.
• One search returns a rotating window of about 30 with no working pagination; rows dedupe by providerId, chains stop after three empty windows, and a finite pool means more start URLs rather than a higher cap.
• resume (default false) uses the resume-store key-value store, skips exhausted URLs, seeds the dedupe set, is saved inside aborting, and emits only new rows.
• delayMillis (250) paces calls to one search; maxConcurrency (10) parallelises start URLs only — one call at a time per URL.
• proxy defaults to { "useApifyProxy": false } (no proxy billed unless enabled); traffic is POSTs to / on liaison-homeguide-prod.stellate.sh, operation Search, anonymousId always sent, respectRobotsTxtFile on.

Billing: pay-per-event — result at $0.00095 each on the FREE plan (down to $0.0008 at GOLD+), plus Actor Start at $0.00005 per run.

Legal & Compliance Disclaimer

HomeGuide Provider Scraper is an independent tool, not affiliated with, endorsed by or sponsored by HomeGuide or its operator, Liaison, Inc.

It performs no login and bypasses no authentication: it reads zipcode and service/midtail out of the search URLs you supply and sends GraphQL POSTs to / on HomeGuide's search gateway host — the only path that host's robots.txt allows — with respectRobotsTxtFile enabled. It never fetches homeguide.com HTML pages; note that homeguide.com's own robots.txt disallows /search, the page whose query string is parsed, and that HomeGuide's terms prohibit copying, scraping, aggregating, framing or redistributing its content by any automated means, including bots, crawlers and scrapers, except for limited search engine indexing.

The dataset holds business profile data only — no email addresses or phone numbers are collected. You are responsible for your own compliance with HomeGuide's terms of service and applicable data-protection law when you store, combine or use the exported data.