Healthgrades Real-Time Data Scraper

Apify Actor — collect Healthgrades provider data at scale: specialty + location search, full provider profiles, and lead-generation contact details, streamed to your dataset in real time.

⚠️ Paid only — free accounts are limited

This Actor is intended for paying Apify users. Free Apify accounts are limited to a small sample of results per run (2 results by default) and are asked to upgrade for full, unlimited data. This restriction is stated here, in the input schema, and in the run logs — it is a policy limit, not a bug. See the FAQ for details.

---

What it does

• 🔎 Search Healthgrades by specialty + location — run many pairs in one go (e.g. dermatologist in Miami, FL and cardiologist in Dallas, TX). Each row includes the provider's name, specialty, rating, review count, and practice location when listed.
• 📍 Provider details — extract full structured profiles from provider profile URLs: ratings, office locations, board certifications, hospital affiliations, insurance plans, languages, and more.
• 🎯 Lead details — an enable, not filter enrichment pass. Where publicly listed, each provider is enriched with a practice website, email addresses, and social profiles. Every provider is exported even when nothing extra is found — output stays complete and per-1,000-provider runtime stays predictable (lead details add a little extra time per provider).
• ⚡ Real-time streaming — records are pushed to your dataset as soon as each provider is processed, so you can watch results arrive and integrate them live. Memory stays flat on long runs.
• 🔔 Webhooks — push every record to your CRM, Slack, Zapier, Make, or Google Sheets as it is scraped.
• 🧾 Transparent run summary — every run writes an OUTPUT object summarizing what was exported, why the run stopped, and (for transparency) the paywall status of the account that ran it.

Outcome

A clean, deduplicated list of healthcare providers — with contactable details where they exist — ready for outreach, market mapping, and healthcare-sales prospecting.

---

Who it is for

• Medical device & pharma sales teams — build territory lists of specialists by city and specialty.
• Healthcare recruiters — find physicians by specialty and location for placement outreach.
• Private-practice growth agencies — prospect practices that have a website and email but weak online presence.
• Health-tech BD teams — map provider density by metro area for launch planning.
• Insurance brokers — build lists of practices to approach with group plans.
• Lead-generation agencies — enrich CRM pipelines with verified practice contact points.
• Market researchers & consultants — analyze provider supply, ratings, and coverage by region.

---

Use cases

• Build a territory list of dermatologists in the 10 biggest US metros, with practice websites and phones.
• Recruit cardiologists within a state: filter by rating and review count, then contact via practice site.
• Enrich a CRM of known providers: pass profile URLs to Provider Details and get full structured profiles back.
• Map the competitive landscape: how many family-medicine practices exist in a county, and which have a website?
• Power an outreach campaign: export providers with leadDetailsComplete = true (website/email/social found) for high-deliverability prospecting.
• Stream into your stack: use the webhook to push each provider into HubSpot/Make/Zapier the moment it is scraped.
• Insurance prospecting: list practices with multiple office locations (larger groups = larger group plans).
• Academic/health-services research: collect structured snapshots of provider listings by region and specialty.

---

Input schema (every field)

Field	Type	Default	Description
enableSearch	boolean	true	Enable specialty + location search.
searchTasks	array of objects	(1 prefill row)	One row per search: query (specialty or doctor name, required), location (e.g. Miami, FL), and optional per-task maxResults. Add as many rows as you like.
maxResultsPerQuery	integer	10	Default maximum providers per search task. A row's own maxResults overrides it; this value applies when a row leaves it empty.
enableProviderDetails	boolean	false	Enable full-profile extraction for providerUrls.
providerUrls	string list	[]	Healthgrades provider profile URLs to scrape in full.
npiNumbers	string list	[]	Reserved for future identifier-based lookups. Currently a no-op.
enableLeadDetails	boolean	true	Enable lead details: enrich each provider with practice website, emails, and social profiles where publicly listed. Providers without findable details are not filtered out.
maxItems	integer	10	Global cap on dataset rows for the run — raise it (e.g. 10000) for large jobs.
language	string	en	Language for names and addresses (en, es, …).
webhookUrl	string	""	Optional. Each record is POSTed to this URL as it is scraped (in addition to the dataset).
webhookFormat	json | slack	json	Payload format for the webhook. json = the full record object; slack = a Slack-message payload.
proxyConfiguration	object	Apify residential, US	Proxy for reliable collection. US residential is on by default; you can change country or provide custom proxy URLs.

Starter input: the Actor opens pre-filled with one small search (one specialty, one city, 10 providers) so you can hit Start and see live results immediately. Raise the per-task and Max total items values for larger jobs.

Example input

{
    "enableSearch": true,
    "searchTasks": [
        { "query": "dermatologist", "location": "Miami, FL", "maxResults": 200 },
        { "query": "cardiologist", "location": "Dallas, TX" }
    ],
    "maxResultsPerQuery": 100,
    "enableLeadDetails": true,
    "maxItems": 5000,
    "webhookUrl": "https://hooks.zapier.com/hooks/catch/12345/abcdef/",
    "webhookFormat": "json"
}

Provider-details-only input

{
    "enableSearch": false,
    "enableProviderDetails": true,
    "providerUrls": [
        "https://www.healthgrades.com/physician/dr-jane-smith-ym34x"
    ],
    "enableLeadDetails": true
}

---

Output schema (field by field)

Each dataset row is one provider. All fields are always present (null/empty when unknown) so exports are stable for spreadsheets and ETL.

Identification

Field	Type	Description
featureType	string	search (search result), provider (full profile), or lookup.
providerId	string | null	Stable identifier from the profile.
providerUrl	string | null	Provider profile URL.
name	string | null	Provider name (e.g. Dr. Jane Smith, MD).
nameSlug	string | null	URL slug for the profile.
prefix / suffix	string | null	Name prefix/suffix when exposed.
scrapedAt	string	ISO-8601 timestamp of collection.

Professional details

Field	Type	Description
specialty	string | null	Primary specialty (e.g. Dermatology).
subspecialties	string[]	Sub-specialties when listed.
gender	string | null	When listed.
npiNumber	string | null	National Provider Identifier when exposed.
yearsExperience	number | null	When listed.
languages	string[]	Languages spoken, when listed.
boardCertifications	array	{ board, specialty, year? } entries.
hospitalAffiliations	string[]	Hospital names.
insurancePlans	string[]	Insurance plans accepted, when listed.
conditionsTreated / proceduresPerformed	string[]	When listed on the profile.
telehealthAvailable	boolean | null	Telehealth flag when exposed.
acceptingNewPatients	boolean | null	When exposed.
awards	string[]	Awards and recognitions.

Ratings & location

Field	Type	Description
overallRating	number | null	Rating (0–5 scale).
reviewCount	integer | null	Number of patient reviews.
location	string | null	Primary location string (e.g. Miami, FL or full address).
locations	array	Office locations: { name?, address?, city?, state?, zipCode?, fullAddress?, phone?, fax?, isPrimary?, latitude?, longitude? }.

Lead details (with enableLeadDetails)

Field	Type	Description
phone / phones	string | null / string[]	Primary phone (formatted) plus up to 3 unique numbers.
fax	string | null	Fax number when listed.
website	string | null	Practice website when found.
email / emails	string | null / string[]	Contact email(s) when publicly listed on the practice site.
socials	string[]	Social profile URLs (LinkedIn, Facebook, Instagram, X/Twitter, YouTube, …).
linkedinUrl, facebookUrl, instagramUrl, twitterUrl	string | null	Flattened social fields for spreadsheet columns.
leadDetailsComplete	boolean | null	true when at least one extra detail (website/email/social) was found; false when none was; null when lead details are disabled. Providers are never filtered out based on this flag.

Search context

Field	Type	Description
searchQuery	string | null	The specialty/search term that produced this row.
searchLocation	string | null	The location used for this row.
searchTaskIndex	integer | null	0-based index of the producing task.
searchTaskLabel	string | null	Human label, e.g. `dermatologist
position	integer | null	Position within the task when available.

Run OUTPUT (key-value store, key OUTPUT)

{
    "totalPushed": 6,
    "requestsMade": 9,
    "spendingLimitReached": false,
    "stoppedReason": "completed",
    "paywall": {
        "detected": true,
        "isPaying": true,
        "pricingTier": "GOLD",
        "blocked": false,
        "limited": false,
        "freeTierMaxItems": null
    },
    "errors": []
}

stoppedReason is one of completed, max_items, spending_limit, free_tier_limit, free_tier_blocked. The paywall object is included for transparency: detected (platform run), isPaying, pricingTier, blocked, limited, and freeTierMaxItems (the cap in force for free accounts).

---

Webhook setup

1. Put a URL (CRM endpoint, Slack incoming webhook, Zapier, Make, n8n, Google Sheets app script) into the Webhook URL input.
2. Pick a format:
   • json — the full provider record (same object as the dataset row), POSTed once per record as it is scraped.
   • slack — a compact Slack message: provider name, specialty, rating, phone, location, and a profile link.
3. Run the Actor. Records arrive one by one, in near real time, while the run continues.

Notes:

• The dataset is always written first — the webhook is an additional push, so a webhook failure never loses data.
• Delivery failures are logged as a fixed sentence and counted in the run log; they never stop the run.
• Webhook payloads contain only the documented record fields — no internal or debug data.

Slack payload example

{
    "text": ":stethoscope: *Dr. Jane Smith, MD*\n*Specialty:* Dermatology  •  *Rating:* 4.8\n*Phone:* (305) 555-0142\nMiami, FL\n<https://www.healthgrades.com/physician/...|View profile>"
}

---

MCP usage

This Actor works with Apify MCP Server, so MCP clients (Claude, Cursor, and other tool-enabled agents) can call it directly.

Typical MCP tool call:

{
    "tool": "apify/actor-run",
    "arguments": {
        "actorId": "your-actor-id/healthgrades-real-time-data-scraper",
        "input": {
            "enableSearch": true,
            "searchTasks": [{ "query": "dermatologist", "location": "Miami, FL" }],
            "maxResultsPerQuery": 50,
            "enableLeadDetails": true
        }
    }
}

Tips for agent workflows:

• Prefer small maxResultsPerQuery (10–50) so the agent can iterate city-by-city.
• Keep enableLeadDetails on when the agent plans outreach — it gets website, emails, and socials directly in the rows.
• Read OUTPUT.paywall before planning large jobs: on free accounts the run stops at the free-tier sample.
• The flat, all-fields-present row shape makes the dataset directly loadable into LLM-friendly CSV/JSONL.

---

Performance & limits

• Streaming: rows are pushed page-sized-chunk at a time; memory stays flat regardless of job size.
• Runtime: search is fast; enableLeadDetails adds a little extra time per provider (it looks up the practice site and checks its contact pages). A 1,000-provider run with lead details typically takes a few hours depending on concurrency and source responsiveness.
• Visible progress: the run logs what it is doing throughout (Processing 120 provider(s) with lead details…, Lead details: 40/120 provider(s) done., Progress: 40 provider(s) exported.), so a long job is never a black box.
• Predictable lead details: a provider that cannot be enriched in time is still exported, and the whole lead-details phase is capped by a time budget (15 minutes by default, LEAD_DETAILS_BUDGET_SECONDS to change it). Once the budget is spent the remaining providers are exported with the details already collected, so a run's length stays predictable and it always finishes.
• Timeout: the Actor is configured for a 10,000-second run timeout — large jobs fit in one run.
• Memory: 512 MB default; 256–2048 MB supported. Memory does not need to scale with result count.
• Recommended concurrency: keep maxItems at or below ~10,000 per run; split bigger jobs by city or specialty.

---

Paid only / free tier (policy)

Apify injects the account's pay status into every platform run, and this Actor reads it at run start:

• Paying users — normal runs, full output, no caps. One confirmation line is logged: Paying user — full output.
• Free (non-paying) users — by default the run exports a maximum sample (2 results), logs a clear warning to upgrade, and finishes gracefully. This cap keeps the free experience useful while keeping paid output worth paying for.
• The free-tier behavior is not an error: the run ends cleanly with a status message and the paywall object in OUTPUT.
• The owner can tune the free-tier behavior and the lead-details time budget with environment variables on the Actor (documented in .env.example).

If you need full, unlimited exports, upgrade your Apify plan — the Actor detects paid plans automatically on the next run.

---

❓ FAQ

Is this really limited on free accounts? Yes. Free Apify accounts get a small sample per run (2 results by default). Paying accounts get full, unlimited output. This is stated up front in the README and input schema, logged at run start, and surfaced in OUTPUT.paywall.

Why did my run stop after 2 results with a message about upgrading? You ran it from a free Apify account. The Actor capped the run to the free sample and exited gracefully. Upgrade to a paid Apify plan and run it again — no other change needed.

Do lead filters remove providers without contact info? No — this Actor intentionally has no lead filters. The Enable lead details option adds website/email/social fields where they can be found, and every provider is exported even when nothing is found. Filtering would make runtime unpredictable and pricing impossible; enabling keeps output complete and per-1,000-provider runtime consistent (lead details add a little extra time per provider).

How current is the data? Everything is collected in real time during your run — there is no cached database behind it.

Can I scrape by URL or NPI number? URLs: yes — enable Provider Details and paste profile URLs. NPI lookups: the field is reserved; use profile URLs for now.

How many providers can I get in one run? Set maxItems high (e.g. 10000). The run timeout is 10,000 seconds; for very large jobs, split by city or specialty.

Does it work outside the US? Healthgrades lists US providers, so a US connection is used by default for reliable collection.

Can I get emails for every provider? No. Emails are included only when publicly listed on a practice's own site. Many practices publish a phone but no email; those rows still export (with leadDetailsComplete: false).

Is any raw or debug data attached to records? No. Records contain only the documented fields — the same fields land in the dataset, in exports, and in webhook payloads.

How do I get Slack notifications per provider? Set webhookUrl to a Slack incoming webhook and webhookFormat to slack.

Can I use this with Zapier / Make / n8n? Yes — point the webhook at a Zapier/Make/n8n catch hook, or fetch the dataset at the end of the run via the API link in the output.

Is the number of reviews a quality signal? It reflects patient feedback volume on the platform. Combine overallRating + reviewCount to prioritize outreach targets.

---

Support

Open an issue on the Actor's page or contact the maintainer through Apify. Please include the run ID (never edit or redact the run's own logs — they never contain sensitive connection data).