WebMD Healthcare Provider Directory Scraper

WebMD Healthcare Provider Directory Scraper

Scrape healthcare provider profiles from WebMD by specialty, state, or city, up to a million per run. Each record includes NPI, ratings, patient reviews, accepted insurance, and practice locations. No login or API key. Export to CSV, JSON, Excel, or XML.

WebMD's provider directory holds millions of U.S. doctors, dentists, and specialists, but manually searching and copying data is slow and inconsistent. This Actor reads the public directory directly, letting you pull structured provider records filtered by medical specialty, state, and city. It handles Cloudflare protection with residential proxies so you get clean, complete datasets without writing a single line of code.

Who uses it	What they scrape WebMD for
Healthcare recruiters	Build a list of active providers in a target region for outreach.
Medical sales teams	Identify practices by specialty and location for equipment or pharma campaigns.
Market researchers	Analyze provider density, ratings, and insurance acceptance by metro area.
App developers	Populate a provider directory or referral tool with fresh WebMD data.

What it does

This Actor collects WebMD healthcare provider profiles by specialty, location, or direct URL, and returns each provider as a flat row with practice details, ratings, and reviews.

• 🏥 Specialty and location search: filter by medical specialty, state, and city to target exactly the providers you need.
• 🔗 Direct URL input: paste any WebMD provider listing page URL to scrape a specific set of results.
• 🛡️ Residential proxy support: built-in Apify residential proxies handle Cloudflare anti-bot protection automatically.
• ⚙️ Concurrency control: adjust parallel requests from 1 to 20 to balance speed and polite scraping.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with WebMD data

📋 Build a regional provider list.

A healthcare recruiter scrapes all family medicine doctors in Texas to create a candidate outreach spreadsheet.

📊 Analyze market coverage.

A pharmaceutical sales manager pulls cardiologists across Florida to map territory coverage against insurance acceptance.

⭐ Monitor provider ratings.

A practice consultant scrapes dermatologists in Los Angeles weekly to track review trends and patient satisfaction scores.

🏢 Populate a provider directory.

An app developer scrapes WebMD for all dentists in California to seed a new dental appointment booking platform.

Why choose this scraper

	What you get
No API needed	Reads the public WebMD directory; no registration, OAuth, or rate-limit headaches.
Structured output	Every provider comes out in a consistent flat schema ready for analysis or upload.
Residential proxies included	Cloudflare protection is handled so your runs complete without blocks.
Flexible targeting	Search by specialty, state, and city, or drop in any listing URL you find on WebMD.

How it compares

This Actor offers broader specialty coverage and flexible URL-based scraping compared to other WebMD scrapers on Apify.

Feature	ParseForge	WebMD Doctor Scraper 🩺	WebMD Scraper - Doctors, Reviews & Provider Leads
Specialty search filter	Yes, 38 specialties in dropdown	Not listed	Yes
State and city location filters	Yes	Not listed	Yes
Direct URL input	Yes	Not listed	Not listed
Accepted insurance data	Yes	Not listed	Yes
Patient reviews text	Yes	Not listed	Yes
NPI number	Yes	Not listed	Yes
Residential proxy built-in	Yes	Yes	Not listed

Configure the run

Drive the Actor from a specialty, state, and city combination, or paste a direct WebMD listing URL. Filters apply as each page is read, so only matching providers land in your dataset. The Input tab lists every parameter.

A first run with the defaults:

{
 "maxItems": 10,
 "specialty": "family-medicine",
 "state": "new-york",
 "proxyConfiguration": {
 "useApifyProxy": true,
 "apifyProxyGroups": [
 "RESIDENTIAL"
 ]
 },
 "maxConcurrency": 5
}

A larger pull:

{
 "maxItems": 200,
 "specialty": "family-medicine",
 "state": "new-york",
 "proxyConfiguration": {
 "useApifyProxy": true,
 "apifyProxyGroups": [
 "RESIDENTIAL"
 ]
 },
 "maxConcurrency": 5
}

Pricing

Pay-per-result: $0.01866 per result collected. You pay only for the results written to your dataset.

Results collected	Approximate cost
100 results	$1.87
1,000 results	$18.66
10,000 results	$186.60

New Apify accounts start with $5 in free credit.

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the WebMD Healthcare Provider Directory Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to WebMD through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

$claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/webmd-healthcare-provider-directory-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting zero results?

Check that your specialty, state, and city slugs are spelled correctly and match WebMD's URL format. If using a Start URL, verify the page lists providers. Also confirm your residential proxies are enabled.

The run fails with timeout or block errors.

WebMD requires residential proxies. Make sure your proxy configuration is set to Apify residential. If you are already using them, lower the max concurrency to 2 or 3 to reduce the request rate.

I only got a few results but expected more.

Check your Max Items setting. Free accounts are capped at 100 items. Also verify that your state and city filters are not too narrow; try broadening to state-only or removing the city filter.

Some provider profiles are missing data fields.

Not every WebMD profile contains all fields. Some providers may not list their NPI, accepted insurance, or patient reviews. The Actor returns whatever is publicly available on each profile page.

The run is very slow.

Residential proxies add latency. You can increase max concurrency up to 20 for faster scraping, but this may increase the chance of rate-limiting. Find a balance that works for your target volume.

FAQ

Question	Answer
Do I need a WebMD account or API key?	No. The Actor reads the public provider directory pages. No login, no API key, and no registration with WebMD is required.
What data fields does the Actor return?	Each provider row includes name, NPI, specialties, ratings, review count, accepted insurance plans, practice addresses, phone numbers, and more. The exact field list is shown in the sample output on this page.
Can I scrape providers across all states at once?	Yes. Leave the state and city fields empty when using a specialty, and the Actor will crawl all available states and cities for that specialty. Be mindful of the max items limit for large runs.
How do I scrape a specific list of providers I found on WebMD?	Paste the full URL of the WebMD listing page into the Start URL field. The Actor will scrape only the providers on that page and any paginated results that follow.
Does this Actor handle Cloudflare protection?	Yes. WebMD uses Cloudflare anti-bot protection. The Actor is pre-configured to use Apify residential proxies, which are required for successful runs.
What specialties can I search?	The specialty dropdown includes 38 common specialties such as family medicine, cardiology, dermatology, psychiatry, dentistry, and many more. You can also use a direct URL for any specialty not in the list.
Can I limit how many providers are scraped?	Yes. Set the Max Items field to any number between 1 and 1,000,000. Free accounts are limited to 100 items per run.
What export formats are supported?	You can export your results to CSV, JSON, Excel, XML, or RSS. The dataset is stored in Apify's key-value store and is available for download or API access.
Is it legal to scrape WebMD?	This Actor reads publicly accessible pages. You are responsible for complying with WebMD's terms of service and applicable laws in your jurisdiction. Consult legal counsel if you have concerns.
How fast can I scrape?	Speed depends on your max concurrency setting and proxy performance. Start with the default of 5 parallel requests and increase if you need faster throughput. Residential proxies are inherently slower than datacenter proxies.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by WebMD LLC. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.