WebMD Doctor Directory Scraper

Extract doctor directory data from WebMD fast and at scale. Collect provider listings with profile links, ratings, location details, and rich metadata in a single run. Built for lead generation, market research, and healthcare directory analysis.

Features

• Fast listing collection - gathers records from directory result pages without opening each doctor profile
• Result cap control - stops as soon as your results_wanted target is reached
• Automatic pagination - continues across pages until target count or max_pages limit
• Rich dataset fields - returns provider, specialty, rating, geo, and location metadata
• Duplicate prevention - avoids duplicate providers across paginated results
• Proxy-ready operation - supports Apify Proxy configuration for reliable production runs

Use Cases

Healthcare Lead Generation

Build targeted doctor lead lists by specialty, state, and city. Export clean provider data for outreach and CRM enrichment.

Local Market Research

Analyze provider density, rating distribution, and specialty coverage in a specific geography. Compare city-level availability for planning and business decisions.

Directory Intelligence

Track listing trends over time by running scheduled collections. Monitor rating changes, active providers, and category movement.

Data Enrichment Pipelines

Use exported records as a source dataset for internal analytics, BI dashboards, or downstream matching workflows.

Competitive Monitoring

Compare provider visibility and quality signals across locations and specialties to support strategic positioning.

---

Input Parameters

Parameter	Type	Required	Default	Description
startUrls	Array of String	No	-	One or more directory URLs to start from. When provided, these override specialty/state/city inputs.
specialty	String	No	"family-medicine"	Medical specialty slug, for example family-medicine, pediatrics, cardiology.
state	String	No	""	State slug filter, for example california, texas, new-york.
city	String	No	""	City slug filter, for example los-angeles, houston.
location	String	No	""	Legacy location input. Use when you need compatibility with older tasks.
results_wanted	Integer	No	50	Maximum number of doctor records to return. Run stops at this number.
max_pages	Integer	No	10	Maximum number of listing pages to visit.
proxyConfiguration	Object	No	{ "useApifyProxy": true }	Proxy setup for reliable large-scale collection.

---

Output Data

Each dataset item can include the following fields.

Field	Type	Description
providerId	String	Unique provider identifier.
profileRecordId	String	Listing record identifier from source.
npi	String	National Provider Identifier when available.
name	String	Full provider name.
firstName	String	First name.
middleName	String	Middle name when available.
lastName	String	Last name.
suffix	String	Name suffix such as Dr. when available.
degree	String	Degree abbreviation when available.
gender	String	Gender label when available.
specialty	String	Primary specialty.
specialties	Array of String	Specialty list from listing data.
specialtyIds	Array	Specialty ID list from source.
profileType	String	Profile tier/type label.
address	String	Full formatted address.
address_street	String	Street address.
address_city	String	City.
address_state	String	State code.
address_zip	String	ZIP code.
stateFullName	String	Full state name when available.
stateSlug	String	State slug.
citySlug	String	City slug.
phone	String	Phone number when available in listing data.
website	String	Website URL when available in listing data.
appointmentUrl	String	Appointment URL when available.
rating	Number	Overall rating value when available.
rating_1	Number	First rating component.
rating_2	Number	Second rating component.
rating_3	Number	Third rating component.
rating_4	Number	Fourth rating component.
reviewCount	Number	Number of reviews.
bio	String	Provider summary text.
bio_text	String	Cleaned plain-text summary.
graduationYear	Number	Graduation year when available.
education	Array of String	Parsed education values when available.
insurance	Array of String	Insurance values when available.
awards	Array	Awards when available.
campaignText	String	Campaign label when available.
campaignLogo	String	Campaign logo URL when available.
image	String	Provider image URL.
url	String	Provider profile URL.
locationId	String	Source location ID.
locationEntityId	String	Source location entity ID.
locationMedicalGroup	String	Medical group name when available.
acceptsMedicare	Boolean	Medicare acceptance when available.
acceptsMedicaid	Boolean	Medicaid acceptance when available.
acceptsNewPatients	Boolean	New patient acceptance when available.
distanceMiles	Number	Distance metric from source listing when available.
latitude	Number	Latitude when available.
longitude	Number	Longitude when available.
source	String	Source domain label.

Note: Fields are only included when data exists in listing results.

---

Usage Examples

Basic Specialty Collection

Collect the first 50 family medicine providers.

{
  "specialty": "family-medicine",
  "results_wanted": 50
}

State + City Targeting

Collect pediatric providers in a specific city.

{
  "specialty": "pediatrics",
  "state": "texas",
  "city": "houston",
  "results_wanted": 120,
  "max_pages": 8
}

Direct URL Start

Use a known directory URL and cap output size.

{
  "startUrls": [
    "https://doctor.webmd.com/providers/specialty/internal-medicine/california"
  ],
  "results_wanted": 200,
  "max_pages": 12,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

---

Sample Output

{
  "providerId": "BE87CA87-62E6-4511-8F21-4E449510B6AF",
  "profileRecordId": "769783",
  "npi": "1245245786",
  "name": "Zhanna Mikulik",
  "firstName": "Zhanna",
  "lastName": "Mikulik",
  "suffix": "Dr.",
  "degree": "MD",
  "gender": "Female",
  "specialty": "Family Medicine",
  "specialties": ["Family Medicine"],
  "specialtyIds": [29259, 29320],
  "profileType": "Basic",
  "address": "6700 University Blvd, Dublin, OH, 43016",
  "address_city": "Dublin",
  "address_state": "OH",
  "address_zip": "43016",
  "rating": 3,
  "rating_1": 3,
  "rating_2": 3,
  "rating_3": 3.125,
  "rating_4": 2.75,
  "reviewCount": 12,
  "bio": "Named to Castle Connolly's Regional Top Doctors list",
  "campaignText": "dr-com-standard",
  "campaignLogo": "https://www.doctor.com/images/logo-dark.png",
  "image": "https://a.doctor.com/img?p=providers/75995%20LOGO%20-Zhanna-Mikulik&w=200",
  "url": "https://doctor.webmd.com/doctor/zhanna-mikulik-be87ca87-62e6-4511-8f21-4e449510b6af-overview",
  "locationId": "A13262620",
  "locationEntityId": "13262620",
  "acceptsNewPatients": true,
  "distanceMiles": 768.74,
  "latitude": 40.098618,
  "longitude": 83.169402,
  "source": "webmd.com"
}

---

Tips for Best Results

Pick Good Starting Scope

• Use both specialty and state for tighter, more relevant datasets.
• Add city when you need local precision.

Control Volume

• Start with results_wanted between 25 and 100 for test runs.
• Increase results_wanted and max_pages only after validating output shape.

Keep Runs Reliable

• Use Apify Proxy, especially for larger collections.
• Prefer residential proxy groups for long production runs.

Improve Data Coverage

• Some optional fields are not present for every listing.
• Use larger sample sizes to increase chance of optional field coverage.

---

Proxy Configuration

For reliable runs, use Apify Proxy:

{
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

---

Integrations

Connect your dataset with:

• Google Sheets - analyze and share records quickly
• Airtable - create searchable healthcare lead tables
• Make - automate enrichment and downstream workflows
• Zapier - trigger CRM and notification actions
• Webhooks - push run data to custom endpoints

Export Formats

• JSON - application workflows and APIs
• CSV - spreadsheet analysis
• Excel - business reporting
• XML - system integration pipelines

---

Frequently Asked Questions

Does the run stop exactly at my requested result count?

Yes. The actor enforces results_wanted and stops when that target is reached.

Can I scrape by city only?

Yes. Provide city with a matching state for best results.

Why are some fields missing in some records?

Listing availability varies by provider. Optional fields appear only when source data includes them.

Can I collect large datasets?

Yes. Increase results_wanted and max_pages, and use proxy settings for reliability.

Can I start from a custom listing URL?

Yes. Use startUrls to begin from any valid WebMD directory URL.

Is this suitable for recurring monitoring?

Yes. Schedule runs and compare datasets over time for trend tracking.

---

Support

For issues or feature requests, use the Actor page issue tracker or Apify Console support channels.

Resources

• Apify Documentation
• Apify API Reference
• Apify Schedules

---

Legal Notice

This actor is intended for legitimate data collection. You are responsible for complying with applicable laws, platform terms, and usage policies in your jurisdiction.