Healthgrades Family Practice Scraper

Extract comprehensive family practice physician data from Healthgrades.com at scale. Collect provider profiles including specialties, locations, ratings, contact information, insurance details, and experience indicators in clean, structured datasets. Perfect for healthcare research, provider directories, medical market intelligence, and practice management.

Features

• Comprehensive Data Extraction — Collect complete provider profiles with all available fields
• Location-Based Search — Target specific states, cities, or regions for precise results
• Clean Structured Output — Removes empty fields and deduplicates array values automatically
• Fast API-Based Collection — Direct API access for rapid, reliable data gathering
• Flexible Result Limits — Control exactly how many providers and pages to collect

Use Cases

Healthcare Market Research

Analyze provider density, specialties, and service availability across regions. Identify market gaps, competitive landscapes, and healthcare access patterns for strategic planning and investment decisions.

Provider Directory Building

Create comprehensive, searchable databases of family practice providers. Power healthcare comparison websites, referral systems, telemedicine networks, and medical marketplace applications.

Medical Practice Management

Support practice operations with competitive analysis, network expansion planning, referral development, and geographic coverage optimization. Benchmark against local competitors and identify growth opportunities.

Healthcare Technology Platforms

Power innovative healthcare solutions including patient matching algorithms, provider recommendation engines, access analysis tools, and insurance network verification systems.

Academic and Policy Research

Build datasets for healthcare access studies, geographic coverage analysis, provider distribution research, and public health policy development.

---

Input Parameters

Parameter	Type	Required	Default	Description
keyword	String	No	"Family Practice"	Search keyword or specialty to filter providers.
location	String	No	""	Geographic location (state, city, or "City, ST" format).
startUrl	String	No	—	Direct Healthgrades search URL (overrides keyword/location).
results_wanted	Integer	No	20	Maximum number of providers to extract (1-500).
max_pages	Integer	No	10	Maximum search result pages to process (1-20).
apiOnly	Boolean	No	true	API-only mode for fastest extraction (always enabled).
proxyConfiguration	Object	No	—	Proxy settings for enhanced reliability.

---

Output Data

Each dataset item contains comprehensive provider information from Healthgrades (only non-empty fields included):

Field	Type	Description
id	String	Unique provider identifier.
displayName	String	Full provider name with credentials.
url	String	Direct link to provider profile.
providerId	String	NPI or internal provider ID.
name	String	Provider full name.
specialty	String	Primary medical specialty.
specialties	Array	List of all specialties.
location	String	City and state of practice.
address	Object	Complete address details.
phone	String	Primary contact phone number.
fax	String	Fax number (if available).
website	String	Provider's website URL.
rating	Number	Overall patient rating (1-5 scale).
reviewCount	Number	Total patient review count.
bio	String	Provider biography and background.
image	String	Profile photo URL.
gender	String	Provider gender.
languages	Array	Languages spoken.
education	Array	Medical schools and training.
certifications	Array	Board certifications.
acceptedInsurance	Array	List of accepted insurance providers.
yearsExperience	Number	Years of medical practice.
acceptingNewPatients	Boolean	Whether accepting new patients.
telehealthAvailable	Boolean	Telehealth service availability.
surveyOverallRatingScore	Number	Detailed rating score.
surveyUserCount	Number	Rating survey participant count.
imageUrl	String	Best available provider image.

---

Usage Examples

Basic Location Search

Extract family practice providers from a specific state:

{
  "location": "Texas",
  "results_wanted": 50
}

City-Level Targeting

Focus on providers in a specific city:

{
  "keyword": "Family Practice",
  "location": "Dallas, TX",
  "results_wanted": 100,
  "max_pages": 5
}

Large-Scale Collection

Gather extensive provider data for regional analysis:

{
  "location": "California",
  "results_wanted": 500,
  "max_pages": 20,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

---

Sample Output

{
  "id": "XSK9M",
  "displayName": "Dr. John McClanahan, DO",
  "url": "https://www.healthgrades.com/physician/dr-john-mcclanahan-xsk9m",
  "providerId": "9876543210",
  "name": "Dr. John McClanahan, DO",
  "specialty": "Family Medicine",
  "specialties": ["Family Practice", "Primary Care"],
  "location": "Waco, TX",
  "address": {
    "line1": "7300 Bosque Blvd",
    "line2": "Waco, TX 76710"
  },
  "phone": "(254) 555-1234",
  "fax": "(254) 555-5678",
  "website": "https://www.drjohnmcclanahan.com",
  "rating": 4.9,
  "reviewCount": 192,
  "bio": "Dr. McClanahan has over 20 years of experience in family medicine...",
  "image": "https://photos.healthgrades.com/img/prov/x/s/k/xsk9m_w120h160.jpg",
  "gender": "Male",
  "languages": ["English", "Spanish"],
  "education": ["Texas A&M Health Science Center"],
  "certifications": ["American Board of Family Medicine"],
  "acceptedInsurance": ["Aetna", "Blue Cross", "United Healthcare"],
  "yearsExperience": 20,
  "acceptingNewPatients": true,
  "telehealthAvailable": true,
  "surveyOverallRatingScore": 4.9,
  "surveyUserCount": 192,
  "imageUrl": "https://photos.healthgrades.com/img/prov/x/s/k/xsk9m_w120h160.jpg"
}

---

Tips for Best Results

Optimize Search Parameters

• Use specific city names for targeted results (e.g., "Austin, TX" instead of "Texas")
• Combine keyword and location for most relevant providers
• Start with smaller result sets (20-50) to test configurations

Scale Up Gradually

• Increase results_wanted and max_pages incrementally for large collections
• Monitor extraction statistics in run logs for performance insights
• Use residential proxies for runs exceeding 100 providers

Ensure Data Quality

• Enable proxy configuration for consistent access across regions
• Verify extracted data against Healthgrades for critical applications
• Handle missing fields gracefully in your processing workflows

---

Integrations

Connect your healthcare data with leading business tools:

• Google Sheets — Export provider data for analysis and reporting
• Airtable — Build searchable provider databases and directories
• Slack — Get notifications when extractions complete
• Make — Create automated healthcare workflows and alerts
• Zapier — Trigger actions based on new provider data
• Webhooks — Send data directly to your custom applications

Export Formats

Download your datasets in multiple formats:

• JSON — For developers and API integrations
• CSV — For spreadsheet analysis and Excel reporting
• Excel — For business intelligence and presentation-ready reports
• XML — For system integrations and legacy applications

---

Frequently Asked Questions

How many providers can I extract?

You can extract up to 500 providers per run. The practical limit depends on your search criteria and proxy configuration.

Can I search by specialty other than Family Practice?

Yes, modify the keyword parameter to search for other specialties like "Internal Medicine" or "Pediatrics".

What if some data fields are missing?

Some providers may not have complete information available on Healthgrades. The scraper only includes non-empty fields in the output.

How accurate is the extracted data?

Data is extracted directly from Healthgrades.com and reflects the information displayed on their platform. Always verify critical data for professional use.

Can I extract providers from multiple states?

Run separate extractions for each state or use broader location terms. Each run focuses on one primary geographic area.

Is proxy usage required?

Proxies are highly recommended for reliable results, especially for larger extractions. Residential proxies provide the best success rates.

How long do extractions typically take?

Most runs complete in 1-5 minutes depending on result count and proxy performance. API-only mode ensures fast extraction.

Can I schedule automated extractions?

Yes, use Apify's scheduling features to run extractions automatically at set intervals.

---

Support

For issues or feature requests, contact support through the Apify Console.

Resources

• Apify Documentation
• API Reference
• Scheduling Runs

---

Legal Notice

This actor is designed for legitimate healthcare data collection purposes. Users are responsible for ensuring compliance with Healthgrades terms of service, HIPAA regulations, and applicable data protection laws. Use extracted data responsibly and respect provider privacy rights.

Research & Analytics

Enable data-driven healthcare insights:

• Healthcare access and availability studies
• Provider distribution analysis
• Medical specialty trends research
• Healthcare quality assessment
• Geographic healthcare disparities research

Business Intelligence

Support strategic healthcare decisions:

• Market penetration analysis
• Service area expansion planning
• Competitive landscape assessment
• Provider recruitment strategies
• Network adequacy evaluation

---

Performance & Reliability

Extraction Speed

• Listing Collection: 20-30 providers per minute (with JSON API)
• Detailed Profiles: 8-15 providers per minute (depending on concurrency)
• HTML Fallback: Automatically engages when API unavailable
• Configurable Concurrency: Balance speed vs. reliability (recommended: 3-5)

Data Quality

• Automatic Deduplication: Prevents duplicate provider records
• Field Validation: Ensures data consistency and accuracy
• Smart Extraction: JSON API preferred, HTML parsing as fallback
• Error Handling: Graceful recovery from temporary failures

Best Practices

• Use Residential Proxies - Significantly improves success rates for large extractions
• Set Reasonable Limits - Start with smaller result sets to test configuration
• Monitor Run Progress - Check logs for extraction statistics and any issues

---

Technical Details

Data Extraction Methods

The scraper employs a sophisticated multi-tier approach:

1. JSON API Priority - Attempts to extract data from internal JSON APIs for speed and reliability
2. HTML Parsing - Falls back to HTML extraction using structured selectors and JSON-LD schema
3. Smart Pagination - Automatically navigates multiple result pages until target count reached
4. Concurrent Processing - Parallel request handling for efficient large-scale extraction

Anti-Detection Features

• User-Agent Rotation - Multiple modern browser user agents
• Session Management - Intelligent cookie and session handling
• Rate Limiting - Respectful request pacing with random delays
• Proxy Support - Full integration with Apify Proxy (residential recommended)

Error Handling

• Automatic Retries - Transient failures automatically retried
• Fallback Mechanisms - Multiple extraction strategies ensure data collection
• Timeout Protection - Maximum runtime limits prevent hung processes
• Detailed Logging - Comprehensive statistics and error reporting

---

Frequently Asked Questions

---

Data Export Formats

Export your extracted data in multiple formats:

• JSON - Complete structured data with all fields
• CSV - Spreadsheet-compatible format for analysis
• Excel - Formatted workbook with data tables
• HTML - Formatted table for web viewing
• XML - Structured markup for integration

Access exports directly from the Apify platform after run completion.

---

Support & Resources

Need Help?

• Check the Apify Documentation for platform guidance
• Review input parameters and examples above
• Examine run logs for detailed execution information
• Test with small extractions before scaling up

Optimization Tips

• Start with specific geographic locations to validate configuration
• Enable residential proxies for runs exceeding 100 providers
• Monitor extraction statistics in run logs

---

Legal & Compliance

Important Usage Guidelines:

• Extracted data is publicly available on Healthgrades.com
• Ensure compliance with applicable data protection regulations
• Respect website terms of service and robots.txt directives
• Use extracted data responsibly and ethically
• Provider information may be subject to HIPAA and healthcare regulations
• Verify data accuracy before use in critical applications

This scraper extracts publicly accessible information and does not access restricted or confidential patient data.

---

Legal Notice

This actor is designed for legitimate healthcare data collection purposes. Users are responsible for ensuring compliance with Healthgrades terms of service, HIPAA regulations, and applicable data protection laws. Use extracted data responsibly and respect provider privacy rights.