Healthgrades Scraper

Healthgrades Doctor Scraper

Scrapes doctor profiles from Healthgrades search results, extracting detailed information including name, specialty, rating, address, phone, and more.

Features

• Targeted Search: Search by medical specialty and location
• Rich Data Extraction: Extracts doctor name, specialty, rating, address, phone, telehealth availability, and biography
• Intelligent Pagination: Automatically scrolls to load more results when needed
• Robust Error Handling: Handles privacy banners and network issues gracefully
• Configurable Limits: Set maximum number of doctors to extract

Input Parameters

• specialty (required): Medical specialty to search for (e.g., "Family Medicine", "Internal Medicine", "Cardiology")
• location (required): Location to search in (e.g., "New York, NY", "Los Angeles, CA")
• max_items (optional): Maximum number of doctors to extract (default: 50, max: 500)

Output Fields

Each doctor record includes:

• name: Doctor's full name and credentials
• profileUrl: Direct link to doctor's Healthgrades profile
• specialty: Medical specialty
• rating: Patient rating score
• address: Practice address
• phone: Contact phone number
• offersTelehealth: Whether telehealth is available
• biography: Professional biography snippet
• scrapedAt: Timestamp when data was extracted

Resources

• Playwright for web scraping in 2023
• Scraping single-page applications with Playwright
• How to scale Puppeteer and Playwright
• Integration with Zapier, Make, GitHub, Google Drive and other apps
• Video guide on getting data using Apify API
• A short guide on how to build web scrapers using code templates:

Getting started

For complete information see this article. To run the Actor use the following command:

$apify run

Deploy to Apify

Connect Git repository to Apify

If you've created a Git repository for the project, you can easily connect to Apify:

1. Go to Actor creation page
2. Click on Link Git Repository button

Push project on your local machine to Apify

You can also deploy the project on your local machine to Apify without the need for the Git repository.

1. Log in to Apify. You will need to provide your Apify API Token to complete this action.

   $apify login
   

2. Deploy your Actor. This command will deploy and build the Actor on the Apify Platform. You can find your newly created Actor under Actors -> My Actors.

   $apify push
   

Documentation reference

To learn more about Apify and Actors, take a look at the following resources:

• Apify SDK for JavaScript documentation
• Apify SDK for Python documentation
• Apify Platform documentation
• Join our developer community on Discord