# Psychology Today Scraper | Therapist & Psychiatrist Leads (`crawlerbros/psychology-today-scraper`) Actor Scrape Psychology Today's therapist, psychiatrist, group practice, and treatment-rehab directories by US state. Returns name, credentials, specialties, insurance, address, and contact info per professional. No login required. - **URL**: https://apify.com/crawlerbros/psychology-today-scraper.md - **Developed by:** [Crawler Bros](https://apify.com/crawlerbros) (community) - **Categories:** Lead generation, Automation, Developer tools - **Stats:** 1 total users, 0 monthly users, 100.0% runs succeeded, 9 bookmarks - **User rating**: 5.00 out of 5 stars ## Pricing from $1.00 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Psychology Today Scraper Extract structured records of therapists, psychiatrists, group practices, and rehab centers from the Psychology Today professional directory — across all 50 US states plus DC. ### What this actor does Psychology Today operates the largest publicly searchable directory of mental-health providers in the United States. This actor walks that directory by category and by state, returning one clean record per professional, with all the information a user would see on the provider's listing and profile pages. You choose one of four categories — **therapists**, **psychiatrists**, **groups**, or **treatment-rehab** — and one or more US states. In `LIST_ONLY` mode the actor extracts everything visible on the listing pages (name, credentials, photo, short description, city, state) and is the fastest option for enumerating a state. In `LIST_AND_DETAILS` mode it additionally opens each provider's profile page and enriches the record with phone, website, specialties, treatment approaches, insurances accepted, languages, client focus, years in practice, gender, and a full street-level address. The result is a ready-to-use dataset for lead generation, market research, referral directories, or any workflow that needs a structured snapshot of the mental-health provider landscape in a given region. ### Key features - Four directory categories: `therapists`, `psychiatrists`, `groups`, `treatment-rehab` - All 50 US states plus Washington, DC - Two scrape modes: `LIST_ONLY` (fast, listing-level) and `LIST_AND_DETAILS` (full profile) - Fine-grained scope controls: `maxItems` global cap and `maxPagesPerState` per-state cap - Configurable polite delay window (`minDelayMs` / `maxDelayMs`) between requests - Transparent residential-proxy fallback — direct requests first, proxy only when blocked - No login, no cookies, no API key required - Zero-null output — empty fields are omitted from each record ### Input | Field | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `category` | enum | yes | `therapists` | Directory to crawl. One of `therapists`, `psychiatrists`, `groups`, `treatment-rehab`. | | `states` | string[] | no | `["new-york"]` | State slugs to crawl (50 US states + DC). Example: `"california"`, `"new-york"`, `"district-of-columbia"`. | | `scrapeMode` | enum | no | `LIST_ONLY` | `LIST_ONLY` for listing-level fields only; `LIST_AND_DETAILS` to also open each profile. | | `maxItems` | integer | no | `10` | Global cap on records emitted across all selected states. | | `maxPagesPerState` | integer | no | `5` | Number of listing pages to walk per state (~20 profiles per page). | | `minDelayMs` | integer | no | `150` | Lower bound (milliseconds) of the random delay between requests. | | `maxDelayMs` | integer | no | `300` | Upper bound (milliseconds) of the random delay between requests. | | `autoProxyFallback` | boolean | no | `true` | Automatically retry through Apify residential proxy when a page looks blocked. | **Example input** ```json { "category": "therapists", "states": ["new-york", "california", "texas"], "scrapeMode": "LIST_AND_DETAILS", "maxItems": 200, "maxPagesPerState": 10, "minDelayMs": 200, "maxDelayMs": 500, "autoProxyFallback": true } ```` ### Output One record per provider. Fields without data are omitted so every key has a real value. ```json { "profileUrl": "https://www.psychologytoday.com/us/therapists/jane-doe-new-york-ny/111111", "professionalName": "Jane Doe", "credentials": "LCSW", "profileId": "111111", "shortDescription": "Compassionate therapist helping adults...", "longDescription": "Welcome! I am a licensed clinical social worker...", "phone": "+12125551234", "website": "https://janedoetherapy.com", "imageUrl": "https://cdn.psychologytoday.com/profile-photos/111111.jpg", "primaryAddress": { "street": "123 W 45th St", "city": "New York", "state": "NY", "zip": "10036" }, "specialties": ["Anxiety", "Depression", "Trauma and PTSD"], "expertise": ["Grief", "Relationship Issues"], "treatmentApproaches": ["Cognitive Behavioral (CBT)", "EMDR"], "insuranceAccepted": ["Aetna", "BlueCross BlueShield", "Cigna"], "clientFocus": ["Adults", "Elders (65+)"], "languages": ["English", "Spanish"], "gender": "Female", "yearsInPractice": 12, "category": "therapists", "stateSlug": "new-york", "scrapedAt": "2026-04-24T12:34:56+00:00" } ``` **Field descriptions** - **`profileUrl`** — canonical Psychology Today profile URL - **`professionalName`** / **`credentials`** — name and credential letters (e.g. LCSW, PhD, MD) - **`profileId`** — stable numeric ID used by Psychology Today - **`shortDescription`** / **`longDescription`** — the listing card blurb and the full profile bio - **`phone`** / **`website`** / **`imageUrl`** — contact details and provider photo (detail mode). For most modern profiles `website` is Psychology Today's interstitial redirect URL (e.g. `https://www.psychologytoday.com/us/profile/{id}/website`) — clicking it in a browser bounces the user to the therapist's external site. - **`emails`** — array of contact emails when the therapist publishes a `mailto:` link directly on their Psychology Today profile or on an externally-linked website that resolves over HTTP without JavaScript (detail mode). Often absent — Psychology Today increasingly hides external website URLs behind a JavaScript-only interstitial that this HTTP-only scraper cannot crack without a full browser. - **`primaryAddress`** — structured address with `street`, `city`, `state`, `zip` (detail mode) - **`specialties`**, **`expertise`**, **`treatmentApproaches`**, **`insuranceAccepted`**, **`clientFocus`**, **`languages`** — taxonomy arrays (detail mode) - **`gender`**, **`yearsInPractice`** — provider demographics (detail mode) - **`category`** / **`stateSlug`** — echo of the input, so each record is self-describing - **`scrapedAt`** — ISO-8601 timestamp of this run ### Use cases - **Lead generation** — build targeted lists of independent practitioners for CRM, EHR, or billing products - **Referral network building** — seed a clinic's referral database with local provider coverage by specialty - **Market research** — map provider density by state, insurance coverage, and specialty mix - **Academic / public-health research** — study therapist distribution, wait-list availability, and access to care - **Directory and aggregator sites** — populate a white-label directory with verified, normalised provider records ### FAQ **Do I need a login or an API key?** No. Psychology Today's directory is publicly accessible. The actor is login-free and cookie-free. **Do I need a proxy?** Not usually. Runs start on Apify's direct egress; the `autoProxyFallback` toggle (on by default) only provisions residential proxy traffic when a direct fetch looks blocked — so you only pay for proxy on pages that need it. **What's the difference between `LIST_ONLY` and `LIST_AND_DETAILS`?** `LIST_ONLY` returns everything visible on the state listing page — fast, and plenty for name / city / specialty counts. `LIST_AND_DETAILS` additionally opens each profile and adds phone, website, full address, specialties, treatment approaches, insurance accepted, languages, client focus, gender, and years in practice. Expect detail-mode runs to take roughly twice as long per profile. **Which categories are supported?** Four: `therapists`, `psychiatrists`, `groups` (group practices), and `treatment-rehab` (inpatient and outpatient treatment centers). **Which states are supported?** All 50 US states plus Washington, DC. Pass the directory slug (e.g. `new-york`, `california`, `district-of-columbia`). **How many results can I expect per state?** Psychology Today shows about 20 profiles per listing page. Large states (California, New York, Texas) have tens of thousands of profiles across hundreds of pages; smaller states may only have a few hundred. Use `maxPagesPerState` and `maxItems` to control scope. **Can I filter by city, specialty, or insurance?** The current version scopes by category and state. City-, specialty-, and insurance-level filters are on the roadmap. **Why is the email field missing?** Psychology Today does not expose therapist email addresses on public profile pages. The actor only populates an email when an explicit `mailto:` link is present, which is uncommon. **Is this compliant with Psychology Today's terms?** The actor reads only publicly accessible directory pages, with a configurable polite delay between requests to avoid hammering. Responsibility for downstream use of the data rests with the caller — please respect Psychology Today's terms and applicable regulations such as CAN-SPAM, TCPA, and GDPR. ### Known limitations - **US-only directory.** Psychology Today also operates country-specific directories (Canada, UK, Australia) — those are not covered by this actor. - **No city-level filter (yet).** Scope is category + state; narrow the results downstream by filtering on `primaryAddress.city` or `shortDescription`. - **Email addresses are usually absent** because Psychology Today does not publish therapist emails on their profile pages. - **Detail mode is roughly 2x slower** than list-only mode because it fetches each profile page individually (with the configured polite delay). - **Very large states** can produce hundreds of pages of results. Set `maxPagesPerState` and `maxItems` generously for exhaustive crawls, or slice by region in post-processing. # Actor input Schema ## `category` (type: `string`): Which Psychology Today directory to crawl. 'therapists' for individual mental-health clinicians, 'psychiatrists' for prescribing doctors, 'groups' for group practices, 'treatment-rehab' for residential / rehab centers. ## `states` (type: `array`): US states to crawl (directory slugs). Leave empty to default to New York. Each state is fetched independently; results are deduped per state. ## `scrapeMode` (type: `string`): LIST\_ONLY returns only listing-page fields (fast). LIST\_AND\_DETAILS additionally fetches each profile page for full bio, phone, specialties, insurance, approaches, and full address. ## `maxItems` (type: `integer`): Total cap across all selected states. The run stops pushing profiles once this many have been emitted. ## `maxPagesPerState` (type: `integer`): Per-state pagination cap. Psychology Today shows ~20 profiles per listing page; set higher to crawl deeper directories. ## `minDelayMs` (type: `integer`): Lower bound of the per-request random delay. Keep reasonable to stay polite and avoid blocks. ## `maxDelayMs` (type: `integer`): Upper bound of the per-request random delay. Each HTTP request waits a uniform random duration in \[minDelayMs, maxDelayMs] before firing. ## `autoProxyFallback` (type: `boolean`): When a listing fetch returns an empty / block-page response, automatically retry that one request via Apify residential proxy. Only blocked pages burn proxy credits. Turn off to guarantee zero proxy spend. ## Actor input object example ```json { "category": "therapists", "states": [ "new-york" ], "scrapeMode": "LIST_ONLY", "maxItems": 10, "maxPagesPerState": 5, "minDelayMs": 800, "maxDelayMs": 1500, "autoProxyFallback": true } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "category": "therapists", "states": [ "new-york" ], "scrapeMode": "LIST_ONLY", "maxItems": 10, "maxPagesPerState": 5, "minDelayMs": 800, "maxDelayMs": 1500 }; // Run the Actor and wait for it to finish const run = await client.actor("crawlerbros/psychology-today-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "category": "therapists", "states": ["new-york"], "scrapeMode": "LIST_ONLY", "maxItems": 10, "maxPagesPerState": 5, "minDelayMs": 800, "maxDelayMs": 1500, } # Run the Actor and wait for it to finish run = client.actor("crawlerbros/psychology-today-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "category": "therapists", "states": [ "new-york" ], "scrapeMode": "LIST_ONLY", "maxItems": 10, "maxPagesPerState": 5, "minDelayMs": 800, "maxDelayMs": 1500 }' | apify call crawlerbros/psychology-today-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=crawlerbros/psychology-today-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Psychology Today Scraper | Therapist & Psychiatrist Leads", "description": "Scrape Psychology Today's therapist, psychiatrist, group practice, and treatment-rehab directories by US state. Returns name, credentials, specialties, insurance, address, and contact info per professional. No login required.", "version": "1.0", "x-build-id": "bJjB0L6XtDRxVjj3E" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/crawlerbros~psychology-today-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-crawlerbros-psychology-today-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/crawlerbros~psychology-today-scraper/runs": { "post": { "operationId": "runs-sync-crawlerbros-psychology-today-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/crawlerbros~psychology-today-scraper/run-sync": { "post": { "operationId": "run-sync-crawlerbros-psychology-today-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "category" ], "properties": { "category": { "title": "Directory category", "enum": [ "therapists", "psychiatrists", "groups", "treatment-rehab" ], "type": "string", "description": "Which Psychology Today directory to crawl. 'therapists' for individual mental-health clinicians, 'psychiatrists' for prescribing doctors, 'groups' for group practices, 'treatment-rehab' for residential / rehab centers.", "default": "therapists" }, "states": { "title": "US states", "uniqueItems": true, "type": "array", "description": "US states to crawl (directory slugs). Leave empty to default to New York. Each state is fetched independently; results are deduped per state.", "items": { "type": "string", "enum": [ "alabama", "alaska", "arizona", "arkansas", "california", "colorado", "connecticut", "delaware", "district-of-columbia", "florida", "georgia", "hawaii", "idaho", "illinois", "indiana", "iowa", "kansas", "kentucky", "louisiana", "maine", "maryland", "massachusetts", "michigan", "minnesota", "mississippi", "missouri", "montana", "nebraska", "nevada", "new-hampshire", "new-jersey", "new-mexico", "new-york", "north-carolina", "north-dakota", "ohio", "oklahoma", "oregon", "pennsylvania", "rhode-island", "south-carolina", "south-dakota", "tennessee", "texas", "utah", "vermont", "virginia", "washington", "west-virginia", "wisconsin", "wyoming" ] }, "default": [ "new-york" ] }, "scrapeMode": { "title": "Scrape mode", "enum": [ "LIST_ONLY", "LIST_AND_DETAILS" ], "type": "string", "description": "LIST_ONLY returns only listing-page fields (fast). LIST_AND_DETAILS additionally fetches each profile page for full bio, phone, specialties, insurance, approaches, and full address.", "default": "LIST_ONLY" }, "maxItems": { "title": "Maximum profiles to return", "minimum": 1, "maximum": 5000, "type": "integer", "description": "Total cap across all selected states. The run stops pushing profiles once this many have been emitted.", "default": 10 }, "maxPagesPerState": { "title": "Maximum listing pages per state", "minimum": 1, "maximum": 100, "type": "integer", "description": "Per-state pagination cap. Psychology Today shows ~20 profiles per listing page; set higher to crawl deeper directories.", "default": 5 }, "minDelayMs": { "title": "Minimum delay between requests (ms)", "minimum": 0, "maximum": 10000, "type": "integer", "description": "Lower bound of the per-request random delay. Keep reasonable to stay polite and avoid blocks.", "default": 800 }, "maxDelayMs": { "title": "Maximum delay between requests (ms)", "minimum": 0, "maximum": 20000, "type": "integer", "description": "Upper bound of the per-request random delay. Each HTTP request waits a uniform random duration in [minDelayMs, maxDelayMs] before firing.", "default": 1500 }, "autoProxyFallback": { "title": "Auto-retry via proxy when blocked", "type": "boolean", "description": "When a listing fetch returns an empty / block-page response, automatically retry that one request via Apify residential proxy. Only blocked pages burn proxy credits. Turn off to guarantee zero proxy spend.", "default": true } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```