# SEC Form ADV Scraper - RIA Firm & Investment Advisor Data (`jungle_synthesizer/sec-form-adv-iapd-crawler`) Actor Extract SEC-registered RIA firm records from IAPD by CRD, firm name, state, or full sweep. Includes identity, SEC file number, registrations, notice filings, brochure URLs, Form CRS URL, and office address. - **URL**: https://apify.com/jungle\_synthesizer/sec-form-adv-iapd-crawler.md - **Developed by:** [BowTiedRaccoon](https://apify.com/jungle_synthesizer) (community) - **Categories:** Business, Lead generation, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## SEC Form ADV IAPD Scraper — RIA Firm Directory Extract Registered Investment Adviser (RIA) firm records from the SEC's [Investment Adviser Public Disclosure](https://adviserinfo.sec.gov/) database. Returns firm identity, SEC file numbers, registration jurisdictions, notice filings, Form ADV Part 2A brochure URLs, Form CRS URLs, and main office address for ~17K SEC-registered RIAs plus tens of thousands of state-registered IAs. ### SEC Form ADV IAPD Scraper Features - Four query modes: by CRD, by firm name, by state, or a full US sweep - Reads SEC's official IAPD JSON API directly — no HTML parsing, no stale selectors - Returns structured firm identity plus a link to each firm's actual Form ADV Part 2A PDF - Formats registrations and notice filings into readable jurisdiction status lines - Skips inactive firms by default and honors a polite rate limit so nothing gets flagged - No proxies, no auth, no browser — just JSON over HTTPS - Pay-per-event pricing: $0.001 per firm record ### Who Uses SEC Form ADV Data? - **M&A advisors** — build lists of acquisition targets by state, registration status, or firm size proxy - **Custodians and tech vendors** — prospect RIAs with clean, structured contact data instead of PDFs - **Compliance teams** — monitor new registrations and notice filings across jurisdictions - **Financial researchers** — pull bulk RIA registration data without assembling it from the public site by hand - **Lead generation** — feed firm names, addresses, and CRD numbers into a CRM as a cold outreach list ### How SEC Form ADV IAPD Scraper Works 1. Pick a query mode. Look up specific CRDs, search by firm name, sweep one or more US states, or run the full national sweep. 2. The scraper pages through the IAPD search index and fetches each firm's detail record from the SEC's JSON API. 3. Each record is assembled into a flat row — firm identity, registration flags, jurisdiction status lines, brochure and CRS URLs, office address — and written to the dataset. ### Input #### Sweep a single state ```json { "mode": "by_state", "states": ["CA"], "maxItems": 500, "includeInactive": false } ```` #### Look up specific firms by CRD ```json { "mode": "by_crd", "crdNumbers": ["106389", "13938", "323342"], "maxItems": 10 } ``` #### Search by firm name ```json { "mode": "by_name", "firmNames": ["Fidelity", "Vanguard Personal Advisor"], "maxItems": 10 } ``` #### Full US sweep (all 50 states + DC) ```json { "mode": "full_sweep", "maxItems": 5000 } ``` #### Input Parameters | Field | Type | Default | Description | |-------|------|---------|-------------| | mode | string | `by_state` | Query mode: `by_crd`, `by_name`, `by_state`, or `full_sweep` | | crdNumbers | string\[] | `[]` | List of CRD numbers to look up directly. Required for `by_crd` mode. | | firmNames | string\[] | `[]` | List of firm-name search terms. Required for `by_name` mode. First match per term is returned. | | states | string\[] | `[]` | Two-letter US state codes (CA, NY, TX). Used with `by_state`. Defaults to `["CA"]` if left empty in that mode. | | includeInactive | boolean | `false` | Include firms whose IA registration is terminated or withdrawn. Default is active firms only. | | maxItems | integer | `100` | Max firm records to return. Keep low for quick runs, raise for bulk sweeps. | ### SEC Form ADV IAPD Scraper Output Fields #### Example firm record ```json { "crd_number": "154406", "firm_name": "AG ASSET ADVISORY, LLC", "firm_other_names": ["AG ASSET ADVISORY, LLC"], "sec_file_number": "80760", "sec_file_number_type": "801", "ia_scope": "ACTIVE", "is_ia_firm": true, "is_sec_registered": true, "is_state_registered": false, "is_era_registered": false, "has_pdf": true, "has_ia_disclosure_flag": false, "adv_filing_date": "03/05/2026", "main_office_street": "1901 AVENUE OF THE STARS, SUITE 200", "main_office_city": "LOS ANGELES", "main_office_state": "CA", "main_office_zip": "90067", "main_office_country": "United States", "registered_jurisdictions": [ "SEC: Approved (effective 2015-01-02)", "California: Terminated (effective 2015-02-06)", "Texas: Terminated (effective 2015-01-07)" ], "notice_filing_jurisdictions": [ "California: Notice Filed (effective 2015-01-02)", "New York: Notice Filed (effective 2016-03-31)", "Texas: Notice Filed (effective 2015-01-06)" ], "registered_jurisdiction_count": 3, "notice_filing_count": 3, "brochure_count": 1, "form_adv_part2_url": "https://files.adviserinfo.sec.gov/IAPD/Content/Common/crd_iapd_Brochure.aspx?BRCHR_VRSN_ID=1015166", "form_crs_url": "https://reports.adviserinfo.sec.gov/crs/crs_154406.pdf", "iapd_profile_url": "https://adviserinfo.sec.gov/firm/summary/154406" } ``` | Field | Type | Description | |-------|------|-------------| | crd\_number | string | Central Registration Depository number — the unique firm ID | | firm\_name | string | Primary firm name as registered with the SEC | | firm\_other\_names | string\[] | Alternate, DBA, and legacy firm names | | sec\_file\_number | string | SEC file number without the prefix | | sec\_file\_number\_type | string | File prefix — `801` is SEC-registered IA, `802` is SEC Exempt Reporting Adviser | | ia\_scope | string | `ACTIVE` or `INACTIVE` | | is\_ia\_firm | boolean | Whether the firm is flagged as an Investment Adviser | | is\_sec\_registered | boolean | Registered directly with the SEC as an IA | | is\_state\_registered | boolean | Registered with one or more state securities regulators as an IA | | is\_era\_registered | boolean | Filed as an Exempt Reporting Adviser (state or SEC) | | has\_pdf | boolean | Whether a Form ADV PDF is available on file | | has\_ia\_disclosure\_flag | boolean | Whether the firm has disclosure events on its IA record | | adv\_filing\_date | string | Most recent Form ADV filing date (MM/DD/YYYY) | | main\_office\_street | string | Main office street address (street1 + street2 combined) | | main\_office\_city | string | Main office city | | main\_office\_state | string | Main office state or region code | | main\_office\_zip | string | Main office postal code | | main\_office\_country | string | Main office country | | registered\_jurisdictions | string\[] | One entry per jurisdiction (SEC or state) formatted as `Jurisdiction: Status (effective YYYY-MM-DD)` | | notice\_filing\_jurisdictions | string\[] | One entry per state notice filing, same format | | registered\_jurisdiction\_count | integer | Count of jurisdictions on the registration record | | notice\_filing\_count | integer | Count of notice-filing jurisdictions | | brochure\_count | integer | Count of Form ADV Part 2 brochures on file | | form\_adv\_part2\_url | string | URL to the firm's most recent Form ADV Part 2A brochure (when available) | | form\_crs\_url | string | URL to the firm's Form CRS Customer Relationship Summary PDF (when the firm filed one) | | iapd\_profile\_url | string | URL to the firm's public IAPD profile page | ### FAQ #### How do I scrape SEC IAPD data? SEC Form ADV IAPD Scraper pulls directly from the SEC's public IAPD JSON API. Pick a mode, provide CRDs, firm names, or state codes, and the scraper returns one row per firm. No scraping of the HTML site, no browser automation. #### How much does this actor cost to run? About $0.001 per firm record plus the actor start fee. A 100-firm run costs roughly $0.20. A full state sweep of 3,000 California RIAs is around $3. A full national sweep of ~17K SEC-registered RIAs is roughly $17. #### Do I need proxies to run this? No. The SEC IAPD JSON API is public, unauthenticated, and does not rate-limit aggressively. The scraper ships with proxies disabled by default and a self-imposed ~7 requests per second pace, well under SEC's published 10/sec guideline. #### What data is and isn't included from Form ADV? The scraper returns everything the IAPD JSON API exposes — identity, registrations, notice filings, brochure and CRS URLs, and address. Form ADV Part 1 fields like AUM, client counts, fee structure, custodians, and disciplinary history live inside the Form ADV PDF and the SEC's bulk compilation zip files, which are a separate pipeline. We link out to each firm's Part 2A brochure PDF so you can download and parse those yourself if needed. #### Can I filter by state? Yes. Pass `mode: "by_state"` with a `states` array of two-letter codes. You can pass one state or many. Each state gets its own paginated sweep, so `["CA", "NY", "TX"]` is three sweeps stitched together under a single `maxItems` cap. #### Why does full\_sweep take so long? Full sweep iterates all 50 states plus DC, paginating each. California alone has over 3,000 active RIAs. At a polite rate limit with one search call plus one detail call per firm, the full national sweep is an hour-plus crawl. For bulk pulls, set a generous `maxItems` and a long timeout. ### Need More Features? Need AUM and client-count fields parsed from the Form ADV PDF, a companion IAR (individual adviser) crawler, or custom filters? [File an issue](https://console.apify.com/actors/issues) or get in touch. ### Why Use SEC Form ADV IAPD Scraper? - **Priced per record** — about $0.001 per firm, which beats paying four figures for a commercial RIA database that ships the same public data - **Structured output** — clean JSON with consistent field names and formatted jurisdiction status lines, so you skip the parsing step that usually eats half a day - **Official data source** — pulls from the SEC's own JSON API, so the data is exactly what the regulator publishes, not a third-party scrape of a scrape # Actor input Schema ## `sp_intended_usage` (type: `string`): Please describe how you plan to use the data extracted by this crawler. ## `sp_improvement_suggestions` (type: `string`): Provide any feedback or suggestions for improvements. ## `sp_contact` (type: `string`): Provide your email address so we can get in touch with you. ## `mode` (type: `string`): How to find firms. by\_crd: look up specific CRD numbers. by\_name: search by firm name. by\_state: paginate all RIA firms in a state. full\_sweep: iterate all US states (slow). ## `crdNumbers` (type: `array`): List of CRD (Central Registration Depository) numbers to look up. Required when mode is by\_crd. Example: \["106389", "13938"] ## `firmNames` (type: `array`): List of firm-name search terms. Required when mode is by\_name. Fuzzy-matches the search index; first match per term is used. ## `states` (type: `array`): Two-letter US state codes (e.g. CA, NY, TX). Used with by\_state mode. If empty in by\_state mode, defaults to CA. ## `includeInactive` (type: `boolean`): If true, also return firms with inactive IA scope (terminated or withdrawn). Default false: active firms only. ## `maxItems` (type: `integer`): Maximum number of firm records to return. Set low (10-50) for quick tests, higher (500+) for production sweeps. ## `proxyConfiguration` (type: `object`): Proxy settings. The SEC IAPD API does not require proxies. Leave as default. ## Actor input object example ```json { "sp_intended_usage": "Describe your intended use...", "sp_improvement_suggestions": "Share your suggestions here...", "sp_contact": "Share your email here...", "mode": "by_state", "includeInactive": false, "maxItems": 100, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "sp_intended_usage": "Describe your intended use...", "sp_improvement_suggestions": "Share your suggestions here...", "sp_contact": "Share your email here...", "mode": "by_state", "includeInactive": false, "maxItems": 100, "proxyConfiguration": { "useApifyProxy": false } }; // Run the Actor and wait for it to finish const run = await client.actor("jungle_synthesizer/sec-form-adv-iapd-crawler").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "sp_intended_usage": "Describe your intended use...", "sp_improvement_suggestions": "Share your suggestions here...", "sp_contact": "Share your email here...", "mode": "by_state", "includeInactive": False, "maxItems": 100, "proxyConfiguration": { "useApifyProxy": False }, } # Run the Actor and wait for it to finish run = client.actor("jungle_synthesizer/sec-form-adv-iapd-crawler").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "sp_intended_usage": "Describe your intended use...", "sp_improvement_suggestions": "Share your suggestions here...", "sp_contact": "Share your email here...", "mode": "by_state", "includeInactive": false, "maxItems": 100, "proxyConfiguration": { "useApifyProxy": false } }' | apify call jungle_synthesizer/sec-form-adv-iapd-crawler --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=jungle_synthesizer/sec-form-adv-iapd-crawler", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "SEC Form ADV Scraper - RIA Firm & Investment Advisor Data", "description": "Extract SEC-registered RIA firm records from IAPD by CRD, firm name, state, or full sweep. Includes identity, SEC file number, registrations, notice filings, brochure URLs, Form CRS URL, and office address.", "version": "1.0", "x-build-id": "8OaPgTpnWTCEEMFRC" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/jungle_synthesizer~sec-form-adv-iapd-crawler/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-jungle_synthesizer-sec-form-adv-iapd-crawler", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/jungle_synthesizer~sec-form-adv-iapd-crawler/runs": { "post": { "operationId": "runs-sync-jungle_synthesizer-sec-form-adv-iapd-crawler", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/jungle_synthesizer~sec-form-adv-iapd-crawler/run-sync": { "post": { "operationId": "run-sync-jungle_synthesizer-sec-form-adv-iapd-crawler", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "sp_intended_usage", "sp_improvement_suggestions" ], "properties": { "sp_intended_usage": { "title": "What is the intended usage of this data?", "minLength": 1, "type": "string", "description": "Please describe how you plan to use the data extracted by this crawler." }, "sp_improvement_suggestions": { "title": "How can we improve this crawler for you?", "minLength": 1, "type": "string", "description": "Provide any feedback or suggestions for improvements." }, "sp_contact": { "title": "Contact Email", "minLength": 1, "type": "string", "description": "Provide your email address so we can get in touch with you." }, "mode": { "title": "Query Mode", "enum": [ "by_crd", "by_name", "by_state", "full_sweep" ], "type": "string", "description": "How to find firms. by_crd: look up specific CRD numbers. by_name: search by firm name. by_state: paginate all RIA firms in a state. full_sweep: iterate all US states (slow).", "default": "by_state" }, "crdNumbers": { "title": "CRD Numbers", "type": "array", "description": "List of CRD (Central Registration Depository) numbers to look up. Required when mode is by_crd. Example: [\"106389\", \"13938\"]", "items": { "type": "string" } }, "firmNames": { "title": "Firm Names", "type": "array", "description": "List of firm-name search terms. Required when mode is by_name. Fuzzy-matches the search index; first match per term is used.", "items": { "type": "string" } }, "states": { "title": "States", "type": "array", "description": "Two-letter US state codes (e.g. CA, NY, TX). Used with by_state mode. If empty in by_state mode, defaults to CA.", "items": { "type": "string" } }, "includeInactive": { "title": "Include Inactive Firms", "type": "boolean", "description": "If true, also return firms with inactive IA scope (terminated or withdrawn). Default false: active firms only.", "default": false }, "maxItems": { "title": "Max Items", "type": "integer", "description": "Maximum number of firm records to return. Set low (10-50) for quick tests, higher (500+) for production sweeps.", "default": 100 }, "proxyConfiguration": { "title": "Proxy Configuration", "type": "object", "description": "Proxy settings. The SEC IAPD API does not require proxies. Leave as default." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```