# SEC Investment Adviser Scraper (`datasiphon/sec-adviser-scraper`) Actor

Downloads the SEC's monthly Form ADV bulk data for registered investment advisers and exempt reporting advisers - where most VC and PE funds file. Free, authoritative, no auth.

- **URL**: https://apify.com/datasiphon/sec-adviser-scraper.md
- **Developed by:** [Kashif Ali](https://apify.com/datasiphon) (community)
- **Categories:** Lead generation, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## SEC Investment Adviser Scraper

Investor lead lists — VC firms, PE firms, hedge funds — straight from the SEC's
monthly Form ADV bulk data.

Every US investment adviser files Form ADV. Firms managing private funds below
the registration threshold file as **exempt reporting advisers (ERAs)**, which
is where most venture capital and private equity firms sit. The SEC publishes
both rosters as a monthly ZIP. This actor resolves the newest file, unzips it
and emits clean records. No browser, no login, no paid database.

### What you get

Filtering the exempt roster to firms running at least one VC fund returns
**2,697 active firms** in about three seconds — with phone numbers, websites,
fund counts and reported assets.

| Field | Notes |
|---|---|
| `firmName`, `legalName` | Primary business name, falling back to legal name |
| `crdNumber`, `secNumber` | Regulatory identifiers, 100% populated |
| `adviserType`, `firmType`, `status` | e.g. `exempt`, `ERA`, `ERA - Active` |
| `address`, `city`, `state`, `country`, `postalCode` | Main office |
| `phone` | Main office telephone, digits only |
| `website` | Firm website |
| `vcFunds`, `peFunds`, `hedgeFunds`, `realEstateFunds`, `otherFunds` | Number of funds of each kind |
| `privateFundCount` | Total private funds |
| `totalGrossAssets` | Gross assets across private funds, in whole dollars |
| `latestFilingDate` | When the firm last filed |
| `sourceFile`, `scrapedAt` | Exact SEC ZIP and run timestamp |

With `includePeople` on, every row also carries one named person:

| Field | Notes |
|---|---|
| `personName` | Direct owner or executive officer, from Form ADV Schedule A |
| `title` | Their role, e.g. `MANAGING MEMBER`; multiple roles joined with `/` |
| `isControlPerson`, `ownershipCode` | Schedule A control flag and ownership band (`A`-`E`) |
| `peopleAsOfFiling` | The Form ADV filing the person was read from |
| `emailGuess`, `emailCandidates`, `emailPattern` | Pattern guesses against the firm's own domain |
| `emailVerified` | Always `false` - see below |

Measured on the VC-filtered exempt roster: firm name, CRD, fund counts and
filing date 100%; phone 99.7%; reported assets 99.0%; website 89.2%.

#### Named people: 80% of VC firms, 3 per firm

The monthly roster carries firms only. People come from the SEC's *complete*
Form ADV archive (702 MB), where Schedule A lists each firm's direct owners and
executive officers. Measured on the VC-filtered exempt roster: **2,171 of 2,697
firms (80.5%) have named people**, for **6,571 person rows** — three per firm on
average, with duplicates collapsed when the same person is listed under several
roles.

The remaining 20% is the archive's cutoff, not a parsing gap: it covers filings
through **2024-12-31**, so a firm that first registered in 2025 or 2026 has no
Schedule A rows yet. `peopleOnly: true` drops those firms.

The first run with `includePeople` downloads the archive and builds the index —
about 12 minutes. The index is cached in a named key-value store, so every later
run reuses it in seconds.

#### Emails are guesses, and say so

Form ADV carries no email column, and there is no free source of verified
adviser emails. So the actor derives candidates from the firm's own website
domain (`hernan.kazah@kaszek.com`, then `hkazah@`, `hernan@`, ...) and flags
every one with `emailVerified: false`. **70.4%** of person rows get a guess; the
rest have no website on file or a name too short to split. Verifying them needs
SMTP probing or a paid validation API — deliberately out of scope.

#### Location coverage is partial, by design

`state` is populated for 63% of firms. That is not a parsing gap: 497 of the
blanks are non-US firms (Israel, United Kingdom, Cayman Islands, Singapore,
China) which have no US state, and around 500 report no main-office location at
all. A `states` filter therefore excludes every foreign firm — which is usually
what you want, but worth knowing before you use it.

### Input

| Field | Type | Default | Description |
|---|---|---|---|
| `adviserTypes` | array | `["exempt"]` | `exempt` (~6,600 firms, where VC/PE file) or `registered` (~15,000) |
| `fundTypes` | array | `[]` | Keep firms running at least one `vc`, `pe`, `hedge`, `realEstate` or `other` fund |
| `states` | array | `[]` | Two-letter main-office state codes |
| `minGrossAssets` | integer | `0` | Drop firms below this gross private-fund asset figure |
| `activeOnly` | boolean | `true` | Keep only active SEC registrations |
| `includePeople` | boolean | `false` | Attach Schedule A owners and officers, one row per person |
| `peopleOnly` | boolean | `false` | Drop firms with no named people (needs `includePeople`) |
| `maxResults` | integer | `50000` | Stop after this many rows |

```json
{
  "adviserTypes": ["exempt"],
  "fundTypes": ["vc"],
  "states": ["CA", "NY", "MA"],
  "minGrossAssets": 50000000
}
```

### Running it

```bash
pip install -r requirements.txt
python src/main.py

python test_parse.py     # discovery, amount parsing and record mapping
python test_people.py    # Schedule A extraction and email inference

apify push
apify call datasiphon/sec-adviser-scraper --input-file=input.json
```

### Three things that will bite you if you fork this

**SEC fingerprints HTTP clients.** A plain request gets `403 Forbidden` from any
IP, with or without a declared User-Agent, and a residential proxy alone is not
enough — it is the TLS handshake that gives the client away. This actor uses
`impit`, which impersonates Chrome down to that handshake, *and* routes through
a residential proxy. Both halves are load-bearing on Apify: from a platform IP
even impit gets a flat `403` on every request. Run the same code on a laptop
and impit alone is enough, which is how you debug parsing bugs for free instead
of $6 a run.

**The complete archive names its columns by Form ADV item code.** `ERA_ADV_Base`
has no column called CRD: item `1E1` is the CRD number and `1D` is the SEC file
number (`801-`/`802-`). Matching on the word "CRD" silently found nothing, which
produced an empty people index and a run that pushed zero rows. `DateSubmitted`
also has to be parsed before it is compared — as strings, `11/13/2012` beats
`3/01/2024` and every firm gets its oldest filing.

**Amounts carry cents.** Form ADV writes gross assets as `78,960,522.00`.
Stripping every non-digit turns that into 7,896,052,200 — a hundredfold
inflation that makes the median VC firm look like an $11bn fund. `to_number`
keeps the decimal point.

### Licence

MIT — built for KeystoneRev Outbound OS.

# Actor input Schema

## `adviserTypes` (type: `array`):

'exempt' = exempt reporting advisers, where most VC and PE funds file (~6,600 firms). 'registered' = SEC-registered investment advisers (~15,000).

## `fundTypes` (type: `array`):

Keep firms reporting at least one fund of these kinds on Form ADV. Leave empty for all advisers.

## `states` (type: `array`):

Two-letter state codes, e.g. CA, NY, MA. Case-insensitive. Leave empty for all.

## `minGrossAssets` (type: `integer`):

Drop firms below this total gross asset figure. 0 keeps everything, including firms that report no assets.

## `activeOnly` (type: `boolean`):

Keep only firms whose SEC status is active. Turn off to include terminated and pending registrations.

## `maxResults` (type: `integer`):

Stop after this many firms in total, across the selected adviser types.

## `proxyConfiguration` (type: `object`):

SEC blocks datacenter IPs, so a residential proxy is used by default.

## `includePeople` (type: `boolean`):

Attach Form ADV Schedule A owners and executive officers, emitting one row per person with a guessed email. The first run downloads a ~700MB SEC archive (about 12 minutes) and caches the result; later runs reuse it.

## `peopleOnly` (type: `boolean`):

Drop firms that have no Schedule A individuals. Requires 'Include named people'.

## Actor input object example

```json
{
  "adviserTypes": [
    "exempt"
  ],
  "fundTypes": [
    "vc"
  ],
  "minGrossAssets": 0,
  "activeOnly": true,
  "maxResults": 50000,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  },
  "includePeople": false,
  "peopleOnly": false
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "adviserTypes": [
        "exempt"
    ],
    "fundTypes": [
        "vc"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("datasiphon/sec-adviser-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "adviserTypes": ["exempt"],
    "fundTypes": ["vc"],
}

# Run the Actor and wait for it to finish
run = client.actor("datasiphon/sec-adviser-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "adviserTypes": [
    "exempt"
  ],
  "fundTypes": [
    "vc"
  ]
}' |
apify call datasiphon/sec-adviser-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,datasiphon/sec-adviser-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/SC9X8h7YeYQ8Peb5s/builds/vB9fEC3M0e3aeIlV0/openapi.json