SEC Investment Adviser Scraper

Investor lead lists — VC firms, PE firms, hedge funds — straight from the SEC's monthly Form ADV bulk data.

Every US investment adviser files Form ADV. Firms managing private funds below the registration threshold file as exempt reporting advisers (ERAs), which is where most venture capital and private equity firms sit. The SEC publishes both rosters as a monthly ZIP. This actor resolves the newest file, unzips it and emits clean records. No browser, no login, no paid database.

What you get

Filtering the exempt roster to firms running at least one VC fund returns 2,697 active firms in about three seconds — with phone numbers, websites, fund counts and reported assets.

Field	Notes
firmName, legalName	Primary business name, falling back to legal name
crdNumber, secNumber	Regulatory identifiers, 100% populated
adviserType, firmType, status	e.g. exempt, ERA, ERA - Active
address, city, state, country, postalCode	Main office
phone	Main office telephone, digits only
website	Firm website
vcFunds, peFunds, hedgeFunds, realEstateFunds, otherFunds	Number of funds of each kind
privateFundCount	Total private funds
totalGrossAssets	Gross assets across private funds, in whole dollars
latestFilingDate	When the firm last filed
sourceFile, scrapedAt	Exact SEC ZIP and run timestamp

With includePeople on, every row also carries one named person:

Field	Notes
personName	Direct owner or executive officer, from Form ADV Schedule A
title	Their role, e.g. MANAGING MEMBER; multiple roles joined with /
isControlPerson, ownershipCode	Schedule A control flag and ownership band (A-E)
peopleAsOfFiling	The Form ADV filing the person was read from
emailGuess, emailCandidates, emailPattern	Pattern guesses against the firm's own domain
emailVerified	Always false - see below

Measured on the VC-filtered exempt roster: firm name, CRD, fund counts and filing date 100%; phone 99.7%; reported assets 99.0%; website 89.2%.

Named people: 80% of VC firms, 3 per firm

The monthly roster carries firms only. People come from the SEC's complete Form ADV archive (702 MB), where Schedule A lists each firm's direct owners and executive officers. Measured on the VC-filtered exempt roster: 2,171 of 2,697 firms (80.5%) have named people, for 6,571 person rows — three per firm on average, with duplicates collapsed when the same person is listed under several roles.

The remaining 20% is the archive's cutoff, not a parsing gap: it covers filings through 2024-12-31, so a firm that first registered in 2025 or 2026 has no Schedule A rows yet. peopleOnly: true drops those firms.

The first run with includePeople downloads the archive and builds the index — about 12 minutes. The index is cached in a named key-value store, so every later run reuses it in seconds.

Emails are guesses, and say so

Form ADV carries no email column, and there is no free source of verified adviser emails. So the actor derives candidates from the firm's own website domain (hernan.kazah@kaszek.com, then hkazah@, hernan@, ...) and flags every one with emailVerified: false. 70.4% of person rows get a guess; the rest have no website on file or a name too short to split. Verifying them needs SMTP probing or a paid validation API — deliberately out of scope.

Location coverage is partial, by design

state is populated for 63% of firms. That is not a parsing gap: 497 of the blanks are non-US firms (Israel, United Kingdom, Cayman Islands, Singapore, China) which have no US state, and around 500 report no main-office location at all. A states filter therefore excludes every foreign firm — which is usually what you want, but worth knowing before you use it.

Input

Field	Type	Default	Description
adviserTypes	array	["exempt"]	exempt (~6,600 firms, where VC/PE file) or registered (~15,000)
fundTypes	array	[]	Keep firms running at least one vc, pe, hedge, realEstate or other fund
states	array	[]	Two-letter main-office state codes
minGrossAssets	integer	0	Drop firms below this gross private-fund asset figure
activeOnly	boolean	true	Keep only active SEC registrations
includePeople	boolean	false	Attach Schedule A owners and officers, one row per person
peopleOnly	boolean	false	Drop firms with no named people (needs includePeople)
maxResults	integer	50000	Stop after this many rows

{
  "adviserTypes": ["exempt"],
  "fundTypes": ["vc"],
  "states": ["CA", "NY", "MA"],
  "minGrossAssets": 50000000
}

Running it

pip install -r requirements.txt
python src/main.py

python test_parse.py     # discovery, amount parsing and record mapping
python test_people.py    # Schedule A extraction and email inference

apify push
apify call datasiphon/sec-adviser-scraper --input-file=input.json

Three things that will bite you if you fork this

SEC fingerprints HTTP clients. A plain request gets 403 Forbidden from any IP, with or without a declared User-Agent, and a residential proxy alone is not enough — it is the TLS handshake that gives the client away. This actor uses impit, which impersonates Chrome down to that handshake, and routes through a residential proxy. Both halves are load-bearing on Apify: from a platform IP even impit gets a flat 403 on every request. Run the same code on a laptop and impit alone is enough, which is how you debug parsing bugs for free instead of $6 a run.

The complete archive names its columns by Form ADV item code. ERA_ADV_Base has no column called CRD: item 1E1 is the CRD number and 1D is the SEC file number (801-/802-). Matching on the word "CRD" silently found nothing, which produced an empty people index and a run that pushed zero rows. DateSubmitted also has to be parsed before it is compared — as strings, 11/13/2012 beats 3/01/2024 and every firm gets its oldest filing.

Amounts carry cents. Form ADV writes gross assets as 78,960,522.00. Stripping every non-digit turns that into 7,896,052,200 — a hundredfold inflation that makes the median VC firm look like an $11bn fund. to_number keeps the decimal point.

Licence

MIT — built for KeystoneRev Outbound OS.