Y Combinator Directory Scraper | Fast & Reliable | $4.5 / 1K

Y Combinator Directory Scraper

A lightweight scraper that crawls the Y Combinator startup directory and extracts structured company data in a flat, tabular form.

Features • Infinite-scroll crawling Automatically scrolls through the directory pages to discover and enqueue every company link. • Rich company profile extraction Captures key details such as batch, founder info, social links, job postings, descriptions, tags and more. • Configurable concurrency & limits Control how many companies to fetch and how many pages to process in parallel. • Flat output schema Produces fully flattened records so they’re ready for CSV, Excel or database ingestion.

Data Scopes

Each output record includes: Field Description batch YC batch name (e.g. “Winter 2024”) company_id Internal YC company ID company_image URL to company logo company_linkedin Company LinkedIn URL company_location Primary location (city, region) company_name Official company name company_x Company Twitter/X URL founders/0–3/{id,name,linkedin,x} Up to four founders’ IDs, names & socials is_hiring true if any open job postings number_of_open_jobs Count of open job listings short_description One-liner or tagline long_description Full “about us” description status Active, Alumni, etc. tags Array of industry or vertical tags tags/0–5 Individual tag entries (up to six) team_size Number of employees primary_partner URL to the YC partner or lead investor url Directory page URL website Company’s own website year_founded Year the company was founded

Example Input

This scraper expects a simple JSON object to configure:

{ "startUrls": "https://www.ycombinator.com/companies?batch=Winter%202024&industry=Industrials", "maxCompanies": 100, "proxyConfiguration": { /* optional proxy settings */ } }

•	startUrls

A comma-separated list of directory pages to begin crawling (typically filtered by batch/industry). • maxCompanies Maximum number of companies to fetch before stopping. • proxyConfiguration (Optional) Pass your proxy settings to avoid IP rate limits.

Example Output

A single record in the resulting dataset might look like:

{ "batch": "Winter 2024", "company_id": 29444, "company_image": "https://.../logos/6652.png", "company_linkedin": "https://linkedin.com/company/vista-power", "company_location": "San Francisco, CA, USA", "company_name": "Vista Power", "company_x": "https://twitter.com/vista_power", "founders/0/id": 293472, "founders/0/name": "Maxim Kraft", "founders/0/linkedin": "https://linkedin.com/in/maximkraft", "founders/0/x": "https://twitter.com/2001kraft", "is_hiring": false, "number_of_open_jobs": 0, "short_description": "Batteries and solar for $0 upfront to help businesses save money", "long_description": "Vista designs, finances, installs, and manages...", "status": "Active", "tags": ["energy-storage","b2b","sustainability","manufacturing","renewable-energy"], "tags/0": "energy-storage", "tags/1": "b2b", "tags/2": "sustainability", "tags/3": "manufacturing", "tags/4": "renewable-energy", "team_size": 3, "primary_partner": "https://www.ycombinator.com/people/brad-flora", "url": "https://www.ycombinator.com/companies/vista-power", "website": "https://vistapower.com", "year_founded": 2023 }

Usage Cases • Market Research Build a dataset of YC-backed startups by batch or industry to identify emerging trends. • Directory Creation Power a searchable directory or website listing with up-to-date company profiles. • Investor Due Diligence Quickly assemble core details on multiple companies for screening or outreach. • Job Aggregation Find startups currently hiring and integrate openings into a job board. • Data Science / Analytics Analyze growth, location distribution, sector focus, hiring patterns, and other metrics across YC batches.

⸻

Start from your directory page URL(s), adjust the limits to suit your needs, and you’ll get a clean, flat CSV/JSON you can plug directly into spreadsheets, BI tools, or databases.

contact me at fatih@fatihtahta.com if you have questions or open an issue here. Happy scrapings!