Y Combinator Companies Scraper

Extract detailed company data from the Y Combinator directory - startup profiles, batch info, hiring signals, locations, and industry taxonomy. This scraper collects structured records from YC listings with automatic pagination, making it easy to build datasets for market research, talent sourcing, and cohort analysis.

Features

• Complete company profiles - Collect name, description, batch, status, location, team size, website, and social links
• Industry taxonomy - Tags, industries, regions, and market category for every company
• Hiring and growth signals - Hiring status, nonprofit flag, top-company marker, and launch date
• Flexible pagination - Set a result limit, max pages, or scrape everything available
• Batch and location filtering - Pass a YC directory URL with query parameters to target specific cohorts or regions
• Clean export-ready data - JSON output with consistent field types for analysis

Use Cases

Startup Market Analysis

Build a comprehensive dataset of YC-funded companies to analyze trends across batches, stages, and industries. Track which sectors are getting funded and compare cohort characteristics over time.

Talent Acquisition Intelligence

Identify companies that are actively hiring, review their team size and stage, and prioritize outreach targets based on growth signals and market categories.

Portfolio and Ecosystem Research

Map the Y Combinator ecosystem by company location, founding year, and industry. Use the taxonomy fields (tags, industries, regions) to segment companies for investment or partnership research.

Batch and Cohort Comparison

Compare company characteristics across YC batches. Filter by batch parameter to examine how startup profiles, funding stages, and hiring patterns have evolved over successive cohorts.

---

Input Parameters

Parameter	Type	Required	Default	Description
url	String	No	https://www.ycombinator.com/companies/industry/all	YC directory URL to scrape. Supports path and query parameters for filtering
scrape_all_companies	Boolean	No	false	Ignore the result limit and collect all available companies (capped by max pages)
results_wanted	Integer	No	20	Maximum number of company records to save
max_pages	Integer	No	5	Safety limit for the number of listing pages to paginate through
proxyConfiguration	Object	No	Apify Proxy	Proxy settings for reliable extraction. Residential proxy recommended for larger runs

---

Output Data

Each dataset item contains the following fields:

Field	Type	Description
company_id	String	Y Combinator company ID
slug	String	Company slug used in profile URL
company_name	String	Company name
url	String	Absolute URL to the YC profile page
short_description	String	One-line summary from the listing
one_liner	String	Tagline from the YC listing
long_description	String	Full company description
batch	String	YC batch (e.g., S25, W24)
status	String	Company status (Active, Acquired, Public, etc.)
tags	Array	Industry tags
company_location	String	Primary company location
year_founded	String	Founding year
founded_raw	String	Raw founding date text from the YC profile
team_size	String	Reported team size
market	String	Market or category tag
website	String	Company website URL
company_linkedin	String	LinkedIn profile URL
company_x	String	X / Twitter profile URL
industries	Array	Industry taxonomy labels
regions	Array	Region taxonomy labels
stage	String	Company stage (Growth, Early, etc.)
top_company	Boolean	Top-company indicator
is_hiring	Boolean	Hiring signal
nonprofit	Boolean	Nonprofit flag
launched_at	String	Launch timestamp in ISO format
company_image	String	URL of the company logo
primary_partner	String	Primary YC partner name
primary_group_partner	Object	Detailed partner info (name, title)
former_names	Array	Former company names
founders	Array	Founder profiles (enabled with detail mode)
open_jobs	Array	Open job listings (enabled with detail mode)
algolia_raw	Object	Raw Algolia record for debugging

---

Usage Examples

Basic Company Collection

Collect 20 companies from the default listing:

{
  "results_wanted": 20,
  "max_pages": 1
}

Targeted Batch Extraction

Scrape 50 companies from a specific YC batch:

{
  "url": "https://www.ycombinator.com/companies/industry/all?batch=S25",
  "results_wanted": 50,
  "max_pages": 3
}

Full Directory Export

Scrape all available companies with a generous page limit:

{
  "scrape_all_companies": true,
  "max_pages": 50
}

---

Sample Output

{
  "company_id": "271",
  "slug": "airbnb",
  "company_name": "Airbnb",
  "url": "https://www.ycombinator.com/companies/airbnb",
  "short_description": "Book accommodations around the world.",
  "one_liner": "Book accommodations around the world.",
  "long_description": "Founded in August of 2008 and based in San Francisco...",
  "batch": "Winter 2009",
  "status": "Public",
  "tags": ["Marketplace", "E-commerce"],
  "company_location": "San Francisco, CA, USA",
  "website": "http://airbnb.com",
  "team_size": "6132",
  "market": "Consumer",
  "industries": ["Consumer", "Travel, Leisure and Tourism"],
  "regions": ["United States of America", "America / Canada"],
  "stage": "Growth",
  "top_company": true,
  "is_hiring": false,
  "nonprofit": false,
  "launched_at": "2012-01-17T22:34:16.000Z"
}

---

Tips for Best Results

Start With a Small Test

Use results_wanted: 20 and max_pages: 1 for initial testing to verify your target URL returns expected data.

Use Proxy for Large Runs

Enable Apify Proxy with residential groups when scraping many pages to maintain consistent extraction.

Target Specific Batches

Use batch query parameters in the URL (e.g., ?batch=W24) to narrow results to a specific YC cohort for faster, more focused runs.

---

Integrations

Connect your scraped data with:

• Google Sheets - Export datasets for sorting and filtering
• Airtable - Build searchable company databases
• Make / Zapier - Trigger automated workflows on new data
• Webhooks - Push records into custom systems
• Slack - Get notified when new companies are collected

Export Formats

• JSON - For APIs and programmatic access
• CSV - For spreadsheet analysis and BI tools
• Excel - For business reporting
• XML - For legacy system integrations

---

Frequently Asked Questions

How many companies can I collect?

You can collect all available companies by enabling scrape_all_companies. The only limit is how many pages you set with max_pages.

Can I filter by YC batch?

Yes. Pass a URL like https://www.ycombinator.com/companies/industry/all?batch=S25 and the actor respects the filter parameters.

Why are some fields null?

Some companies do not publish every field. Partner assignments, founding dates, and team sizes may be missing for certain records.

Does the actor need an API key?

No. All data is collected from the public Y Combinator directory. Algolia credentials are fetched automatically on each run.

Can I extract founder or job data?

Currently founders and open jobs are included as empty arrays in the output. A future update will add optional detail scraping for these fields.

How fast is the extraction?

The actor uses API-level data access for listing and enrichment, making it significantly faster than browser-based scrapers. A typical run of 100 companies completes in under a minute.

---

Support

For bug reports, feature requests, or questions, contact support through the Apify Console.

Resources

• Apify Documentation
• API Reference
• Scheduling Runs

---

Legal Notice

This actor is designed for legitimate data collection purposes. Users are responsible for ensuring compliance with Y Combinator's terms of service and applicable laws. Use data responsibly and respect rate limits.