🚀 Y Combinator (YC) Startups & Directory Scraper

A high-performance Apify Actor written in Python to extract comprehensive company metadata, founder profiles, social links, tech tags, and open job postings directly from Y Combinator's directory API and pages.

---

✨ Features

• ⚡ Direct Algolia API Queries: Queries Y Combinator's underlying search index asynchronously without heavy browser automation (~128MB RAM footprint).
• 👥 Founder Profiles & Bio Details: Extract founder names, titles, bios, LinkedIn, Twitter, and avatar URLs.
• 💼 Job Postings Integration: Optionally scrape job listings (salary ranges, equity ranges, experience, visa sponsorship, required skills) for each company.
• 🏷️ Rich Filtering: Filter by YC batch (e.g., W24, S23), industry, hiring status, top YC companies, or search keywords.
• 📊 Standardized Pydantic Output: Outputs clean, validated JSON records compatible with Apify Datasets and export formats (JSON, CSV, Excel).

---

📥 Input Parameters

The Actor accepts the following JSON input options:

Field	Type	Default	Description
query	string	null	Keyword filter (e.g., "AI", "Fintech", "DevOps").
batches	array	["W24", "S24"]	List of YC batch codes to filter (e.g. ["W24"], ["S23"]).
industries	array	[]	Filter by industry (e.g. ["B2B"], ["Healthcare"]).
isHiringOnly	boolean	false	When enabled, only returns companies with active job listings.
topCompaniesOnly	boolean	false	When enabled, only returns companies flagged as top YC companies.
scrapeJobs	boolean	false	When enabled, scrapes individual job postings for each company.
maxItems	integer	100	Maximum number of companies to scrape (set 0 for unlimited).

Example Input

{
  "query": "AI",
  "batches": ["W24", "S24"],
  "isHiringOnly": true,
  "scrapeJobs": true,
  "maxItems": 50
}

---

📤 Output Schema

Each item pushed to the dataset represents a full Y Combinator startup record:

{
  "id": 29381,
  "name": "Example AI",
  "slug": "example-ai",
  "yc_url": "https://www.ycombinator.com/companies/example-ai",
  "website": "https://example.ai",
  "one_liner": "Autonomous workflow automation platform",
  "long_description": "Example AI helps teams automate complex multi-step workflows using generative AI...",
  "batch": "W24",
  "batch_name": "Winter 2024",
  "status": "Active",
  "industry": "B2B",
  "subindustry": "Engineering, Product and Design",
  "tags": ["AI", "Developer Tools", "B2B"],
  "locations": ["San Francisco, CA, USA"],
  "team_size": 12,
  "is_hiring": true,
  "is_top_company": false,
  "linkedin_url": "https://linkedin.com/company/example-ai",
  "twitter_url": "https://twitter.com/example_ai",
  "github_url": "https://github.com/example-ai",
  "founders": [
    {
      "full_name": "Jane Doe",
      "title": "CEO & Co-founder",
      "founder_bio": "Previously AI researcher at Stanford...",
      "linkedin_url": "https://linkedin.com/in/janedoe",
      "twitter_url": "https://twitter.com/janedoe"
    }
  ],
  "jobs": [
    {
      "id": 98765,
      "title": "Senior AI Engineer",
      "url": "https://www.ycombinator.com/companies/example-ai/jobs/98765-senior-ai-engineer",
      "location": "San Francisco, CA",
      "job_type": "Full-time",
      "salary_range": "$160k - $220k",
      "equity_range": "0.25% - 0.50%",
      "skills": ["Python", "PyTorch", "LLMs"]
    }
  ]
}

---

🚀 Running Locally

Step 1: Install Dependencies

$pip install -r requirements.txt

Step 2: Run the Actor

$python -m src

---

🐳 Docker Support

To build and run containerized:

docker build -t yc-company-scraper .
docker run -it yc-company-scraper