# Y Combinator Scraper with Founders (`scrapers-hub/y-combinator-scraper-with-founders`) Actor Y Combinator scraper to extract startup profiles, founder names, company details, websites, industries, batch information, locations, and other publicly available data 🚀📊 Perfect for startup research, investor sourcing, competitor analysis, and market intelligence. - **URL**: https://apify.com/scrapers-hub/y-combinator-scraper-with-founders.md - **Developed by:** [Scrapers Hub](https://apify.com/scrapers-hub) (community) - **Categories:** Developer tools, Lead generation, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $1.99 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Arbeitsagentur Scraper | Bundesagentur für Arbeit Collects structured public job-listing data from the **Bundesagentur für Arbeit** job portal (arbeitsagentur.de) via its **official REST API** — identifiers, titles, employers, descriptions, categories, locations, work arrangements, contract signals, publication dates, and application links. Output is normalized JSON, ready for analytics, ETL, enrichment, and recurring monitoring. Two input modes: build a search with filters, or provide direct Bundesagentur search-result URLs in `startUrls`. ### Input parameters | Parameter | Type | Description | Default | |---|---|---|---| | `keyword` | string | Search terms (`was`). | – | | `location` | string | City / postal code / region (`wo`). Overrides the location in direct URLs. | – | | `employment_type` | string | `job`, `education`, `early_career`, `self_employed` → `angebotsart` (1/4/34/2). | `job` | | `radius` | string | `10_km` … `200_km` → `umkreis`. | – | | `is_remote` | boolean | Remote / home-office only → `arbeitszeit=ho`. | – | | `is_suitable_for_career_change` | boolean | Career-change-friendly roles (client-side). | – | | `beginning_date` | array | Preferred start periods `from_now_on`, `may_2026`, … (client-side). | – | | `fixed_term` | array | `temporary`, `indefinite` → `befristung` (1/2). | – | | `include_agency_jobs` | boolean | Include private placement (pav) jobs. Off → `pav=false`. | – | | `exclude_external_jobs` | boolean | Drop external job-board listings (client-side). | – | | `exclude_temporary_work` | boolean | Drop temp-agency work → `zeitarbeit=false`. | – | | `disabled_only_jobs` | boolean | Only accessibility-marked jobs → `behinderung=true`. | – | | `publication_date` | string | `today`/`yesterday`/`last_1_week`/`last_2_weeks`/`last_4_weeks` → `veroeffentlichtseit` (0/1/7/14/28). | – | | `work_schedule` | array | `full_time`/`part_time`/`shift_night_weekend`/`part_time_job` → `arbeitszeit` (vz/tz/snw/mj). | – | | `startUrls` | array | Direct Bundesagentur search-result URLs. | – | | `limit` | integer | Max jobs to collect. | `50000` | #### How filters are applied - **Server-side** (Bundesagentur query): `employment_type`, `keyword`, `location`, `radius`, `is_remote`, `work_schedule`, `publication_date`, `fixed_term`, `include_agency_jobs`, `exclude_temporary_work`, `disabled_only_jobs`. - **Client-side** (after detail fetch): `exclude_external_jobs`, `is_suitable_for_career_change`, `beginning_date`. The actor keeps paginating until `limit` matching jobs are collected. - In `startUrls` mode the configured filter params are overlaid on each URL (URL-provided values win); the core search (`was`/`wo`/`umkreis`) comes from the URL. #### Example inputs ```json { "keyword": "Data Engineer", "location": "Berlin", "employment_type": "job", "radius": "25_km", "is_remote": true, "publication_date": "last_1_week", "limit": 50 } ```` ```json { "startUrls": [ "https://www.arbeitsagentur.de/jobsuche/suche?angebotsart=1&was=softwareentwickler&wo=Hamburg&umkreis=15" ], "exclude_external_jobs": true, "include_agency_jobs": false, "work_schedule": ["full_time"], "limit": 100 } ``` ### Output One `job` record per listing, written to the default dataset. Recommended idempotency key: `type + ":" + id`. Key fields: `type`, `id`, `url` (external application link when present, else the portal page), `source`, `sourceContext`, `title`, `company`, `description`, `category`, `categories`, `listingType`, `contractType`, `workModes`, `miniJob`, `careerChangeFriendly`, `disabilityFriendly`, `privatePlacementService`, `temporaryAgency`, `managedListing`, `dates`, `compensation`, `locations`, `distanceKm`, `remoteWork` (`available`, `arrangementType`), `contactReference`, `applicationUrl`, `postingPartner`, `referenceNumber`, and `sourceData` (`search` + `detail` provenance layers). ### How it works | Step | Endpoint | |------|----------| | Search (paginated) | `GET /pc/v4/jobs?was=…&angebotsart=1&page=…&size=…&…filters` | | Detail per job | `GET /pc/v4/jobdetails/{base64(refnr)}` | Called with the public header `X-API-Key: jobboerse-jobsuche` (no account needed). Detail pages are fetched 8 at a time with automatic retries on rate-limit / transient errors. ### Project layout ``` .actor/ actor.json · input_schema.json · dataset_schema.json · Dockerfile requirements.txt # apify, httpx (pinned) src/ __main__.py · main.py (orchestration) · api.py (REST client) · mappers.py (normalization) ``` ### Run locally ```bash pip install -r requirements.txt ## put input in storage/key_value_stores/default/INPUT.json, then: python -m src ``` ### Deploy to Apify ```bash apify push ``` # Actor input Schema ## `employment_type` (type: `string`): Select which type of opportunity to search for when using the built-in query builder. ## `keyword` (type: `string`): Add job titles, skills, technologies, or other search terms. Leave empty to rely on the terms already included in your direct search URL. ## `location` (type: `string`): City, district, postal code, or region. If you also provide direct search URLs, this value takes priority over the location in those links. ## `radius` (type: `string`): How wide the location-based search should be. Smaller radiuses keep results local. ## `is_remote` (type: `boolean`): Limit results to listings marked as remote or home-office friendly. ## `is_suitable_for_career_change` (type: `boolean`): Prioritize listings suitable for applicants changing careers or starting a new professional path. ## `beginning_date` (type: `array`): Select one or more start periods to focus on jobs beginning immediately or within specific upcoming months. ## `fixed_term` (type: `array`): Select the contract types you want to include so results better match your preferred employment terms. ## `include_agency_jobs` (type: `boolean`): Includes additional job postings from private recruitment agencies. Leave disabled to keep results focused on direct employer postings. ## `exclude_external_jobs` (type: `boolean`): Removes listings that point to external job boards or third-party sources. ## `exclude_temporary_work` (type: `boolean`): Removes temporary-work and labor-leasing (Arbeitnehmerüberlassung) positions from the results. ## `disabled_only_jobs` (type: `boolean`): Limit the results to positions explicitly marked as suitable for disabled applicants. ## `publication_date` (type: `string`): Focus on fresh postings, such as jobs published today, yesterday, or within the last few weeks. ## `work_schedule` (type: `array`): Select one or more schedule types, such as full-time, part-time, or shift-based work. ## `startUrls` (type: `array`): Paste Bundesagentur search-result URLs to scrape those result pages directly. Useful when you already configured a search on the site and want to reuse it. ## `limit` (type: `integer`): The largest number of job listings the actor should save before stopping. Lower limits are useful for testing. ## Actor input object example ```json { "employment_type": "job", "keyword": "Elektroingenieur", "is_remote": false, "is_suitable_for_career_change": false, "include_agency_jobs": false, "exclude_external_jobs": false, "exclude_temporary_work": false, "disabled_only_jobs": false, "limit": 50 } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "keyword": "Elektroingenieur", "limit": 50 }; // Run the Actor and wait for it to finish const run = await client.actor("scrapers-hub/y-combinator-scraper-with-founders").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "keyword": "Elektroingenieur", "limit": 50, } # Run the Actor and wait for it to finish run = client.actor("scrapers-hub/y-combinator-scraper-with-founders").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "keyword": "Elektroingenieur", "limit": 50 }' | apify call scrapers-hub/y-combinator-scraper-with-founders --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=scrapers-hub/y-combinator-scraper-with-founders", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Y Combinator Scraper with Founders", "description": "Y Combinator scraper to extract startup profiles, founder names, company details, websites, industries, batch information, locations, and other publicly available data 🚀📊 Perfect for startup research, investor sourcing, competitor analysis, and market intelligence.", "version": "1.0", "x-build-id": "UG084FEPq5XWHWIWp" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/scrapers-hub~y-combinator-scraper-with-founders/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-scrapers-hub-y-combinator-scraper-with-founders", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/scrapers-hub~y-combinator-scraper-with-founders/runs": { "post": { "operationId": "runs-sync-scrapers-hub-y-combinator-scraper-with-founders", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/scrapers-hub~y-combinator-scraper-with-founders/run-sync": { "post": { "operationId": "run-sync-scrapers-hub-y-combinator-scraper-with-founders", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "employment_type": { "title": "Choose an Employment Type", "enum": [ "job", "education", "early_career", "self_employed" ], "type": "string", "description": "Select which type of opportunity to search for when using the built-in query builder.", "default": "job" }, "keyword": { "title": "Enter Keywords to Match Relevant Jobs", "type": "string", "description": "Add job titles, skills, technologies, or other search terms. Leave empty to rely on the terms already included in your direct search URL." }, "location": { "title": "Choose a Location to Search Around", "type": "string", "description": "City, district, postal code, or region. If you also provide direct search URLs, this value takes priority over the location in those links." }, "radius": { "title": "Set the Search Radius", "enum": [ "10_km", "15_km", "25_km", "50_km", "100_km", "200_km" ], "type": "string", "description": "How wide the location-based search should be. Smaller radiuses keep results local." }, "is_remote": { "title": "Show Remote-Friendly Jobs Only", "type": "boolean", "description": "Limit results to listings marked as remote or home-office friendly.", "default": false }, "is_suitable_for_career_change": { "title": "Focus on Career-Change-Friendly Roles", "type": "boolean", "description": "Prioritize listings suitable for applicants changing careers or starting a new professional path.", "default": false }, "beginning_date": { "title": "Choose a Preferred Start Date", "type": "array", "description": "Select one or more start periods to focus on jobs beginning immediately or within specific upcoming months.", "items": { "type": "string", "enum": [ "from_now_on", "may_2026", "june_2026", "july_2026", "august_2026", "september_2026", "october_2026", "november_2026", "december_2026", "january_2027", "february_2027", "march_2027", "april_2027", "may_2027", "june_2027", "july_2027", "august_2027", "september_2027", "october_2027", "february_2028", "april_2028" ], "enumTitles": [ "from now on", "May 2026", "June 2026", "July 2026", "August 2026", "September 2026", "October 2026", "November 2026", "December 2026", "January 2027", "February 2027", "March 2027", "April 2027", "May 2027", "June 2027", "July 2027", "August 2027", "September 2027", "October 2027", "February 2028", "April 2028" ] } }, "fixed_term": { "title": "Choose Contract Duration Types", "type": "array", "description": "Select the contract types you want to include so results better match your preferred employment terms.", "items": { "type": "string", "enum": [ "temporary", "indefinite" ], "enumTitles": [ "Temporary (befristet)", "Indefinite (unbefristet)" ] } }, "include_agency_jobs": { "title": "Include Jobs Posted by Recruitment Agencies", "type": "boolean", "description": "Includes additional job postings from private recruitment agencies. Leave disabled to keep results focused on direct employer postings.", "default": false }, "exclude_external_jobs": { "title": "Remove Listings from External Job Boards", "type": "boolean", "description": "Removes listings that point to external job boards or third-party sources.", "default": false }, "exclude_temporary_work": { "title": "Hide Temporary Work Listings", "type": "boolean", "description": "Removes temporary-work and labor-leasing (Arbeitnehmerüberlassung) positions from the results.", "default": false }, "disabled_only_jobs": { "title": "Show Jobs Marked for Disabled Applicants", "type": "boolean", "description": "Limit the results to positions explicitly marked as suitable for disabled applicants.", "default": false }, "publication_date": { "title": "Choose How Recent the Listings Should Be", "enum": [ "today", "yesterday", "last_1_week", "last_2_weeks", "last_4_weeks" ], "type": "string", "description": "Focus on fresh postings, such as jobs published today, yesterday, or within the last few weeks." }, "work_schedule": { "title": "Choose Preferred Work Schedules", "type": "array", "description": "Select one or more schedule types, such as full-time, part-time, or shift-based work.", "items": { "type": "string", "enum": [ "full_time", "part_time", "shift_night_weekend", "part_time_job" ], "enumTitles": [ "Full-time", "Part-time", "Shift/Night/Weekend", "Part-time job (Minijob)" ] } }, "startUrls": { "title": "Provide Direct Search Result URLs", "type": "array", "description": "Paste Bundesagentur search-result URLs to scrape those result pages directly. Useful when you already configured a search on the site and want to reuse it.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "limit": { "title": "Set the Maximum Number of Jobs to Collect", "minimum": 1, "maximum": 100000, "type": "integer", "description": "The largest number of job listings the actor should save before stopping. Lower limits are useful for testing.", "default": 50000 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```