# Omni Jobs Board Scraper (`automation-lab/omni-jobs-board-scraper`) Actor Scrape Reed and Arbeitnow job listings into one deduped dataset with salary, company, location, dates, URLs, and remote fields. - **URL**: https://apify.com/automation-lab/omni-jobs-board-scraper.md - **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community) - **Categories:** Jobs - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Omni Jobs Board Scraper Scrape Reed.co.uk and Arbeitnow job listings into one normalized, deduped dataset for recruiting, sourcing, lead generation, and labor-market analysis. ### What does Omni Jobs Board Scraper do? Omni Jobs Board Scraper searches multiple public job boards and exports clean job records. It currently supports: - ✅ Reed.co.uk job search pages - ✅ Arbeitnow public job board API - ✅ Keyword search - ✅ Location filtering - ✅ Source selection - ✅ Recency filtering - ✅ URL or title/company/location deduplication - ✅ Optional cleaned job descriptions ### Who is it for? Recruiters use it to monitor active hiring across public boards. Talent sourcers use it to discover companies hiring for specific roles. Sales teams use it to build hiring-intent lead lists. Job-board operators use it to benchmark listings and categories. Labor-market analysts use it to collect structured job demand data. Agencies use it to export fresh roles for client reporting. ### Why use this actor? Running several single-source scrapers creates duplicate rows and different field names. This actor gives you one normalized export with consistent fields. You can search Reed and Arbeitnow in one run, dedupe results, and download the output as JSON, CSV, Excel, XML, or RSS through Apify datasets. ### Supported job boards The MVP focuses on reliable HTTP sources. | Source | Region | Method | Notes | | --- | --- | --- | --- | | Reed.co.uk | United Kingdom | HTML with embedded JSON | Strong salary and company fields | | Arbeitnow | Germany / remote | Public JSON API | Good tags, job type, and remote signals | ### What data can you extract? The actor outputs one row per deduped job record. | Field | Description | | --- | --- | | `source` | Source board: `reed` or `arbeitnow` | | `jobId` | Source-specific job identifier | | `title` | Job title | | `company` | Hiring company | | `location` | Job location text | | `country` | Normalized country when known | | `remoteType` | Remote, hybrid, on-site, or source-provided remote label | | `salaryText` | Salary text when available | | `salaryMin` | Minimum salary as a number when available | | `salaryMax` | Maximum salary as a number when available | | `currency` | Salary currency when known | | `postedAt` | Posting date | | `expiresAt` | Expiration date when available | | `jobUrl` | Public job URL | | `applyUrl` | Apply URL when available | | `description` | Cleaned description when enabled | | `employmentType` | Full-time, part-time, contract, or source labels | | `category` | Category, taxonomy, or tags | | `dedupeKey` | Key used for duplicate detection | | `scrapedAt` | Timestamp for this scrape | ### How much does it cost to scrape job board listings? The actor uses pay-per-event pricing. There is a small **$0.005 run start** event and a tiered per-job-record event. Current BRONZE price is **$0.0001 per job record** (about **$0.10 per 1,000 job records**). Higher-volume tiers receive lower per-record prices. For small test runs, keep `maxItems` low. For production monitoring, increase `maxItems` after you confirm the search terms and source mix return the records you need. ### Quick start 1. Open the actor on Apify. 2. Enter a keyword such as `software engineer`. 3. Enter a location such as `London` or leave it empty. 4. Choose Reed, Arbeitnow, or both. 5. Set `maxItems`. 6. Run the actor. 7. Export the dataset in your preferred format. ### Input options #### Keyword Use a role, skill, department, or hiring phrase. Examples: - `software engineer` - `data analyst` - `sales manager` - `nurse` - `python` #### Location Use a city, region, country, or remote keyword. Examples: - `London` - `Manchester` - `Berlin` - `Remote` #### Sources Choose one or more supported sources. Use only Reed for UK-focused searches. Use Arbeitnow for Germany and remote-friendly searches. Use both for a broader unified export. #### Maximum job records `maxItems` controls the maximum number of deduped jobs saved to the dataset. The actor may fetch more source rows internally because duplicates and filters can remove records. #### Posted within days Use this optional filter when you only need fresh listings. For example, set `postedWithinDays` to `7` for jobs posted in the last week. #### Dedupe by Use `titleCompanyLocation` for cross-board dedupe. Use `url` when you want every unique source URL preserved. #### Include descriptions Descriptions can be long. Turn `includeDescription` on for NLP, keyword enrichment, and full-text analysis. Leave it off for smaller exports and faster downstream processing. ### Example input ```json { "keyword": "software engineer", "location": "London", "sources": ["reed", "arbeitnow"], "maxItems": 25, "postedWithinDays": 30, "dedupeBy": "titleCompanyLocation", "includeDescription": false } ```` ### Example output ```json { "source": "reed", "jobId": "56945474", "title": "Senior Software Engineer", "company": "Example Analytics", "location": "London", "country": "United Kingdom", "remoteType": "Hybrid", "salaryText": null, "salaryMin": 65000, "salaryMax": 80000, "currency": "GBP", "postedAt": "2026-05-27T07:40:04.147", "expiresAt": "2026-07-08T23:59:59", "jobUrl": "https://www.reed.co.uk/jobs/56945474", "applyUrl": "https://www.reed.co.uk/jobs/56945474", "description": null, "employmentType": "Full-time", "category": "Software Engineer", "dedupeKey": "senior software engineer|example analytics|london", "scrapedAt": "2026-06-19T00:00:00.000Z" } ``` ### Tips for better results Use specific keywords for targeted exports. Use broader keywords when you are measuring market size. Run one source at a time if you need to compare boards. Enable descriptions only when you need them. Use recency filters for alerting workflows. Use title/company/location dedupe for cross-board reporting. ### Common workflows #### Hiring-intent lead generation Search for roles that suggest software, staffing, benefits, office, or training demand. Export company names and locations. Enrich the companies in your CRM. #### Recruiter monitoring Run daily searches for your niche. Filter to jobs posted in the last 1-3 days. Send new rows to Slack, Google Sheets, or Airtable. #### Labor-market analysis Run repeated keyword searches. Store datasets by date. Track salary bands, locations, and category changes over time. #### Job-board benchmarking Compare Reed and Arbeitnow volume for similar keywords. Measure duplicate rates with the dedupe key. Identify categories that deserve dedicated monitoring. ### Integrations Use Apify integrations to connect the dataset to: - Google Sheets for simple review - Make for no-code workflows - Zapier for alerts - Airtable for recruiting operations - Slack for daily notifications - Webhooks for custom pipelines - S3 or Google Cloud Storage for archives ### API usage #### Node.js ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('automation-lab/omni-jobs-board-scraper').call({ keyword: 'software engineer', location: 'London', maxItems: 25, }); console.log(run.defaultDatasetId); ``` #### Python ```python from apify_client import ApifyClient client = ApifyClient('MY-APIFY-TOKEN') run = client.actor('automation-lab/omni-jobs-board-scraper').call(run_input={ 'keyword': 'software engineer', 'location': 'London', 'maxItems': 25, }) print(run['defaultDatasetId']) ``` #### cURL ```bash curl -X POST "https://api.apify.com/v2/acts/automation-lab~omni-jobs-board-scraper/runs?token=$APIFY_TOKEN" \ -H 'Content-Type: application/json' \ -d '{"keyword":"software engineer","location":"London","maxItems":25}' ``` ### MCP usage Use this actor from Claude Desktop or Claude Code through Apify MCP. MCP URL: ```text https://mcp.apify.com/?tools=automation-lab/omni-jobs-board-scraper ``` Claude Code setup: ```bash claude mcp add apify-jobs https://mcp.apify.com/?tools=automation-lab/omni-jobs-board-scraper ``` Claude Desktop JSON configuration: ```json { "mcpServers": { "apify-jobs": { "url": "https://mcp.apify.com/?tools=automation-lab/omni-jobs-board-scraper" } } } ``` Example prompts: - "Find 25 software engineer jobs in London and summarize the top hiring companies." - "Scrape remote data analyst jobs and group them by company." - "Run a fresh job-board search and return only jobs posted this week." ### Data quality notes Different job boards expose different fields. Some records may not include salary. Some records may not include a separate apply URL. Descriptions are cleaned from source HTML when enabled. Deduplication is best-effort and configurable. ### Limits The actor only scrapes public job listing data. It does not log in to job boards. It does not apply to jobs. It does not collect applicant data. It does not bypass private or authenticated pages. ### FAQ #### Can I scrape both boards in one run? Yes. Keep `sources` set to `["reed", "arbeitnow"]` for a unified cross-board dataset. #### Can I export jobs to CSV or Excel? Yes. Use the Apify dataset export buttons or API after the run finishes. ### Troubleshooting #### Why did I get fewer rows than maxItems? The source may have fewer matching jobs, filters may remove old records, or dedupe may collapse duplicates. Try a broader keyword, remove the location, or include both sources. #### Why are salary fields empty? Not every job board or job listing publishes salary data. When salary data is available in structured fields, the actor exports it. #### Why are descriptions missing? Descriptions are optional. Set `includeDescription` to `true` if you need full text. ### Legality and responsible use This actor extracts publicly available job listing information. You are responsible for using the data legally and respecting source website terms, privacy laws, and applicable regulations. Do not use scraped data for spam, discrimination, or unlawful profiling. ### Related scrapers Other automation-lab actors that can complement this workflow: - https://apify.com/automation-lab/indeed-scraper - https://apify.com/automation-lab/reed-jobs-scraper - https://apify.com/automation-lab/remoteok-jobs-scraper - https://apify.com/automation-lab/jobicy-jobs-scraper ### Changelog #### 0.1 Initial version with Reed and Arbeitnow support, normalized job output, dedupe, recency filter, and optional descriptions. ### Support If a run fails or a source changes its page structure, open an issue from the Apify run page and include the run ID. ### Summary Omni Jobs Board Scraper gives recruiters, analysts, and revenue teams a practical way to collect normalized public job listings from multiple boards without merging separate exports by hand. # Actor input Schema ## `keyword` (type: `string`): Job title, skill, or hiring keyword to search for. ## `location` (type: `string`): City, region, country, or remote keyword. Leave empty for broad searches. ## `sources` (type: `array`): Job boards to include in the unified output. ## `maxItems` (type: `integer`): Maximum number of deduped job records to save. ## `postedWithinDays` (type: `integer`): Only keep jobs posted within this many days. Leave empty to keep all visible jobs. ## `dedupeBy` (type: `string`): Choose whether duplicate detection uses exact URLs or normalized title + company + location. ## `includeDescription` (type: `boolean`): Save cleaned job descriptions. Disable for smaller, faster exports. ## Actor input object example ```json { "keyword": "software engineer", "location": "London", "sources": [ "reed", "arbeitnow" ], "maxItems": 20, "dedupeBy": "titleCompanyLocation", "includeDescription": false } ``` # Actor output Schema ## `datasetUrl` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "keyword": "software engineer", "location": "London", "sources": [ "reed", "arbeitnow" ], "maxItems": 20, "dedupeBy": "titleCompanyLocation", "includeDescription": false }; // Run the Actor and wait for it to finish const run = await client.actor("automation-lab/omni-jobs-board-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "keyword": "software engineer", "location": "London", "sources": [ "reed", "arbeitnow", ], "maxItems": 20, "dedupeBy": "titleCompanyLocation", "includeDescription": False, } # Run the Actor and wait for it to finish run = client.actor("automation-lab/omni-jobs-board-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "keyword": "software engineer", "location": "London", "sources": [ "reed", "arbeitnow" ], "maxItems": 20, "dedupeBy": "titleCompanyLocation", "includeDescription": false }' | apify call automation-lab/omni-jobs-board-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=automation-lab/omni-jobs-board-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Omni Jobs Board Scraper", "description": "Scrape Reed and Arbeitnow job listings into one deduped dataset with salary, company, location, dates, URLs, and remote fields.", "version": "0.1", "x-build-id": "iPTexUmidxsSkevEc" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/automation-lab~omni-jobs-board-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-automation-lab-omni-jobs-board-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/automation-lab~omni-jobs-board-scraper/runs": { "post": { "operationId": "runs-sync-automation-lab-omni-jobs-board-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/automation-lab~omni-jobs-board-scraper/run-sync": { "post": { "operationId": "run-sync-automation-lab-omni-jobs-board-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "keyword": { "title": "Keyword", "type": "string", "description": "Job title, skill, or hiring keyword to search for.", "default": "software engineer" }, "location": { "title": "Location", "type": "string", "description": "City, region, country, or remote keyword. Leave empty for broad searches.", "default": "London" }, "sources": { "title": "Sources", "type": "array", "description": "Job boards to include in the unified output.", "items": { "type": "string" }, "default": [ "reed", "arbeitnow" ] }, "maxItems": { "title": "Maximum job records", "minimum": 1, "maximum": 10000, "type": "integer", "description": "Maximum number of deduped job records to save.", "default": 50 }, "postedWithinDays": { "title": "Posted within days", "minimum": 1, "maximum": 365, "type": "integer", "description": "Only keep jobs posted within this many days. Leave empty to keep all visible jobs." }, "dedupeBy": { "title": "Dedupe by", "enum": [ "url", "titleCompanyLocation" ], "type": "string", "description": "Choose whether duplicate detection uses exact URLs or normalized title + company + location.", "default": "titleCompanyLocation" }, "includeDescription": { "title": "Include descriptions", "type": "boolean", "description": "Save cleaned job descriptions. Disable for smaller, faster exports.", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```