Job Board Aggregator avatar

Job Board Aggregator

Pricing

from $2.00 / 1,000 job listings

Go to Apify Store
Job Board Aggregator

Job Board Aggregator

Aggregate remote and on-site job listings from Remotive, Arbeitnow, and custom job board URLs. Extract title, company, location, salary, remote flag, description, and apply URL.

Pricing

from $2.00 / 1,000 job listings

Rating

0.0

(0)

Developer

Harsh

Harsh

Maintained by Community

Actor stats

0

Bookmarked

1

Total users

0

Monthly active users

5 days ago

Last modified

Categories

Share

Aggregate job listings from multiple sources into a single structured dataset. Built with TypeScript, Crawlee, and Cheerio.

Features

  • Remotive API — fetches remote jobs from https://remotive.com/api/remote-jobs
  • Arbeitnow API — fetches jobs from https://www.arbeitnow.com/api/job-board-api with automatic pagination
  • Custom job boards — optional Cheerio scraping for any HTML job listing page
  • Structured output — title, company, location, salary, remote flag, description, apply URL
  • Rate limiting — configurable concurrency and requests-per-minute caps
  • Retries — failed requests are retried up to 3 times with logging

Input

FieldTypeDefaultDescription
includeRemotivebooleantrueEnable Remotive API source
includeArbeitnowbooleantrueEnable Arbeitnow API source
customJobBoardUrlsstring[][]Optional HTML job board URLs
searchQuerystringKeyword filter for Remotive
categorystringRemotive category slug
maxJobsinteger500Max jobs across all sources
maxRequestsPerCrawlinteger200Max HTTP requests
maxConcurrencyinteger2Parallel request limit
maxRequestsPerMinuteinteger30Global rate limit

Example input

{
"includeRemotive": true,
"includeArbeitnow": true,
"customJobBoardUrls": [],
"searchQuery": "typescript",
"maxJobs": 100,
"maxConcurrency": 2,
"maxRequestsPerMinute": 30
}

Output

Each dataset item contains:

FieldDescription
titleJob title
companyCompany name
locationLocation or region
salarySalary text when available
remoteWhether the job is remote
descriptionPlain-text job description
urlApply or detail page URL
sourceremotive, arbeitnow, or custom
scrapedAtISO timestamp

How it works

  1. The Actor builds start requests for each enabled source.
  2. Remotive and Arbeitnow responses are parsed as JSON.
  3. Arbeitnow pagination follows links.next until maxJobs is reached.
  4. Custom boards use Cheerio heuristics for job cards and enqueue next-page links.
  5. Results are pushed to the default dataset with per-job logging.

Local development

npm install
apify run

Edit storage/key_value_stores/default/INPUT.json for local test input.

Deploy

apify login
apify push

Publish to the Apify Store with PPE pricing:

$node ../../scripts/publish-factory.js job-board-aggregator '{"resultPriceUsd":0.002,"resultTitle":"Job listing","resultDescription":"Charged per job listing returned"}'

Notes

  • Remotive requests should be kept infrequent (their API advises max ~4 calls/day for production sync).
  • Arbeitnow does not expose salary in its public API.
  • Custom HTML scraping uses common CSS patterns; highly custom sites may need dedicated selectors.
  • Respect each source's terms of service and attribution requirements.