The Muse Jobs Search Scraper

This Actor extracts rich, verified public job records from The Muse. It uses native HTTPS requests and Cheerio to read server-rendered job cards, public React Flight payloads, public JobPosting data, and the visible detail description.

The implementation runs locally or on Apify Cloud using direct public HTTPS. It uses Apify dataset and key-value storage, but does not use a proxy, spoof fingerprints, solve challenges, or fabricate an application URL. A normal browser user-agent and navigation headers are sent to the public site.

Modes

• Search: use query, optional location, and bounded maxPages/maxItems.
• Single: provide one official jobUrl.
• Multiple: provide a bounded jobUrls or directUrls list.

The Muse job URLs are slug-based and do not expose an ID-only public route. jobId and jobIds are therefore optional identity checks that must be paired with matching public URLs.

Example search input:

{
  "mode": "search",
  "query": "software engineer",
  "maxItems": 3,
  "maxPages": 1,
  "maxConcurrency": 3
}

Example single input:

{
  "mode": "single",
  "jobUrl": "https://www.themuse.com/jobs/arcadia/principal-software-engineerplatforms",
  "maxItems": 1
}

Dataset

Every emitted row is buffered until the detail page has been parsed and identity-checked. Required fields are record metadata, mode, job ID, title, official job URL, company, location, full description, source URLs, and scrape timestamp. Optional publisher fields are omitted when The Muse does not publish them, so successful rows contain no null, blank, placeholder, empty-array, or empty-object values.

The output includes:

• job identity, canonical URL, state, listing type, normalized title, ATS ID/provider, dates, salary, employment type, categories, levels, skills, benefits, and remote-work signals;
• full plain-text and HTML descriptions plus labeled detail sections;
• company IDs, profile/logo/site, description, size, employee range, industries, perks, diversity attributes, and public videos;
• structured locations, physical addresses, applicant locations, application URL when the source publishes a real external link;
• source payloads, response receipts, extraction method, provenance, data-quality flags, and field coverage.

The local validator checks required-field completion, detail verification, duplicate IDs/URLs, and recursively rejects null, blank, placeholder, empty-array, and empty-object values.

Local verification

From this directory:

npm install --ignore-scripts --no-audit --no-fund
npm test
npm run lint
apify validate-schema .actor/input_schema.json
apify run --purge --input-file INPUT.json
npm run validate

The reproducible fixtures INPUT.json, INPUT-single.json, and INPUT-multiple.json exercise all three modes. The local run writes the default dataset and the OUTPUT, OUTPUT_SUMMARY, RUN_HEALTH, and RUN_DIAGNOSTICS records under storage.