Indeed Job Scraper

Scrapes job listings from Indeed.com - works globally with 60+ country domains including US, UK, Germany, Switzerland, Austria, and more. Supports multiple search keywords with automatic deduplication across queries.

Three Scraping Modes

This scraper supports three distinct modes for different use cases:

Mode	Input	Output	Use Case
SEARCH MODE	Search queries + location + country	Basic job data from SERP	Fast discovery of new jobs
SEARCH + DETAILS MODE	includeJobDetails: true	Full job data with descriptions	Complete data collection in one run
DIRECT URLS MODE	directUrls: [...]	Full job data + job_status	Still-alive checks, re-scraping specific jobs

Features

• Multi-query support: Run multiple search keywords in one run, results deduplicated across queries
• Global coverage: Works with any Indeed country domain (ch, de, at, us, uk, fr, etc.)
• Two-phase scraping: Collects jobs from search results, then fetches full details in batches
• Anti-bot bypass: Uses CheerioCrawler with Chrome-like TLS fingerprinting and residential proxies
• Rich data extraction: Title, company, location, salary, description, company ratings, and more
• Standardized output: Consistent JobListing schema across all job scrapers
• search_query tracking: Each result includes which query found it

Input

Parameter	Type	Default	Description
searchQueries	string[]	["Software Engineer"]	One or more search keywords. Each runs as a separate search, deduplicated.
searchQuery	string	""	Single keyword (legacy, backward compatible). Use searchQueries for multiple.
location	string	""	City, state, or postal code
country	string	"ch"	Indeed country domain (ch, de, at, us, uk, etc.)
sortBy	string	"date"	Sort order: date (newest first) or relevance
maxResultsPerQuery	integer	100	Max results per keyword (deduplicated)
maxResults	integer	0	Total cap across all queries. 0 = unlimited.
includeJobDetails	boolean	false	Fetch full job descriptions (slower but more data)
directUrls	string[]	[]	Direct job URLs to scrape (skips search mode)
proxyConfiguration	object	Residential	Apify proxy settings

Mode 1: SEARCH MODE (Fast)

Quick search without visiting detail pages:

{
  "searchQueries": ["Developer", "Engineer"],
  "location": "Zurich",
  "country": "ch",
  "maxResultsPerQuery": 100,
  "includeJobDetails": false
}

Mode 2: SEARCH + DETAILS MODE (Complete Data)

Full data collection with detail pages:

{
  "searchQueries": ["Software Engineer", "Backend Developer"],
  "location": "Zurich",
  "country": "ch",
  "maxResultsPerQuery": 50,
  "includeJobDetails": true
}

Mode 3: DIRECT URLS MODE (Still Alive Checks)

When directUrls is provided, the scraper operates in direct mode:

{
  "directUrls": [
    "https://ch.indeed.com/viewjob?jk=abc123def456",
    "https://de.indeed.com/viewjob?jk=xyz789uvw012",
    "https://www.indeed.com/viewjob?jk=qrs345tuv678"
  ],
  "country": "ch"
}

Supported Countries

• Switzerland (ch.indeed.com)
• Germany (de.indeed.com)
• Austria (at.indeed.com)
• United States (indeed.com)
• United Kingdom (uk.indeed.com)
• France (fr.indeed.com)
• Italy (it.indeed.com)
• Spain (es.indeed.com)
• Netherlands (nl.indeed.com)
• Belgium (be.indeed.com)
• Canada (ca.indeed.com)
• Australia (au.indeed.com)
• India (in.indeed.com)
• Singapore (sg.indeed.com)
• Japan (jp.indeed.com)
• Brazil (br.indeed.com)
• Mexico (mx.indeed.com)

Output

Each job listing includes:

{
  "id": "abc123xyz",
  "title": "Software Engineer",
  "company": "Tech Corp",
  "location": "Zurich, ZH",
  "canton": null,
  "job_status": "online",
  "top_listing": true,
  "employment_type": "full-time",
  "workload_min": null,
  "workload_max": null,
  "remote_option": "hybrid",
  "salary_text": "CHF 120'000 - 150'000 per year",
  "description_snippet": "We are looking for...",
  "description_full": "Full job description...",
  "requirements": [],
  "posted_at": "2024-12-04T10:00:00.000Z",
  "expires_at": null,
  "source_url": "https://ch.indeed.com/viewjob?jk=abc123xyz",
  "source_platform": "indeed.com (ch)",
  "contact_salutation": null,
  "contact_firstname": null,
  "contact_lastname": null,
  "contact_email": null,
  "contact_phone": null,
  "contact_position": null,
  "contact_raw": null,
  "apply_url": "https://company.com/apply",
  "apply_email": null,
  "company_url": null,
  "company_description": null,
  "company_rating": 4.2,
  "company_review_count": 156,
  "company_social_urls": null,
  "company_benefits": ["Health insurance", "Flexible hours"],
  "search_query": "Software Engineer",
  "scraped_at": "2024-12-05T12:00:00.000Z"
}

Output Fields

Field	Description
id	Indeed job key (from jk= parameter)
title	Job title
company	Company name
location	City/location as displayed
canton	Swiss canton code (for Swiss jobs only)
job_status	Job availability: online, offline, expired, unknown
top_listing	Boolean - if job is sponsored/featured (Search mode only)
employment_type	full-time, part-time, contract, temporary, internship, apprenticeship
remote_option	remote, hybrid, or null
salary_text	Salary as displayed
description_snippet	First 500 characters from search results
description_full	Complete job description (Detail mode only)
requirements	Array of job requirements (usually empty for Indeed)
posted_at	Publication date (parsed from relative dates like "3 days ago")
expires_at	Expiration date (usually not available)
source_url	Link to job posting on Indeed
source_platform	"indeed.com (ch)", "indeed.com (de)", etc.
company_rating	Company rating on Indeed (0-5 stars)
company_review_count	Number of company reviews
company_benefits	Array of benefits (Detail mode only)
search_query	Which search keyword found this job (null for direct URLs)
scraped_at	Timestamp when job was scraped

Example Usage

Multi-keyword search across roles

{
  "searchQueries": ["Software Engineer", "Backend Developer", "DevOps"],
  "location": "Zurich",
  "country": "ch",
  "maxResultsPerQuery": 50,
  "maxResults": 0,
  "includeJobDetails": true
}

Search for remote jobs in Germany

{
  "searchQueries": ["remote developer"],
  "location": "",
  "country": "de",
  "maxResultsPerQuery": 100
}

Quick SERP-only search (no detail pages)

{
  "searchQueries": ["Marketing"],
  "location": "Berlin",
  "country": "de",
  "maxResultsPerQuery": 200,
  "includeJobDetails": false
}

Direct URLs - Still Alive Check

{
  "directUrls": [
    "https://ch.indeed.com/viewjob?jk=abc123def456",
    "https://ch.indeed.com/viewjob?jk=xyz789uvw012"
  ],
  "country": "ch"
}

How It Works

1. Search Phase: For each keyword, crawls Indeed search result pages collecting job cards with basic info
2. Deduplication: Jobs are deduplicated by ID across all keywords
3. Detail Phase (optional): Visits each job's detail page in batches of 50 using fresh crawlers
4. Validation: Each job is validated against the JobListing schema before saving
5. Pagination: Automatically follows pagination up to per-query and total limits

Proxy Recommendations

Indeed has anti-bot protection. For best results:

• Use Residential proxies (required -- datacenter proxies get 403)
• Keep maxResultsPerQuery reasonable (<=200 per query)
• Add delays between runs if scraping frequently

Local Development

# Install dependencies
npm install

# Build TypeScript
npm run build

# Run locally
apify run --purge

# Push to Apify
apify push

Common Use Cases

1. Job market analysis: Track hiring trends across countries with multiple keywords
2. Competitive intelligence: Monitor competitors' hiring across roles
3. Job aggregation: Build job search platforms with comprehensive coverage
4. Still-alive monitoring: Check if previously scraped jobs are still active (Direct URLs mode)
5. Post-deduplication enrichment: Scrape basic data first, then fetch details for new jobs only
6. Career research: Analyze salary ranges and requirements across multiple roles

---

Part of the Santamaria Job Scrapers Suite - Professional-grade job data for the DACH region and beyond.

Need help with integration, aggregation, or custom scraping solutions? Contact us at contact@alessandrosantamaria.com