Indeed Jobs Intelligence Scraper

Scrape Indeed job search results and job detail pages with an HTTP-only Apify Actor. The actor collects job postings, enriches them with structured detail data, normalizes salaries, extracts skills, resolves apply links, and can compare runs to detect new, updated, expired, or reposted jobs.

This actor is built for recruiting teams, job boards, labor market analytics, lead generation, salary benchmarking, and anyone who needs clean Indeed job data without running a browser-based scraper.

What You Get

• Search Indeed by keyword, location, country, radius, date posted, job type, remote preference, and sort order.
• Start from ready-made Indeed search URLs, job URLs, or direct job IDs.
• Fetch full job details, including description, company, salary, benefits, employment type, dates, and source URLs.
• Enrich company profiles from Indeed company pages, including rating, company size, revenue, industry, CEO, links, header image, and company description when available.
• Normalize salary strings into min/max, currency, unit, and annualized values.
• Extract skills from job descriptions and attributes using rule-based skill dictionaries.
• Resolve apply links from Indeed buttons, redirect URLs, and job payloads, then detect common ATS providers such as Workday, Greenhouse, Lever, SmartRecruiters, Taleo, and Indeed Apply.
• Add structured location, requirements, emails, occupation tags, hiring-demand flags, remote flags, and compatibility fields for analytics.
• Deduplicate jobs by job key, URL, title, company, and location.
• Track changes between runs with delta monitoring.
• Produce company-level hiring intelligence and run quality reports.

Output

The default dataset contains one item per job. Job items use a stable output shape: optional strings are emitted as null, optional arrays as [], and optional objects as {}. This prevents Apify tables from showing missing columns as undefined when Indeed does not expose a field for a specific job.

null does not automatically mean a parsing error. It usually means the job or company page did not expose that field. For example, many jobs do not publish salary, some government jobs do not expose an Indeed company logo, and many job descriptions do not contain emails. The actor avoids inventing values: review counts, ratings, and job types such as Full-time are not treated as salary.

Depending on the data available on Indeed, items can include:

{
  "jobKey": "f52b1b3f85fbccca",
  "title": "Senior Python Developer",
  "companyName": "Example Company",
  "location": "New York, NY, US",
  "salaryText": "USD 115000 - USD 180000 per year",
  "salaryMin": 115000,
  "salaryMax": 180000,
  "salaryCurrency": "USD",
  "salaryUnit": "year",
  "salaryAnnualMin": 115000,
  "salaryAnnualMax": 180000,
  "descriptionText": "Full job description...",
  "benefits": ["Health insurance", "Paid time off"],
  "extractedSkills": {
    "programmingLanguages": ["python", "sql"],
    "cloud": ["aws"],
    "all": ["aws", "python", "sql"]
  },
  "locationData": {
    "city": "New York",
    "countryCode": "US",
    "formattedAddressLong": "New York, NY, US"
  },
  "occupation": ["Software Developers", "Computer & Mathematical Occupations"],
  "hiringDemand": {
    "isHighVolumeHiring": false,
    "isUrgentHire": false
  },
  "companyNumEmployees": "10,000+",
  "companyRevenue": "more than $10B (USD)",
  "rating": { "rating": 4.1, "count": 1234 },
  "requirements": ["5+ years of Python experience"],
  "emails": [],
  "applyUrl": "https://jobs.example.com/job/senior-python-developer/123?utm_source=Indeed",
  "externalApplyUrl": "https://jobs.example.com/job/senior-python-developer/123?utm_source=Indeed",
  "finalApplyUrl": "https://jobs.example.com/job/senior-python-developer/123?utm_source=Indeed",
  "atsProvider": "Company Site",
  "applyUrlStatus": "external",
  "detailFetchError": null,
  "jobUrl": "https://www.indeed.com/viewjob?jk=...",
  "canonicalUrl": "https://www.indeed.com/viewjob?jk=...",
  "sourceUrl": "https://www.indeed.com/jobs?q=python+developer&l=New+York"
}

If a search or detail page is blocked after all retries, the actor follows the same defensive pattern as a production scraper: it keeps the run alive and writes a diagnostic error item to the default dataset instead of silently returning an empty dataset.

{
  "type": "error",
  "errorType": "searchPageError",
  "sourceUrl": "https://www.indeed.com/jobs?q=python+developer&l=New+York",
  "searchQuery": "python developer",
  "searchLocation": "New York, NY",
  "page": 1,
  "error": "Blocked by target site, status=403",
  "detectedAt": "2026-05-19T09:57:51.000000+00:00"
}

The actor also writes additional run-scoped datasets:

• companies: company-level hiring summary with open job count, top roles, locations, remote share, average annual salary, and top skills.
• changes: delta monitoring output with new, updated, expired, and reposted jobs.
• summary: run quality report with scraped count, failed pages, duplicate count, missing salary/apply URL rates, retry stats, and runtime.

Input

Use SEARCH mode to scrape jobs by query and location:

{
  "mode": "SEARCH",
  "country": "us",
  "queries": ["python developer", "data analyst"],
  "locations": ["New York, NY", "Remote"],
  "maxResults": 100,
  "postedWithinDays": "7",
  "sort": "date",
  "enrichmentMode": "full",
  "enrichCompanyProfiles": true
}

Use DETAIL mode for known job URLs or job IDs:

{
  "mode": "DETAIL",
  "country": "us",
  "jobIds": ["f52b1b3f85fbccca"],
  "startUrls": [
    { "url": "https://www.indeed.com/viewjob?jk=f52b1b3f85fbccca" }
  ],
  "enrichmentMode": "full",
  "enrichCompanyProfiles": true
}

Main Settings

• country: Indeed country/domain, such as us, gb, ca, au, de, fr, in, and others.
• queries: keywords or job titles.
• locations: locations combined with each query.
• startUrls: Indeed search URLs or job detail URLs.
• jobIds: Indeed job keys to fetch directly.
• maxResults: maximum jobs to output.
• maxPagesPerSearch: pagination safety limit per search.
• postedWithinDays: date filter, for example 1, 3, 7, 14, or 30.
• jobType: full-time, part-time, contract, temporary, internship, or commission.
• remote: remote, hybrid, or on-site preference.
• enrichmentMode: fast for search cards only or full for job detail enrichment.
• dedupe: skip duplicate jobs.
• normalizeSalary: add structured salary fields.
• extractSkills: add rule-based skill extraction.
• resolveApplyUrls: follow apply redirects when available.
• enrichCompanyProfiles: fetch one Indeed company page per unique company URL and merge company profile fields.
• enableDeltaMonitoring: compare current jobs with the previous run.
• proxyConfiguration: Apify proxy settings. Residential proxies are recommended.

Proxy Recommendation

Indeed can return security checks to datacenter or low-trust IPs. For reliable runs, use Apify Residential proxies and match the proxy country to the Indeed domain:

{
  "useApifyProxy": true,
  "apifyProxyGroups": ["RESIDENTIAL"],
  "apifyProxyCountry": "US"
}

Technical Notes

This actor is HTTP-only. It does not run Playwright, Puppeteer, Selenium, or a headless browser during production scraping. Browser automation is used only for research and debugging outside the actor.

The HTTP client uses Chrome-like TLS/HTTP2 impersonation via curl_cffi, short request timeouts, retry budgets for network errors and blocks, and fresh sessions for retries.

Limitations

• Available fields depend on what Indeed returns for each job, country, and page type.
• Some apply links stay inside Indeed Apply and may not expose an external ATS URL.
• Salary normalization is best-effort and depends on the salary text or structured salary data available on the page.
• Delta monitoring compares jobs seen by this actor across runs. If a run collects no jobs, the previous state is not overwritten.

Local Development

Run the local parser smoke test:

$python scripts/smoke_test.py

Deploy with Apify CLI:

$deploy.bat