Indeed Comprehensive Crawler

Overview

The Ultimate Indeed Scraper is a powerful tool designed for the Apify platform, tailored to bypass protections and efficiently extract job listing information directly from Indeed. It's engineered to navigate and parse data from Indeed's job listings and company profiles, offering users a seamless experience in collecting valuable job market insights.

Features

• Customizable Searches: Input URLs directly from Indeed to target specific job searches or categories.
• Comprehensive Data Extraction: Gathers detailed information from job listings, including job titles, company names, locations, salary estimates, job descriptions, and more.
• Company Intelligence:
  • CEO information with photos
  • Company founding year and revenue
  • Employee size range and industry classification
  • Active job categories and locations

How to Use

1. Set Up: Ensure you have an Apify account and access to the Indeed Scraper Browser Automation actor.
2. Input Configuration:
   • Start URLs: Enter your target Indeed job listing URLs. For example: https://de.indeed.com/jobs?q=medizinische+Fachangestellte&sc=0bf%3Aexrec()%2Ck
   • Max jobs to scrape (optional): Set the maximum number of job listings to scrape. Default is 20.
   • Enable company details scraping (new option)
3. Proxy Configuration: Configure proxy settings for enhanced reliability and to avoid IP blocks.
4. Run the Scraper: Launch the scraper on the Apify platform.
5. Data Collection: Extracted data will be available in your preferred format (JSON, CSV, etc.).

Input Example

{
    "startUrls": [
        { "url": "https://de.indeed.com/jobs?q=medizinische+Fachangestellte&sc=0bf%3Aexrec()%2Ck" }
    ],
    "includeCompanyDetails": true,
    "resolveExternalApplyUrl": true,
    "flattenOutput": true,
    "expandToCities": true,
    "maxJobsToScrape": 20,
    "proxyConfiguration": {
        "useApifyProxy": true
    }
}

Input Field Notes

• Resolve External Apply URL (resolveExternalApplyUrl, default true): For jobs that apply on a client site or ATS, the scraper follows Indeed's /vaclk / /rc/clk redirect chain to capture the real destination (Greenhouse, Workday, ADP, etc.) into externalApplyUrl instead of leaving an Indeed click-tracker URL.
• Flatten Output (flattenOutput, default false): Output one flat record per job (nested salary, location, employer, ranking fields lifted to the top level) instead of Indeed's raw nested API response.
• Expand to Cities (expandToCities, default true): When the l= parameter is a country or large region (e.g. Scotland, East of England Region, United Kingdom), the scraper transparently replaces that URL with one per major city and deduplicates the results by jobId. Indeed caps mobile SERP pagination per query, so the union typically yields 2-5× more unique jobs. Supported regions cover the UK (and its constituent countries plus the nine English ITL-1 regions), USA, Canada, Ireland, Australia, Germany, France, Netherlands, Spain, and Italy. A parent radius=0 (Indeed's UI emits this for region chips) is auto-bumped to radius=25 for the per-city queries. Set to false to disable fan-out.

Output Structure

The scraper's output includes detailed information for each job listing. Here's a sample output with explanations for each field:

{
  "source_link": "",  // Original source link if the job is aggregated from another site
  "adBlob": "...",  // Encoded string containing additional job information
  "adId": "433403349",  // Unique identifier for the advertisement
  "additionalLocationLink": {
    "label": "+3 Orte",  // Label for additional locations
    "url": "..."  // URL to view jobs in additional locations
  },
  "advn": "625754421564269",  // Advertiser identification number
  "company": "Ober Scharrer Gruppe GmbH'",  // Name of the company offering the job
  "companyRating": 0,  // Company rating on Indeed (if available)
  "companyReviewCount": 0,  // Number of company reviews on Indeed
  "createDate": 1721054703000,  // Timestamp of when the job was created
  "displayTitle": "MFA / Medizinische Fachangestellte (w/m/d) für den OP",  // Job title as displayed
  "formattedLocation": "76532 Baden-Baden",  // Formatted job location
  "formattedRelativeTime": "vor 30+ Tagen",  // Relative time since job was posted
  "jobkey": "ea2f2dbc75f8a438",  // Unique identifier for the job listing
  "jobTypes": [  // Types of employment offered
    "Vollzeit",
    "Teilzeit"
  ],
  "locationCount": 4,  // Number of locations this job is available in
  "pubDate": 1721019600000,  // Timestamp of when the job was published
  "salarySnippet": {  // Salary information (if available)
    "currency": "EUR",
    "salaryTextFormatted": false
  },
  "snippet": "...",  // Brief excerpt or summary of the job description
  "title": "MFA / Medizinische Fachangestellte (w/m/d) für den OP"  // Job title
}

Field Explanations

• source_link: If the job is aggregated from another site, this field contains the original link.
• adBlob: An encoded string that may contain additional job information.
• adId: A unique identifier for the job advertisement.
• additionalLocationLink: Information about other locations where this job is available.
• advn: The advertiser's identification number.
• company: The name of the company offering the job.
• companyRating: The company's rating on Indeed, if available.
• companyReviewCount: The number of reviews the company has on Indeed.
• createDate: The timestamp when the job listing was created.
• displayTitle: The job title as it's displayed on the listing.
• formattedLocation: The job's location in a formatted string.
• formattedRelativeTime: How long ago the job was posted, in a human-readable format.
• jobkey: A unique identifier for the specific job listing.
• jobTypes: An array of job types (e.g., full-time, part-time) offered for this position.
• locationCount: The number of locations where this job is available.
• pubDate: The timestamp when the job was published.
• salarySnippet: Information about the salary, if provided.
• snippet: A brief excerpt or summary of the job description.
• title: The title of the job position.

📋 Enhanced Output Structure

Company Details Output (New Section)

"companyDetails": {
    "aboutSectionViewModel": {
        "aboutCeo": {
            "name": "Sophie Bellon",
            "photoUrls": {
                "48px": "https://...photo48.jpg",
                "96px": "https://...photo96.jpg",
                "512px": "https://...photo512.jpg"
            }
        },
        "aboutCompany": {
            "founded": 1966,
            "revenue": "RRv1_OVER_10B",
            "employeeRange": "ERv1_10000_PLUS",
            "industry": "Catering & Verpflegungsdienstleistungen",
            "description": "Die Sodexo Group...",
            "headquarters": "Issy-les-Moulineaux"
        }
    },
    "jobsSectionViewModel": {
        "totalJobCount": 65,
        "jobCategories": [
            {
                "displayName": "Reinigungsdienste",
                "jobCount": 47,
                "key": "sanitation"
            }
        ],
        "activeListings": [
            {
                "title": "Reinigungskraft",
                "location": "Wien, W",
                "jobTypes": ["Teilzeit"],
                "postDate": "vor 30+ Tagen"
            }
        ]
    }
}

New Output Fields Explanation

• CEO Details:
  • Name and multiple resolution photos
  • Executive leadership insights
• Company Metadata:
  • Founding year and revenue bracket
  • Employee count range and industry classification
  • Physical headquarters location
• Organizational Insights:
  • Job distribution by category
  • Geographic presence analysis
  • Active job postings catalog

Benefits (Enhanced)

• Competitive Analysis: Compare company hiring patterns and organizational structures
• Talent Acquisition: Identify companies with high hiring activity in specific sectors
• Market Research: Analyze industry distributions and company growth indicators
• Leadership Networking: Access CEO and executive team information

Support

• For issues or feature requests, please use the Issues section of this actor.
• If you need customization or have questions, feel free to contact the author:
  • Author's website: https://muhamed-didovic.github.io/
  • Email: muhamed.didovic@gmail.com
  • My Apify Actors/Scrapers: https://apify.com/memo23

Additional Services

• Request customization or whole dataset: muhamed.didovic@gmail.com
• If you need anything else scraped, or this actor customized, email: muhamed.didovic@gmail.com
• For API services of this scraper (no Apify fee, just usage fee for the API), contact: muhamed.didovic@gmail.com