> A lightweight, anti-bot-resilient Apify Actor that scrapes public LinkedIn job postings at scale — without a browser and without login.
`Apify Actor` · `TypeScript` · `Crawlee` · `CheerioCrawler` · `HTTP-first` · `Residential Proxy`
The **LinkedIn Jobs Scraper** collects job listings from LinkedIn's public guest endpoints using a fast, HTTP-only crawler. It runs in **guest mode by default** (no credentials required), optionally accepts an `li_at` cookie for richer data, and uses **query segmentation** to break past LinkedIn's ~1,000-results-per-query limit. Results are streamed directly to the dataset, keeping runs light on memory and compute.
---
### Table of Contents
- [Features](#features)
- [How It Works](#how-it-works)
- [Input Reference](#input-reference)
- [Example Input](#example-input)
- [Example Output](#example-output)
- [Guest Mode vs. `li_at` Cookie Mode](#guest-mode-vs-li_at-cookie-mode)
- [Anti-Bot & Rate-Limit Strategy](#anti-bot--rate-limit-strategy)
- [Performance & Cost](#performance--cost)
- [Limitations](#limitations)
- [Troubleshooting](#troubleshooting)
- [Optional Playwright Fallback](#optional-playwright-fallback)
- [Running Locally & Deploying](#running-locally--deploying)
- [Legal & Compliance](#legal--compliance)
---
### Features
- **Lightweight, HTTP-first crawling.** Built on [Crawlee](https://crawlee.dev/)'s `CheerioCrawler`, which fetches and parses raw HTML without launching a headless browser. This makes runs dramatically faster and cheaper than browser-based scrapers. `CheerioCrawler` uses `got-scraping` internally, so realistic browser-like headers and TLS fingerprints are applied automatically.
- **No-login guest mode.** By default the Actor targets LinkedIn's **public guest endpoints** and requires no account or credentials.
- **Optional `li_at` cookie.** Provide a session cookie to unlock richer, more complete data. The cookie is treated as a secret and **is never written to logs or the dataset**.
- **Anti-bot resilience.** Combines Apify **Residential Proxy** with an automatic **session pool** that rotates IPs on rate-limit, soft-ban, or CAPTCHA signals — plus retries with exponential backoff and configurable request throttling.
- **Query segmentation to exceed the ~1,000-result cap.** Automatically splits a single search into multiple filtered sub-queries (by posted time, experience level, and work type) so you can collect far more than the ~1,000 results LinkedIn returns for any single query. Global de-duplication by `jobId` prevents duplicates across segments.
- **Full job fields.** Extracts a comprehensive set of fields including title, company, location, salary, seniority, employment type, job function, industries, applicant count, and the full job description (HTML and plain text).
- **Streaming output.** Each job is pushed to the dataset as soon as it is parsed (`Actor.pushData`), so memory stays flat even on very large runs.
---
### How It Works
The Actor uses a **two-stage architecture** driven by a Crawlee `RequestQueue`:
1. **Search list stage (`SEARCH_LIST`).** The Actor requests LinkedIn's guest job-search endpoint, parses each job card, extracts the `jobId`, and enqueues a detail request for it. It then enqueues the next page of results (`start += 25`) until it runs out of results, reaches `maxResults`, or hits the per-segment page limit.
2. **Detail stage (`JOB_DETAIL`).** For each queued job, the Actor fetches the guest job-posting endpoint and parses the full detail (description, criteria, seniority, etc.), then streams the finished record to the dataset. If `includeJobDescription` is `false`, records are pushed directly from the list stage and the detail stage is skipped for speed.
#### Segmentation Strategy (in plain terms)
LinkedIn only returns roughly the first **1,000 results** for any single search, no matter how many jobs actually match. To get more, the Actor splits one broad search into many narrower ones that together cover the same ground:
1. First it segments by **posted time** (e.g. past 24 hours / past week / past month).
2. Then, within each of those, it segments by **experience level** (6 levels).
3. Then, within each of those, it segments by **work type** (on-site / remote / hybrid).
Each combination is its own search with its own pagination, so each one gets its own ~1,000-result budget. Because different segments can surface the same job, a global **de-duplication set keyed on `jobId`** ensures every job is emitted only once. Segmentation is enabled by default and can be turned off with `enableSegmentation: false`.
---
### Input Reference
Every input field is defined in the Actor's input schema. All fields except `keywords` are optional.
| Field | Type | Default | Description |
|---|---|---|---|
| `keywords` | string | *(required)* | Search terms for the job title/role, e.g. `"software engineer"`. |
| `location` | string | `"United States"` | Geographic location to search within, e.g. a country, city, or region. |
| `maxResults` | integer | `100` | Maximum number of jobs to collect. Use `0` for no explicit cap (bounded by LinkedIn availability and segmentation). |
| `timePosted` | string (enum) | `"any"` | Filter by posting recency. One of `any`, `24h`, `week`, `month`. Maps to LinkedIn's `f_TPR`. |
| `experienceLevel` | array of string (enum) | `[]` | Filter by seniority. Values: `internship`, `entry-level`, `associate`, `mid-senior`, `director`, `executive`. Maps to `f_E`. |
| `jobType` | array of string (enum) | `[]` | Filter by employment type. Values: `full-time`, `part-time`, `contract`, `temporary`, `volunteer`, `internship`. Maps to `f_JT`. |
| `workSchedule` | array of string (enum) | `[]` | Filter by work arrangement. Values: `on-site`, `remote`, `hybrid`. Maps to `f_WT`. |
| `easyApplyOnly` | boolean | `false` | If `true`, restrict results to Easy Apply jobs. Maps to `f_AL`. |
| `includeJobDescription` | boolean | `true` | If `true`, fetch each job's detail page for the full description and extra fields. Set to `false` for faster, lighter list-only runs. |
| `enableSegmentation` | boolean | `true` | If `true`, split the search into filtered sub-queries to exceed the ~1,000-result-per-query cap. |
| `liAtCookie` | string (secret) | `""` | Optional LinkedIn `li_at` session cookie for richer data. Treated as a secret and never logged or stored. |
| `proxyConfiguration` | object | `{ "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] }` | Apify proxy configuration. Residential proxies are strongly recommended for reliability. |
| `maxConcurrency` | integer | `50` | Maximum number of requests processed in parallel. |
| `maxRequestRetries` | integer | `3` | Maximum retry attempts per request before it is marked failed. |
| `maxRequestsPerMinute` | integer | `200` | Global throttle on request rate to reduce the chance of rate limiting. |
---
### Example Input
```json
{
"keywords": "software engineer",
"location": "United States",
"maxResults": 200,
"timePosted": "week",
"experienceLevel": ["mid-senior", "associate"],
"jobType": ["full-time"],
"workSchedule": ["remote", "hybrid"],
"easyApplyOnly": false,
"includeJobDescription": true,
"enableSegmentation": true,
"proxyConfiguration": {
"useApifyProxy": true,
"apifyProxyGroups": ["RESIDENTIAL"]
},
"maxConcurrency": 50,
"maxRequestRetries": 3,
"maxRequestsPerMinute": 200
}
````
***
### Example Output
Each dataset item represents one job posting:
```json
{
"jobId": "3801234567",
"title": "Senior Software Engineer",
"company": "Acme Corporation",
"companyUrl": "https://www.linkedin.com/company/acme-corp",
"location": "San Francisco, CA",
"workplaceType": "Hybrid",
"salary": "$150,000 - $190,000/yr",
"postedAt": "2026-07-02T00:00:00.000Z",
"jobUrl": "https://www.linkedin.com/jobs/view/3801234567",
"description": "We are looking for a Senior Software Engineer to join our platform team. You will design, build, and operate scalable backend services...",
"descriptionHtml": "We are looking for a Senior Software Engineer to join our platform team. You will design, build, and operate scalable backend services...
", "seniorityLevel": "Mid-Senior level", "employmentType": "Full-time", "jobFunction": ["Engineering", "Information Technology"], "industries": ["Software Development", "Technology, Information and Internet"], "applicantsCount": 47, "easyApply": false, "criteria": [ { "name": "Seniority level", "value": "Mid-Senior level" }, { "name": "Employment type", "value": "Full-time" }, { "name": "Job function", "value": "Engineering and Information Technology" }, { "name": "Industries", "value": "Software Development" } ], "scrapedAt": "2026-07-09T12:34:56.000Z" } ``` #### Output Fields | Field | Type | Description | |---|---|---| | `jobId` | string | LinkedIn's unique identifier for the posting. | | `title` | string | Job title. | | `company` | string | Hiring company name. | | `companyUrl` | string | null | LinkedIn URL of the company page. | | `location` | string | Job location as displayed by LinkedIn. | | `workplaceType` | string | null | On-site, Remote, or Hybrid. | | `salary` | string | null | Salary text if published. | | `postedAt` | string (ISO 8601) | null | When the job was posted. | | `jobUrl` | string | Public URL of the job posting. | | `description` | string | null | Full job description as plain text. | | `descriptionHtml` | string | null | Full job description as HTML. | | `seniorityLevel` | string | null | Seniority level. | | `employmentType` | string | null | Employment type (Full-time, Contract, etc.). | | `jobFunction` | array of string | Job function categories. | | `industries` | array of string | Relevant industries. | | `applicantsCount` | integer | null | Number of applicants, when available. | | `easyApply` | boolean | Whether the posting supports LinkedIn Easy Apply. | | `criteria` | array of object | Name/value pairs of the job criteria block. | | `scrapedAt` | string (ISO 8601) | Timestamp when the record was scraped. | The dataset ships with two views: - **overview** — a compact table: `title`, `company`, `location`, `postedAt`, `jobUrl`. - **detailed** — all fields above. *** ### Guest Mode vs. `li_at` Cookie Mode #### Guest Mode (default) The Actor targets LinkedIn's **public guest endpoints** and needs no account. This is the recommended starting point: - No credentials to manage and no risk to a personal account. - Fully covers the core fields most users need. - Lower friction and simpler to run reliably. #### `li_at` Cookie Mode (optional) Supplying a valid `li_at` session cookie can unlock **richer and more complete data** on some pages (for example, more consistent applicant counts or fields that guest pages omit). > **Security warning.** A `li_at` cookie is equivalent to being logged into that LinkedIn account. Handle it with care: > > - **Use a secondary / throwaway account**, never your primary personal account. Automated access may trigger account restrictions or bans. > - The cookie field is marked as a **secret**; its value is **never printed to logs and never stored in the dataset** (it is redacted as `[REDACTED]` everywhere). > - Cookies expire. If richer fields stop appearing, refresh the cookie (see [Troubleshooting](#troubleshooting)). > - Never commit cookies to source control or share them. *** ### Anti-Bot & Rate-Limit Strategy LinkedIn actively defends against automated access. This Actor uses several layers to stay resilient: - **Residential proxy rotation.** By default it routes traffic through Apify's `RESIDENTIAL` proxy group, which presents real consumer IPs that are far less likely to be blocked than datacenter IPs. - **Session pool with automatic retirement.** A pool of sessions is maintained; when a session encounters a **429 (rate limit)**, **999 (soft ban)**, or a **CAPTCHA / challenge** response, that session is retired so the next request uses a fresh IP and identity. - **Retries with exponential backoff.** Failed requests are retried up to `maxRequestRetries` times, with increasing delays to avoid hammering the target. - **Request throttling.** `maxRequestsPerMinute` caps the global request rate, and `maxConcurrency` bounds parallelism — both help you stay under LinkedIn's radar. - **Realistic fingerprints.** `CheerioCrawler` (via `got-scraping`) applies browser-like headers and TLS fingerprints automatically. - **Defensive parsing.** Every field is extracted with null-safe helpers wrapped in per-card `try/catch`, so a markup change or a missing field yields `null`/defaults instead of failing an entire page. If you still see frequent blocks, lower `maxConcurrency` and `maxRequestsPerMinute`, and keep Residential proxies enabled. *** ### Performance & Cost This Actor is designed to be **lightweight ("ringan")**: because it uses `CheerioCrawler` (raw HTTP + HTML parsing) instead of a headless browser, it consumes far fewer compute units (CUs) than browser-based alternatives. Practical guidance for keeping usage light: - **Start small.** Set a modest `maxResults` (e.g. `50`–`200`) for your first runs to validate your inputs before scaling up. - **Skip detail pages when you can.** Set `includeJobDescription: false` to collect list-level fields only — this roughly halves the number of requests and is significantly faster and cheaper. - **Disable segmentation for small jobs.** If you only need a few hundred results, set `enableSegmentation: false` to avoid issuing many sub-queries. - **Tune concurrency to your needs.** Lower `maxConcurrency` and `maxRequestsPerMinute` reduce peak resource usage and block risk; raise them (carefully) for throughput. - **Proxy costs.** Residential proxy traffic is billed separately from compute. Fetching detail pages increases proxy usage, so `includeJobDescription: false` also reduces proxy spend. > Actual compute units depend on result volume, whether detail pages are fetched, and retry rates. Monitor your first few runs in the Apify console to calibrate. *** ### Limitations - **~1,000 results per query cap.** LinkedIn returns at most ~1,000 results for any single search. The Actor mitigates this with **query segmentation** (enabled by default), but the total reachable set is still bounded by the filter combinations available. - **JS-rendered content.** Some content is rendered client-side and may not appear in the raw HTML served to an HTTP client. Such fields may be `null`. For those cases, see the [Optional Playwright Fallback](#optional-playwright-fallback). - **Markup changes.** LinkedIn changes its markup periodically, which can break selectors. Parsing is defensive and selectors are centralized to ease updates, but occasional maintenance may be required. - **Terms of Service.** Scraping LinkedIn may conflict with LinkedIn's Terms of Service. You are responsible for how you use this Actor — see [LEGAL\_NOTES.md](./LEGAL_NOTES.md). *** ### Troubleshooting **Getting 429 / 999 / CAPTCHA responses or frequent retries** - Ensure `proxyConfiguration` uses Apify **Residential** proxies. - Lower `maxConcurrency` (e.g. to `10`–`20`) and `maxRequestsPerMinute` (e.g. to `60`–`100`). - Increase `maxRequestRetries` slightly to give sessions more chances to recover. **Empty or very few results** - Verify `keywords` and `location` are valid and not overly narrow. - Relax filters (`timePosted`, `experienceLevel`, `jobType`, `workSchedule`, `easyApplyOnly`). - Confirm the proxy is working (check run logs for proxy errors). - Try again later — transient blocking can temporarily reduce results. **Missing detail fields (description, applicants, etc.)** - Ensure `includeJobDescription: true`. - Some fields are only reliably available with a valid `li_at` cookie — consider cookie mode. **Cookie no longer works / richer fields disappeared** - `li_at` cookies expire. Log in with your secondary account, copy a fresh `li_at` value, and update the `liAtCookie` input. **Fewer results than expected on large searches** - Make sure `enableSegmentation: true`. - Remember the ~1,000-per-query cap; add more specific filters to create additional segments. *** ### Optional Playwright Fallback For the rare pages whose content is only available after client-side JavaScript execution, a **Playwright-based browser fallback** could be added as an escape hatch. This is **documented but not implemented by default**, because a headless browser is heavier, slower, and more expensive in compute units — which conflicts with this Actor's lightweight goal. If you need it, the intended approach is to swap `CheerioCrawler` for `PlaywrightCrawler` (or add a secondary crawler) for the specific requests that require rendering, keeping the same proxy, session, and parsing strategy. Enable this only when HTTP-first parsing genuinely cannot retrieve the data you need. *** ### Running Locally & Deploying #### Prerequisites - [Node.js 20+](https://nodejs.org/) - [Apify CLI](https://docs.apify.com/cli): `npm install -g apify-cli` - An Apify account and API token for proxy access: `apify login` #### Run locally ```bash ## Install dependencies npm install ## Build (TypeScript -> JavaScript) npm run build ## Provide your input in ./storage/key_value_stores/default/INPUT.json ## then run the Actor locally apify run ``` Local run output is written to `./storage/datasets/default/`. #### Deploy to the Apify platform ```bash ## Push the Actor to your Apify account apify push ``` Once deployed, you can run the Actor from the Apify console, via the API, or on a schedule. *** ### Legal & Compliance Please read [LEGAL\_NOTES.md](./LEGAL_NOTES.md) before using this Actor. It covers LinkedIn's Terms of Service, `robots.txt`, GDPR and personal-data handling, ethical scraping practices, and an important disclaimer. **You are solely responsible for ensuring your use complies with all applicable laws and terms.** # Actor input Schema ## `keywords` (type: `string`): Job search keywords (e.g. job title, skill, company). ## `location` (type: `string`): Geographic location to search within (e.g. "United States", "Berlin, Germany"). ## `maxResults` (type: `integer`): Maximum number of job postings to scrape. Set to 0 for no limit (bounded only by LinkedIn availability & segmentation). ## `timePosted` (type: `string`): Filter jobs by how recently they were posted. ## `experienceLevel` (type: `array`): Filter jobs by required experience level (maps to LinkedIn f\_E). ## `jobType` (type: `array`): Filter jobs by employment type (maps to LinkedIn f\_JT). ## `workSchedule` (type: `array`): Filter jobs by workplace type / work schedule (maps to LinkedIn f\_WT). ## `easyApplyOnly` (type: `boolean`): Only return jobs that support LinkedIn Easy Apply (maps to f\_AL). ## `includeJobDescription` (type: `boolean`): Fetch the full detail page for each job (description, criteria, etc.). Disabling is faster but returns list-level fields only. ## `enableSegmentation` (type: `boolean`): Split the search into multiple segmented queries (by posted time, experience level, workplace type) to bypass LinkedIn's ~1000 results-per-query cap. ## `liAtCookie` (type: `string`): Optional LinkedIn li\_at session cookie for richer data. Its value is never logged or stored in the dataset. ## `proxyConfiguration` (type: `object`): Proxy settings. Residential Apify Proxy is strongly recommended to reduce blocking. ## `maxConcurrency` (type: `integer`): Maximum number of concurrent requests. ## `maxRequestRetries` (type: `integer`): Maximum number of retries per request on failure (e.g. blocking, network error). ## `maxRequestsPerMinute` (type: `integer`): Throttle the crawler to at most this many requests per minute. ## Actor input object example ```json { "keywords": "software engineer", "location": "United States", "maxResults": 100, "timePosted": "any", "experienceLevel": [], "jobType": [], "workSchedule": [], "easyApplyOnly": false, "includeJobDescription": true, "enableSegmentation": true, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] }, "maxConcurrency": 50, "maxRequestRetries": 3, "maxRequestsPerMinute": 200 } ``` # Actor output Schema ## `dataset` (type: `string`): The dataset containing all scraped job listings. ## `summary` (type: `string`): The JSON summary of this run containing execution stats. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '