# Haystack Jobs Scraper (`msalmanshah/haystack-jobs-scraper`) Actor

Scrapes tech job listings from haystackapp.io by keyword, location and filters.

- **URL**: https://apify.com/msalmanshah/haystack-jobs-scraper.md
- **Developed by:** [Muhammad Salman Shah](https://apify.com/msalmanshah) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 job scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Haystack Jobs Scraper

An Apify Actor (Node.js / Crawlee) that scrapes tech job listings from
[haystackapp.io](https://haystackapp.io) by keyword, location and filters.

### How it works

haystackapp.io has two very different rendering modes, so this Actor uses a
**hybrid crawler**:

1. **Playwright crawler** opens the filterable search page
   (`https://haystackapp.io/jobs?q=...&location=...`). This page's initial
   HTML is empty ("0 jobs found") — the real results are fetched by React
   after the page loads — so a real browser is required just to collect the
   list of job URLs.
2. **Cheerio crawler** then fetches each individual job page
   (`https://haystackapp.io/jobs/{uuid}`) with plain HTTP requests. These
   pages *are* fully server-rendered, so a lightweight, fast HTTP crawler is
   enough to extract full job details (title, description, skills, apply
   link, etc.) — no browser needed for this part.

This keeps the Actor fast and cheap: only one browser page is ever opened,
no matter how many jobs you scrape.

### Input

| Field | Type | Description |
|---|---|---|
| `keyword` | string | Job title / keyword, e.g. `"Software Engineer"` |
| `location` | string | City, e.g. `"London"` |
| `country` | select | Restrict to one country |
| `workMode` | select | `onsite`, `hybrid`, or `remote` |
| `maxItems` | integer | Max number of jobs to return (default 50) |
| `proxyConfiguration` | object | Apify Proxy config (recommended) |

### Output

One dataset item per job, e.g.:

```json
{
  "url": "https://haystackapp.io/jobs/785dd571-1826-4dbe-8008-7596b2e5d430",
  "title": "Lead Software Engineer - Proxy/SSE Network Security",
  "headerRaw": "J.P. Morgan London, UK Posted 24 Jul 2026",
  "postedDate": "24 Jul 2026",
  "workType": "On Site",
  "level": "Mid Senior",
  "skills": ["AWS", "Encryption", "OAuth", "SAML", "Agile", "DNS", "HTTP", "HTTPS", "JWT", "SAFe", "Zero Trust"],
  "description": "...",
  "applyUrl": "https://.../track-redirect?job_id=...",
  "scrapedAt": "2026-08-02T12:00:00.000Z"
}
```

### ⚠️ Before you run this in production

I built and structured this Actor entirely from the site's **server-rendered
HTML**, fetched via a text-based tool — I was not able to render the page in
a real browser to inspect it with devtools during development. That means:

- The job-link discovery selector (`a[href*="/jobs/"]` filtered by a UUID
  regex) is solid — it's based on the actual URLs the site returns.
- The **"load more" / pagination handling** on the search page is a
  best-effort guess (it tries clicking anything that looks like a "load
  more" button, and also scrolls, in case it's infinite-scroll instead).
  This is the part most likely to need adjustment.
- The **detail-page field extraction** (`company`, `workType`, `level`)
  uses text-pattern heuristics rather than confirmed CSS classes, since I
  couldn't inspect the live DOM's class names. `title`, `skills`,
  `description`, and `applyUrl` are extracted more robustly and should be
  reliable.

**Recommended next step:** run the Actor once with `maxItems: 5` on the
Apify platform (or locally with `apify run`), open the dataset, and compare
a couple of records against the live pages. If `company` comes back `null`
or `workType`/`level` look off, open devtools on a job page, find the real
selectors, and I'll tighten the Cheerio parsing in `src/main.js` — that's a
quick fix once we can see real output.

### Local development

```bash
npm install
apify run
```

(Requires the [Apify CLI](https://docs.apify.com/cli). Input is read from
`storage/key_value_stores/default/INPUT.json` — create one based on the
input schema above, or set input via `apify run` on the platform.)

### Deploy to Apify

```bash
apify login
apify push
```

# Actor input Schema

## `keyword` (type: `string`):

Search term, e.g. "Software Engineer", "Product Manager". Leave empty to browse all jobs (combine with location/filters).

## `location` (type: `string`):

City to search in, e.g. "London", "Berlin". Leave empty for all locations.

## `country` (type: `string`):

Restrict search to one country.

## `workMode` (type: `string`):

Filter by on-site, hybrid or remote roles.

## `maxItems` (type: `integer`):

Maximum number of job listings to return.

## `proxyConfiguration` (type: `object`):

Recommended: use Apify Proxy so requests don't come from a single IP.

## Actor input object example

```json
{
  "keyword": "Software Engineer",
  "location": "London",
  "country": "",
  "workMode": "",
  "maxItems": 50,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `jobs` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "keyword": "Software Engineer",
    "location": "London"
};

// Run the Actor and wait for it to finish
const run = await client.actor("msalmanshah/haystack-jobs-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "keyword": "Software Engineer",
    "location": "London",
}

# Run the Actor and wait for it to finish
run = client.actor("msalmanshah/haystack-jobs-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "keyword": "Software Engineer",
  "location": "London"
}' |
apify call msalmanshah/haystack-jobs-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,msalmanshah/haystack-jobs-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/7dMGWzONkanq3hM1Y/builds/fefWIoTsUY0m98fnK/openapi.json