AI Job Search Agent — Web Job Finder (BYOK)

An AI agent (BYO API key — Anthropic or Mistral) that hunts the open web for postings matching your query — company career pages, niche boards, anywhere a single job board wouldn't cover.

Bring your own key. Pick a provider in the input: anthropic (default) uses Claude with its built-in web-search tool; mistral uses keenable (no-auth open-web search) for discovery plus a Mistral model for per-page judging/extraction. Leave provider unset and just supply whichever key you have — the Actor auto-selects the matching provider. Either way, model usage is billed separately by the provider you picked, see Pricing below. Without a valid key for the selected provider the run still succeeds — it produces a single dataset row explaining how to supply a key, instead of failing.

Two providers, same output

	provider: "anthropic" (default)	provider: "mistral"
Discovery	One Claude agent loop forms its own queries, browses, and judges results (built-in web_search_20250305 tool)	keenable web search (no key needed) — queries built mechanically from your keywords/titleMustMatch
Page judging + extraction	Same agent loop, inline	One Mistral call per candidate page — vetoes index/aggregator/closed/stale pages, extracts the rest
Key required	anthropicApiKey	mistralApiKey
Output shape	Identical — same fields, same downstream liveness check either way

Pick mistral if you'd rather not hold an Anthropic key, or want a cheaper/faster run — testing found it succeeds more reliably per attempt and finishes faster when Claude's agent loop does too, at a small precision cost the extraction prompt corrects for (rejects closed/stale/third-party-repost pages explicitly).

What this scraper returns

Each result is one flat JSON record per job posting. Before a result is returned, its URL is checked for basic liveness (see Verification below).

Field	Meaning
id	Stable identifier derived from the posting URL
source	Always "web_search" for this Actor
title	Job title as posted
company	Hiring company / organisation
location	Location / duty station (may include remote hints)
url	Direct link to the posting
postedAt	Posting date where the source provides it (YYYY-MM-DD)
snippet	Short description excerpt
salary	Salary / compensation text exactly as stated on the posting (e.g. "$120k–$150k", "Competitive"), extracted by the discovery/extraction LLM. null when the posting states no salary — the agent never guesses a figure
matchScore	LLM-judged 0–100 rating of how well the posting matches your profile (keywords, titles, locations, remote, seniority, and your free-text userDescription). Results are sorted best-match-first. null when the model returned no usable score
matchReasoning	One short sentence from the LLM explaining the matchScore
isNew	true only on delta runs (onlyNewSinceLastRun), marking a posting not seen in a previous run. Absent on normal runs
verified	true when the liveness check confirmed the URL reachable at run time; false when the check couldn't complete (timeout, connection error) but the result was kept (see below)

How to scrape AI job search with this Actor

1. Click Try for free / Run — no login to the target site, no cookies, no proxies to configure.
2. Adjust the input (keyword, filters, maxItems) or keep the defaults.
3. Run it and export the dataset as JSON, CSV or Excel, or read it over the API.

A typical run takes 1–3 minutes (the agent performs up to 10 live web searches). If you lower the run timeout, allow at least 300 seconds so the run isn't killed mid-search.

Run it from your own code:

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("nomad-agent/web-search-scraper").call(run_input={"maxItems": 15, "anthropicApiKey": "sk-ant-..."})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["title"], "—", item["company"], item["url"])

Or a single HTTP call that runs the Actor and returns items in one response:

curl -X POST \
  "https://api.apify.com/v2/acts/nomad-agent~web-search-scraper/run-sync-get-dataset-items?token=<YOUR_APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"maxItems": 15, "anthropicApiKey": "sk-ant-..."}'

Verification

"Verified" means one thing only: the posting URL is HTTP-reachable and does not redirect to the site's homepage/root. After the discovery agent proposes candidate postings, this Actor issues a HEAD (falling back to GET) request to each URL with a 10-second timeout and modest concurrency. A result is dropped only when:

• the final response is 404 or 410 (the posting is gone), or
• the URL redirects to the site's root path (/ or empty path) — a common pattern for expired listings.

Anything else — a normal 200, a 403/5xx from a bot-defensive site, a timeout, or a connection error — is kept, since those don't reliably distinguish "blocked us" from "actually dead." Results kept because the check couldn't complete (timeout/connection error) carry verified: false so you can tell them apart from confirmed-reachable ones. This is a structural HTTP liveness check, not a claim about listing accuracy, freshness, or relevance — that judgment still comes from the discovery agent and any downstream scoring you apply.

Run-level counts (rawPostings, titleExcluded, domainExcluded, urlsChecked, kept, dropped, deltaSkipped) are written to the run's key-value store under VERIFICATION_STATS and logged at the end of each run.

Input

Field	Type	Default	Notes
provider	string (select)	"anthropic"	anthropic (Claude + built-in web search) or mistral (keenable web search + Mistral extraction). See Two providers above.
anthropicApiKey	string	— (required when provider: "anthropic")	Your Anthropic API key (sk-ant-…).
model	string	"claude-haiku-4-5-20251001"	Claude model for the discovery agent (provider: "anthropic" only). Haiku is fast and inexpensive; switch to Sonnet or Opus for higher-quality extraction on complex pages.
mistralApiKey	string	— (required when provider: "mistral")	Your Mistral API key. Used only for per-page judging/extraction — the search itself (keenable) needs no key.
mistralModel	string (select)	"mistral-medium-latest"	Mistral model for per-page judging/extraction (provider: "mistral" only). Testing found Small matches Medium on quality for this task at ~5x the speed — switch to Small for faster/cheaper runs.
keywords	array	—	Role or technology keywords the agent should search for (e.g. "frontend", "react", "typescript"). Each entry is one keyword or short phrase.
locations	array	—	Preferred locations or "remote". Biases results toward these and skips obvious mismatches.
remote	string (select)	"any"	Remote work preference: any, remote-only, hybrid, on-site. provider: "anthropic" only — the mistral path doesn't currently read this field.
seniority	string (select)	"any"	Target seniority level: any, junior, mid, senior, lead. provider: "anthropic" only — the mistral path doesn't currently read this field.
titleMustMatch	array	—	Both providers: preferred/searched title terms. On mistral, used to build the keenable search queries when keywords is empty.
titleExclude	array	—	Postings whose title contains any of these terms are dropped — enforced client-side after discovery, both providers.
maxItems	integer	15	Maximum number of job postings to return (1–30). Either provider may return fewer if it can't find enough good candidates, or if some fail the liveness check.
maxAgeHours	integer	168	Preferred maximum age of postings in hours (minimum 24). provider: "anthropic" only — the mistral path's extraction prompt has its own fixed ~30-day freshness cutoff instead.
userDescription	string	—	Free-text description of what you are looking for. Primary signal for both discovery and the per-posting matchScore/matchReasoning. Read by both providers' scoring step.
onlyNewSinceLastRun	boolean	false	Delta / monitoring mode. Only outputs postings not seen in a previous run that also had this flag on; already-seen postings are dropped before push (not billed). State is tracked per Actor in a dedicated key-value store, keyed by each posting's id. New records carry isNew: true. See Delta mode.

Note on seniority: this field used to accept free text and now uses a fixed dropdown (any / junior / mid / senior / lead). If you have a saved input configuration with an old free-text value that isn't one of these options, it will fail validation on the next run — open the input and re-select from the dropdown.

Delta mode / monitoring

Turn on onlyNewSinceLastRun to only get postings you haven't seen before. Each run records the id of every posting it emits (in a dedicated per-Actor key-value store); a later run with the flag on drops any posting whose id was already recorded, before it is pushed or billed. New postings carry isNew: true.

This is the cheapest way to run the Actor on a schedule: pay only for genuinely new matches. State survives across runs and is trimmed to the most recent 50,000 ids. If the state store can't be opened for some reason, the run continues normally without dedup rather than failing.

Match scoring & ranking

Every posting is scored 0–100 (matchScore) by the same LLM that judges it, against your keywords, titleMustMatch, locations, remote, seniority, and — most importantly — your free-text userDescription. A one-sentence matchReasoning explains each score. Results in the dataset are sorted best-match-first. This is prompt-driven, not a hardcoded rule set, so it weighs role content in context rather than matching exact title tokens.

Output example

{
  "id": "ws-3f9a1c2b7d",
  "source": "web_search",
  "title": "Computational Linguist",
  "company": "DeepJudge",
  "location": "Zurich / Remote EU",
  "url": "https://deepjudge.ai/careers/computational-linguist",
  "postedAt": "2026-06-25",
  "snippet": "Found on company careers page during agent web search.",
  "salary": "CHF 120,000–150,000 / year",
  "matchScore": 88,
  "matchReasoning": "Strong NLP/linguistics fit, remote-EU friendly, senior level as requested.",
  "verified": true
}

Integrations

Send results straight to Google Sheets, Slack, Make, Zapier or any webhook via Apify integrations — no code required, or pull the dataset over the API.

Pricing

This is a BYOK Actor, so a run costs you two things: what Apify charges, and what your own LLM provider charges. Both are listed here — no surprises on your provider bill.

1. Apify side — pay per event: $0.005 per Actor start and $0.003 per job returned (tiered down to $0.0023 on higher Apify plans). 100 jobs ≈ $0.31. No subscription, no rental — you pay only for what you fetch.

2. Your provider key — billed to you directly, not through Apify:

Provider	What your key pays for	Typical cost per run
mistral	Extraction only. Web search runs through keenable, which needs no key and costs you nothing. Your key pays only for the small per-page judging calls.	~$0.01
anthropic	The Claude agent loop plus Anthropic's built-in web-search tool, which Anthropic bills at $10 per 1,000 searches. This Actor caps the agent at 6 searches per run (~$0.06), plus tokens.	~$0.06–0.10

Pick mistral to keep costs down — it is roughly 6–10× cheaper per run and needs no paid search tool. Pick anthropic when you want Claude's agent loop to form and refine its own queries and are happy to pay for it. See Anthropic's pricing and Mistral's pricing.

Use cases

• Long-tail job discovery beyond big boards
• Passive-candidate tooling (find who hires for X)
• Niche-role hunting (rare stacks, rare titles)
• Backfilling gaps in board coverage

FAQ

Is it legal to scrape AI job search? This Actor reads only publicly available job postings — data any visitor can see without logging in. No personal data behind authentication is touched. Review the target site's terms and your local regulations for your specific use case.

Do I need an account on the target site? No. Postings are discovered via web search and fetched from public pages — no login, cookies or session tokens.

Does "verified" mean the listing is accurate or still accepting applications? No. It means the URL was HTTP-reachable and did not redirect to the site's homepage at check time (see Verification). A posting can still have closed between the check and when you view it.

How many jobs can I get? maxItems caps the run between 1 and 30 (default 15). The agent may return fewer than the cap if it can't find enough good candidates or if some fail the liveness check.

Does it extract salary? Yes. The discovery/extraction LLM copies the salary/compensation text exactly as stated on the posting (e.g. "$120k–$150k / year", "Competitive"). When a posting states no salary, salary is null — the agent never guesses or invents a figure.

Something broken or missing? Open an issue on the Actor's Issues tab — it is monitored and reliability fixes ship fast.

Is this Actor useful to you? A quick ⭐ review on the Actor's Reviews tab helps other search-data users find it — and tells us what to build next.

Related Actors

• LinkedIn Jobs Scraper — No Login, No Cookies
• Research & Academic Jobs Scraper — 10 Sources
• Web Developer Jobs Scraper — 10 Boards in One

---

From the maker of Oink — an open-source, AI-powered job-search bot for Telegram that runs on these Actors. Try the free bot, get a managed instance at oinkjobsearch.com, or browse the full catalog of 50+ Actors.