AI/ML Jobs Scraper
Pricing
from $1.00 / 1,000 results
Scrape AI/ML jobs from 150+ companies via Greenhouse, Lever, RemoteOK, WeWorkRemotely, and YC. Extracts skills, frameworks, salary, and experience. Optional AI enrichment. Change detection and webhook alerts.
All notable changes to the AI Jobs Intelligence Platform are documented here.
src/lib/jobs/adapters/otta.ts — New adapter fetching from Otta's public API. Includes salary data (84% of listings), tech stack, experience levels, and company intel.src/lib/jobs/adapters/ashby.ts — New adapter fetching from Ashby's official Job Posting API. Includes compensation data when companies publish salary bands.SourceName type to include "otta" and "ashby".InputSchema to include new sources in the sources enum.TIER_SOURCES to include new sources for all tiers.company-urls.ts.src/lib/jobs/adapters/yc.ts — New adapter fetching from YC Work at a Startup API (workatastartup.com/api/jobs). Includes salary data and apply URLs.deterministic (free, keyword-only), hybrid (LLM for uncertain jobs), ai (LLM for every job).llama-3.1-8b-instant) and OpenRouter (meta-llama/llama-3.1-8b-instruct:free).apify-actor-start and apify-default-dataset-item synthetic events.ai-enrichment event for LLM-enriched jobs.created, updated, unchanged, or removed between runs.src/main.ts — Apify Actor entrypoint using Actor.main(). Reads typed input via Zod schema, validates configuration, sets up proxy, runs ingestion pipeline, and persists run summary to Key-Value Store.INPUT_SCHEMA.json — Apify platform input schema with UI field definitions, select editors, and defaults..actor/actor.json — Actor manifest with build tag, environment variable declarations, and file references.Dockerfile — Production container image using apify/actor-node:22 base image.src/lib/ai/types.ts — AIProvider interface and AIEnrichment type. Defines the enrichment contract that both providers implement.src/lib/ai/groq-provider.ts — GroqProvider class using Groq's OpenAI-compatible chat completions API (llama-3.1-8b-instant model). Includes retry, timeout, and JSON response validation.src/lib/ai/openrouter-provider.ts — OpenRouterProvider class using OpenRouter API. Same interface as Groq, allowing providers to be swapped via input config.src/lib/ai/prompt.ts — Structured LLM prompt builder. Requests a JSON object with 19 enrichment fields. Description truncated to 2500 characters to control token usage.src/lib/ai/parse.ts — parseEnrichmentResponse() using Zod for LLM output validation. All fields are optional to handle partial/malformed LLM responses gracefully.src/lib/ai/enrichment.ts — createEnricher() factory implementing three classification modes:
deterministic: always returns null (no LLM calls)hybrid: skips LLM for high-confidence jobs (aiRelevanceScore ≥ 40, skills detected, experience level detected)ai: sends every job to the LLMsrc/lib/jobs/ingest.ts — Complete rewrite removing all Drizzle ORM / PostgreSQL dependencies. Key changes:
Actor.openKeyValueStore() with a STATE key storing a hash-map of StateEntry objects between runs.Actor.openDataset() → dataset.pushData() instead of SQL INSERT/UPDATE.withRetry() — one source failing after 3 attempts is skipped with a warning, not a crash.removed if all their known sources were part of this run.applyFilters() for keyword and company input filters.buildOutputRecord() producing flat, schema-consistent dataset records.jobsFiltered, jobsOutput fields in addition to existing counts.src/lib/retry.ts — withRetry<T>() with exponential backoff, jitter, and configurable max attempts. fetchWithTimeout() using AbortController for per-request timeout enforcement.src/lib/fetcher.ts — createFetcher(proxyUrl?) returning a proxy-aware fetch wrapper. Uses undici.ProxyAgent when a proxy URL is provided; falls back to global fetch otherwise.src/lib/logger.ts — Thin wrapper mapping pino-style (data, msg) calls to Actor.log.info/warning/error/debug() calls. Preserves the existing adapter calling convention.src/lib/input-schema.ts — Zod schema and inferred ApifyInput type. Single source of truth for input validation used by both main.ts and ingest.ts.All four source adapters are functionally identical to the previous Express-server versions. Only changes:
pino singleton to Actor.log wrapper (src/lib/logger.ts).fetcher: Fetcher parameter (defaults to global fetch) enabling proxy injection.err objects cast to String(err) for serialization compatibility with Actor.log.src/lib/jobs/classify.ts — No logic changes. Import paths updated for NodeNext ESM resolution.src/lib/jobs/taxonomy.ts — Identical to previous version.src/lib/jobs/dedupe.ts — Identical to previous version.src/lib/jobs/types.ts — SourceName type moved here from ingest.ts to avoid circular imports.artifacts/jobs-dashboard/ — React + Vite frontend dashboard (replaced by Apify Dataset as the data delivery mechanism).artifacts/api-server/ — Express 5 REST API server (replaced by Actor.main() entrypoint).lib/db/ — Drizzle ORM + PostgreSQL schema (replaced by Apify KeyValueStore for state and Dataset for output).lib/api-spec/ — OpenAPI 3.1 specification (no longer applicable — Actor uses structured dataset output).lib/api-zod/ — Generated Zod schemas from OpenAPI spec (removed with api-spec).lib/api-client-react/ — Generated React Query hooks (removed with dashboard).apify-actor/ as a standalone workspace package (@workspace/ai-jobs-actor).tsc directly (NodeNext module resolution, ESM output) instead of esbuild.package.json type set to "module" for native ESM compatibility with Apify SDK v3.