Arbeitsagentur Scraper | Bundesagentur für Arbeit

Collects structured public job-listing data from the Bundesagentur für Arbeit job portal (arbeitsagentur.de) via its official REST API — identifiers, titles, employers, descriptions, categories, locations, work arrangements, contract signals, publication dates, and application links. Output is normalized JSON, ready for analytics, ETL, enrichment, and recurring monitoring.

Two input modes: build a search with filters, or provide direct Bundesagentur search-result URLs in startUrls.

Input parameters

Parameter	Type	Description	Default
keyword	string	Search terms (was).	–
location	string	City / postal code / region (wo). Overrides the location in direct URLs.	–
employment_type	string	job, education, early_career, self_employed → angebotsart (1/4/34/2).	job
radius	string	10_km … 200_km → umkreis.	–
is_remote	boolean	Remote / home-office only → arbeitszeit=ho.	–
is_suitable_for_career_change	boolean	Career-change-friendly roles (client-side).	–
beginning_date	array	Preferred start periods from_now_on, may_2026, … (client-side).	–
fixed_term	array	temporary, indefinite → befristung (1/2).	–
include_agency_jobs	boolean	Include private placement (pav) jobs. Off → pav=false.	–
exclude_external_jobs	boolean	Drop external job-board listings (client-side).	–
exclude_temporary_work	boolean	Drop temp-agency work → zeitarbeit=false.	–
disabled_only_jobs	boolean	Only accessibility-marked jobs → behinderung=true.	–
publication_date	string	today/yesterday/last_1_week/last_2_weeks/last_4_weeks → veroeffentlichtseit (0/1/7/14/28).	–
work_schedule	array	full_time/part_time/shift_night_weekend/part_time_job → arbeitszeit (vz/tz/snw/mj).	–
startUrls	array	Direct Bundesagentur search-result URLs.	–
limit	integer	Max jobs to collect.	50000

How filters are applied

• Server-side (Bundesagentur query): employment_type, keyword, location, radius, is_remote, work_schedule, publication_date, fixed_term, include_agency_jobs, exclude_temporary_work, disabled_only_jobs.
• Client-side (after detail fetch): exclude_external_jobs, is_suitable_for_career_change, beginning_date. The actor keeps paginating until limit matching jobs are collected.
• In startUrls mode the configured filter params are overlaid on each URL (URL-provided values win); the core search (was/wo/umkreis) comes from the URL.

Example inputs

{ "keyword": "Data Engineer", "location": "Berlin", "employment_type": "job",
  "radius": "25_km", "is_remote": true, "publication_date": "last_1_week", "limit": 50 }

{ "startUrls": [
    "https://www.arbeitsagentur.de/jobsuche/suche?angebotsart=1&was=softwareentwickler&wo=Hamburg&umkreis=15" ],
  "exclude_external_jobs": true, "include_agency_jobs": false,
  "work_schedule": ["full_time"], "limit": 100 }

Output

One job record per listing, written to the default dataset. Recommended idempotency key: type + ":" + id.

Key fields: type, id, url (external application link when present, else the portal page), source, sourceContext, title, company, description, category, categories, listingType, contractType, workModes, miniJob, careerChangeFriendly, disabilityFriendly, privatePlacementService, temporaryAgency, managedListing, dates, compensation, locations, distanceKm, remoteWork (available, arrangementType), contactReference, applicationUrl, postingPartner, referenceNumber, and sourceData (search + detail provenance layers).

How it works

Step	Endpoint
Search (paginated)	GET /pc/v4/jobs?was=…&angebotsart=1&page=…&size=…&…filters
Detail per job	GET /pc/v4/jobdetails/{base64(refnr)}

Called with the public header X-API-Key: jobboerse-jobsuche (no account needed). Detail pages are fetched 8 at a time with automatic retries on rate-limit / transient errors.

Project layout

.actor/  actor.json · input_schema.json · dataset_schema.json · Dockerfile
requirements.txt      # apify, httpx (pinned)
src/  __main__.py · main.py (orchestration) · api.py (REST client) · mappers.py (normalization)

Run locally

pip install -r requirements.txt
# put input in storage/key_value_stores/default/INPUT.json, then:
python -m src

Deploy to Apify

$apify push