Arbeitsagentur.de Job Scraper

Scraper for job listings from Arbeitsagentur.de (Bundesagentur für Arbeit), the German Federal Employment Agency's official job portal - one of the largest job boards in Germany with over 1 million active listings.

Features

• Fast Search API: Uses the official REST API for quick job listing retrieval
• ML-Powered Captcha Solving: Extracts contact details (email, phone, contact person) behind the captcha using a trained neural network (98% accuracy)
• Hybrid Architecture: Search via API (fast), details via Playwright browser automation (complete data)
• Comprehensive Data: Job title, company, location, employment type, salary info, requirements, and contact details
• Filter Options: Search by keywords, location, federal state (Bundesland), and employment type
• Direct URL Mode: Check status of specific job listings
• Standardized Output: Compatible with the Santamaria ecosystem JobListing schema

How It Works

1. Search Phase: The actor uses the public REST API (v4/jobs) to search for jobs based on your criteria
2. Detail Extraction (optional): For each job, Playwright navigates to the detail page
3. Captcha Solving: An ONNX neural network model solves the captcha protecting contact information
4. Data Extraction: Contact email, phone, and person details are extracted and merged with search data

Input

Field	Type	Description	Default
searchQuery	string	Keywords to search (Was)	-
location	string	City or region (Wo)	-
bundesland	string	Federal state code	All states
employmentType	string	Type of employment	All types
maxResults	integer	Maximum results to scrape	10
includeJobDetails	boolean	Extract contact details with captcha solving (~$0.01/job)	false
directUrls	array	Specific job URLs to scrape	-
proxyConfiguration	object	Proxy settings	Residential

Bundesland Codes

Code	State
BW	Baden-Württemberg
BY	Bayern (Bavaria)
BE	Berlin
BB	Brandenburg
HB	Bremen
HH	Hamburg
HE	Hessen
MV	Mecklenburg-Vorpommern
NI	Niedersachsen
NW	Nordrhein-Westfalen
RP	Rheinland-Pfalz
SL	Saarland
SN	Sachsen
ST	Sachsen-Anhalt
SH	Schleswig-Holstein
TH	Thüringen

Employment Types

Code	Description
VOLLZEIT	Full-time
TEILZEIT	Part-time
MINIJOB	Mini job
AUSBILDUNG	Apprenticeship
PRAKTIKUM	Internship
FREIBERUFLICH	Freelance
HEIMARBEIT	Remote work

Output

Each job listing includes:

{
  "id": "10000-1234567890-S",
  "title": "Elektriker (m/w/d)",
  "company": "Elektro Meier GmbH",
  "location": "80331 München, Bayern",
  "country": "DE",
  "canton": null,
  "salary_min": null,
  "salary_max": null,
  "salary_currency": "EUR",
  "employment_type": "full-time",
  "workload_min": null,
  "workload_max": null,
  "remote_option": null,
  "description_snippet": "Wir suchen einen erfahrenen Elektriker...",
  "description_full": "...",
  "requirements": ["Abgeschlossene Ausbildung", "Führerschein Klasse B"],
  "company_benefits": [],
  "posted_at": "2026-01-20T00:00:00Z",
  "expires_at": null,
  "source_url": "https://www.arbeitsagentur.de/jobsuche/jobdetail/10000-1234567890-S",
  "source_platform": "arbeitsagentur.de",
  "contact_salutation": null,
  "contact_firstname": "Hans",
  "contact_lastname": "Müller",
  "contact_email": "jobs@elektro-meier.de",
  "contact_phone": "+49891234567",
  "apply_url": "https://www.elektro-meier.de/karriere",
  "apply_email": "jobs@elektro-meier.de",
  "company_url": null,
  "scraped_at": "2026-01-26T12:00:00Z"
}

Usage

Quick Search (without contact details)

{
  "searchQuery": "Elektriker",
  "location": "München",
  "maxResults": 50,
  "includeJobDetails": false
}

Full Extraction (with contact details)

{
  "searchQuery": "Pflege",
  "bundesland": "BY",
  "maxResults": 20,
  "includeJobDetails": true
}

Via API

curl -X POST "https://api.apify.com/v2/acts/santamaria~arbeitsagentur-de-scraper/runs" \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "searchQuery": "Softwareentwickler",
    "location": "Berlin",
    "employmentType": "VOLLZEIT",
    "maxResults": 100,
    "includeJobDetails": true
  }'

Performance & Cost

Mode	Speed	Cost per Job	Data
Search only (includeJobDetails: false)	~25 jobs/sec	~$0.001	Basic job info
With contacts (includeJobDetails: true)	~1 job/3 sec	~$0.01	Full info + contacts

Tip: Use includeJobDetails: false (default) for high-volume scraping, enable only when you need contact details.

Technical Details

Captcha Solver

• Model: CNN + Bidirectional LSTM with CTC loss
• Input: 250x50 grayscale image
• Accuracy: 98% (trained on 10,000 solved captchas)
• Character set: 0-9, a-z (29 characters)
• Format: ONNX for fast inference

Preprocessing Pipeline

1. Grayscale conversion
2. OTSU thresholding (adaptive binarization)
3. Morphological operations
4. Normalization to [0, 1]

Limitations

1. Rate Limiting: Respectful delays between requests to avoid blocks
2. Salary Data: German job listings rarely include salary information
3. Job Expiration: API doesn't provide expiration dates
4. Contact Availability: Not all listings have contact details behind captcha

Version History

• 2.0.0 (2026-01-26):
  • Added Playwright browser automation for detail extraction
  • Integrated ML-powered captcha solver (ONNX)
  • Contact details extraction (email, phone, contact person)
  • Hybrid architecture: API search + browser details
• 1.0.5 (2024-12-23): Standalone actor with inlined utilities
• 1.0.0 (2024-12-22): Initial implementation with v4 API

Support

For issues or feature requests: GitHub Issues

---

Part of the Santamaria Job Scrapers Suite - Professional-grade job data for the DACH region.

Built with Apify | Arbeitsagentur.de