Willhaben.at Car Scraper

A lightweight scraper for extracting car listings from willhaben.at.

Features

• Extract car listings from custom search URLs
• Control pagination with configurable page limits
• Export data in JSON format with detailed vehicle information
• Support for multiple search URLs in a single run

Note: Willhaben.at limits results to maximum 20 pages per search. For larger datasets, split your searches into smaller chunks using different filter combinations in your URLs.

---

Configuration

start_urls (required)

An array of URL objects containing Willhaben.at search pages to scrape.

• Type: Array of objects with url property
• Format: Each object must contain a url field with a valid Willhaben.at search URL
• Example:

  "start_urls": [
    { "url": "https://www.willhaben.at/iad/gebrauchtwagen/auto/gebrauchtwagenboerse?rows=30" }
  ]
  

How to get search URLs:

1. Go to willhaben.at
2. Navigate to the car section (Gebrauchtwagen/Auto)
3. Use the search filters (make, model, price range, location, etc.)
4. Copy the URL from your browser's address bar
5. Add it to the start_urls array

scrape_page_limit (optional)

Limits the number of search result pages to scrape from each start URL.

• Type: Number
• Default: No limit (scrapes all available pages, up to Willhaben.at's maximum of 20)
• Example: "scrape_page_limit": 1 (scrapes only the first page)
• Use case: Useful for testing or when you only need recent listings

Configuration Examples

Minimal configuration (scrape all pages)

{
  "start_urls": [
    { "url": "https://www.willhaben.at/iad/gebrauchtwagen/auto/gebrauchtwagenboerse?rows=30" }
  ]
}

With page limit (test mode)

{
  "start_urls": [
    { "url": "https://www.willhaben.at/iad/gebrauchtwagen/auto/gebrauchtwagenboerse?rows=30" }
  ],
  "scrape_page_limit": 1
}

Multiple search URLs

{
  "start_urls": [
    { "url": "https://www.willhaben.at/iad/gebrauchtwagen/auto/gebrauchtwagenboerse?PRICE_FROM=0&PRICE_TO=10000" },
    { "url": "https://www.willhaben.at/iad/gebrauchtwagen/auto/gebrauchtwagenboerse?PRICE_FROM=10000&PRICE_TO=20000" }
  ],
  "scrape_page_limit": 5
}

---

Output

Output Data Structure

Each scraped car listing includes detailed information about:

• Basic Info: ID, URL, heading/description
• Vehicle Details: Make, model, model specification, year, car type (body style)
• Pricing: Price amount, formatted price
• Technical Specs: Power (kW/PS), fuel type, transmission, mileage, number of seats
• Features: Equipment list (resolved to readable names)
• Condition: Condition report, number of previous owners, warranty status
• Location: Country, state, district, postal code, exact coordinates
• Seller Info: Type (private/dealer), organization details
• Media: Main image, all photo URLs
• Additional: Published date, last updated timestamp, exterior color

Output Example

{
  "id": "1654277607",
  "url": "https://www.willhaben.at/iad/gebrauchtwagen/d/auto/bmw-5er-reihe-525d-xdrive-1654277607/",
  "description": "BMW 5er-Reihe 525d xdrive",
  "photos": [
    "https://cache.willhaben.at/mmo/7/165/427/7607_1988883143.jpg",
    "https://cache.willhaben.at/mmo/7/165/427/7607_-2021590471.jpg",
    "https://cache.willhaben.at/mmo/7/165/427/7607_-804916147.jpg"
  ],
  "highlight": [
    {
      "prefix": null,
      "value": "2009",
      "postfix": "EZ"
    },
    {
      "prefix": null,
      "value": "352.000",
      "postfix": "km"
    },
    {
      "prefix": null,
      "value": "197",
      "postfix": "PS (145 kW)"
    }
  ],
  "attributes": {
    "attribute": [
      {
        "name": "BODY_DYN",
        "values": [
          "Seit 2013 in meinem Besitz. Es wurde immer alles fachgerecht repariert..."
        ]
      },
      {
        "name": "CAR_MODEL/MAKE",
        "values": ["BMW"]
      },
      {
        "name": "CAR_MODEL/MODEL",
        "values": ["5er-Reihe"]
      },
      {
        "name": "CAR_MODEL/MODEL_SPECIFICATION",
        "values": ["525d xdrive"]
      },
      {
        "name": "PRICE/AMOUNT",
        "values": ["4900.0"]
      },
      {
        "name": "PRICE_FOR_DISPLAY",
        "values": ["€ 4.900"]
      },
      {
        "name": "YEAR_MODEL",
        "values": ["2009"]
      },
      {
        "name": "MILEAGE",
        "values": ["352000"]
      },
      {
        "name": "ENGINE/EFFECT",
        "values": ["145"]
      },
      {
        "name": "ENGINE/FUEL_RESOLVED",
        "values": ["Diesel"]
      },
      {
        "name": "TRANSMISSION_RESOLVED",
        "values": ["Automatik"]
      },
      {
        "name": "CAR_TYPE",
        "values": ["Kombi / Family Van"]
      },
      {
        "name": "CONDITION_RESOLVED",
        "values": ["Gebrauchtwagen"]
      },
      {
        "name": "NO_OF_OWNERS",
        "values": ["2"]
      },
      {
        "name": "NOOFSEATS",
        "values": ["5"]
      },
      {
        "name": "LOCATION",
        "values": ["Wien, 04. Bezirk, Wieden"]
      },
      {
        "name": "POSTCODE",
        "values": ["1040"]
      },
      {
        "name": "STATE",
        "values": ["Wien"]
      },
      {
        "name": "COORDINATES",
        "values": ["48.191558837890625,16.370309829711914"]
      },
      {
        "name": "EXTERIORCOLOURMAIN",
        "values": ["5"]
      },
      {
        "name": "PUBLISHED_String",
        "values": ["2024-08-27T00:00:00Z"]
      },
      {
        "name": "EQUIPMENT_RESOLVED",
        "values": [
          "Regensensor",
          "Fahrerairbag",
          "Nebelscheinwerfer",
          "Leichtmetall-/Alufelgen",
          "Antriebsschlupfregelung (ASR)",
          "CD Player",
          "Nichtraucherfahrzeug",
          "Mittelarmlehne",
          "Lederlenkrad",
          "Kurvenfahrlicht",
          "Kopfairbags",
          "Schisack",
          "Multifunktionslenkrad",
          "elektr. Sitze",
          "Lichtsensor",
          "Parksensor hinten",
          "ISOFIX",
          "Beifahrerairbags",
          "Head-Up-Display",
          "Tempomat",
          "Bluetooth / USB",
          "Stabilitätsprogramme (ESP)",
          "Lederausstattung",
          "Radio",
          "Klimaautomatik",
          "elektr. Spiegel",
          "Dachreling",
          "Servolenkung",
          "Parksensor vorne & hinten",
          "ABS",
          "Tagfahrlicht",
          "elektr. Fensterheber",
          "Zentralverriegelung",
          "Klimaanlage",
          "Reifendrucksensor",
          "Navigationssystem",
          "Bordcomputer",
          "Xenon"
        ]
      },
      {
        "name": "WARRANTY_RESOLVED",
        "values": ["Nein"]
      }
    ]
  },
  "seller": {
    "type": "Privat",
    "iconType": "NONE",
    "upsellingOrgLogo": null
  }
}

---

Limitations

• Willhaben.at limits search results to maximum 20 pages per query
• No built-in proxy support in this version
• Single domain support (willhaben.at)
• Country-specific domains may have variations (not all tested)

Workaround for large datasets: Split your searches using different filter combinations:

• By price range: €0-10,000, €10,000-20,000, etc.
• By location/region (federal states)
• By make/model
• By year range
• By body type (Limousine, Kombi, SUV, etc.)

---

Support

If this scraper doesn't match your needs or you encounter issues, please contact:

• Email: realspidery@gmail.com
• X/Twitter: @realspidery

For feature requests or bugs, please provide:

• Your input configuration
• Error messages or logs
• Expected vs actual behavior