Pricing

$1.49 / 1,000 results

Go to Store

🔍🚙 AutoScout24 Scraper (PPR)

Try for free

Developed by

3x1t

Effortlessly scrape car data from the largest pan-European online car marketplace, AutoScout24. Get access to millions of entries of cars, motorbikes, etc. across Europe. Fast, cheap & reliable. Pay-per-result version.

5.0 (1)

Pricing

$1.49 / 1,000 results

Total users

Monthly users

Runs succeeded

>99%

Issues response

4.8 hours

Last modified

6 days ago

E-commerce

Automation

Lead generation

Back to issues Create new issue

Actor returning SUCCESS despite 95% request failure rate - Missing 25% of expected data

Closed

MerialSHK opened this issue

Hello, I've encountered a serious issue with the 3x1t/autoscout24-scraper-ppr actor that resulted in significant data loss. Problem Summary:

Completed runs: 1,965 (all reported as "SUCCEEDED") Expected data: 54,346 listings (based on AutoScout24's numbers) Actually received: 40,688 listings Missing: 13,658 listings (25% data loss)

Root Cause: The actor reports runs as "SUCCEEDED" even when the internal request success rate is extremely low. Here's an example from one of the "successful" runs: 2025-06-08T21:57:14.210Z INFO CheerioCrawler: Final request statistics: { "requestsFinished": 18, "requestsFailed": 356, "retryHistogram": [17,1,null,null,null,356], "requestsTotal": 374, "crawlerRuntimeMillis": 238450 }

2025-06-08T21:57:14.212Z INFO CheerioCrawler: Error analysis: { "totalErrors": 356, "uniqueErrors": 1, "mostCommonErrors": ["356x: Request blocked - received 429 status code."] } This shows:

Total requests: 374 Failed requests: 356 (95% failure rate!) All failures due to: "429 - Too Many Requests" Yet the run status: "SUCCEEDED" Results: 0 (which is good for pay-per-result pricing)

Impact:

While I appreciate the pay-per-result model meant no charges for failed runs, the main issue is: My monitoring system couldn't detect these failures because runs reported as "SUCCEEDED" I only discovered 25% of data was missing after analyzing the complete dataset I now need to identify and re-scan hundreds of URLs to get the missing data This requires significant development effort to build tools to identify and re-process failed URLs

Expected Behavior:

Runs with >50% request failures should be marked as FAILED (not SUCCEEDED) Better rate limit handling to avoid 429 errors Clear indication in run status when significant request failures occur

Request:

Could you please investigate why the actor reports SUCCESS despite massive request failures? Can you implement proper run status that reflects actual success/failure? Can you provide guidance on configuration to avoid rate limiting? Is there a way to automatically retry failed requests within the same run?

The main issue isn't the cost (thanks to pay-per-result), but the fact that I had no visibility into these failures until after processing all data. I've had to build custom analysis tools to identify affected URLs and will need to implement a workaround to re-scan the missing 25% of data. Thank you for your attention to this matter. Best regards

3x1t (3x1t)

Hey,

Thanks for reporting this issue. Unfortunately, I have not been able to reproduce the issue when stress testing the Actor, so the next steps will be based on my expertise, rather than observation. Fortunately, the solution for this issue should be quite simple.

The problem most probably stems from requesting a certain webpage (e.g. https://www.autoscout24.at/haendler/autopark-gmbh-innsbruck/bewertungen) for additional seller/company data that are included in each output entry. This webpage is most probably the main culprit - it has some level of rate limiting.

Immediate possible solution:

You can try to set the field lightningMode to true in the input. Doing this will make the actor stop scraping additional details that are not directly relevant to the ads/listings. This will result in these fields getting removed from the dealerDetails field in every output entry:

rating (only averageScore and count remain available under this object)
reviews
locale
aboutUs
brands
openingHours
heroImageUrl

Example difference between setting `lightningMode` to `true` or `false` (simplified)

Setting `lightningMode` to `false` (default behaviour)

{
    "title": "Porsche 911 GT3 Touring *Liftsystem* *Vollschalensitze*",
    "previewImage": "https://prod.pictures.autoscout24.net/listing-images/2e6a0d1d-0d70-44e8-b94a-bc621b7a1339_ce773023-2442-4022-9361-84f4b530b3cb.jpg/2560x1920.webp",
    "vehicleType": "Car",
    "bodyType": "Coupe",
    "brand": "Porsche",
    "model": "911",
    "modelVersion": "GT3 Touring *Liftsystem* *Vollschalensitze*",
    "url": "https://www.autoscout24.com/offers/porsche-911-gt3-touring-liftsystem-vollschalensitze-gasoline-silver-2e6a0d1d-0d70-44e8-b94a-bc621b7a1339",
    "price": {
        "total": {
            "amount": 299911,
            "currency": "EUR",
            "formatted": "€ 299,911"
        }
    },
    "createdDate": "2025-05-09T13:05:41.041Z",
    "modifiedDate": "2025-06-03T09:10:04.705Z",
    "description": "...",
    "attributes": {
      ...
    },
    "features": [
      ...
    ],
    "images": [
      ...
    ],
    "id": "2e6a0d1d-0d70-44e8-b94a-bc621b7a1339",
    "priceRating": null,
    "dealerDetails": {
        "id": 6611372,
        "name": "Autohaus Peternel GmbH",
        "sellerType": "Dealer",
        "phones": [
            {
                "type": "Mobile",
                "number": "+43 (0)664 - 3448448"
            }
        ],
        "email": "mercedes.benz@peternel.at",
        "url": "https://www.autoscout24.at/haendler/autohaus-peternel-gmbh/ueber-uns",
        "address": "Halbenrainer Straße, 8490 Bad Radkersburg, AT",
        "addressStructured": {
            "street": "Halbenrainer Straße",
            "city": "Bad Radkersburg",
            "countryCode": "AT",
            "zip": "8490"
        },
        "activeSince": "2010-07-01T00:00:00Z",
        "iconUrl": "https://prod.pictures.autoscout24.net/dealer-info/6611372-original-a486b890-19d9-4c5e-b4bd-7d662ca2fdf7.jpg/resize/200x100%3E/quality/90",
        "reviewPageUrl": "https://www.autoscout24.at/haendler/autohaus-peternel-gmbh/bewertungen",
        "rating": {
            "averageScore": 4.9,
            "count": 324,
            "recommendPercentage": 99,
            "gradesAverage": [
                {
                    "stars": 5,
                    "label": "Gesamteindruck"
                },
                {
                    "stars": 5,
                    "label": "Erreichbarkeit"
                },
                {
                    "stars": 5,
                    "label": "Zuverlässigkeit"
                },
                {
                    "stars": 5,
                    "label": "Angebotsbeschreibung"
                },
                {
                    "stars": 5,
                    "label": "Kaufprozess"
                }
            ]
        },
        "reviews": [],
        "locale": "de-AT",
        "aboutUs": "Herzlich Willkommen im Autohaus Peternel GmbH!\n\nBei uns finden Sie ein großes Angebot an Jung- und Gebrauchtwagen, nutzen Sie die attraktiven Preise, das geschulte Personal und natürlich unsere Kompetenz.\n\nBesuchen Sie uns in unserem Autohaus und lassen Sie sich von unserem großen Fahrzeugangebot inspirieren - Kommen Sie und fahren Sie bei uns probe. Unser Verkaufsteam präsentiert Ihnen die neuesten Modelle und ist gerne für Sie da!\n\nWir messen unseren Erfolg an der Zufriedenheit unserer Kunden. ",
        "brands": ["Mercedes-Benz", "smart", "Mazda"],
        "openingHours": [
            "Mo 08:00-12:00, 13:00-17:00",
            "Tu 08:00-12:00, 13:00-17:00",
            "We 08:00-12:00, 13:00-17:00",
            "Th 08:00-12:00, 13:00-17:00",
            "Fr 08:00-12:00, 13:00-17:00"
        ],
        "heroImageUrl": "https://prod.pictures.autoscout24.net/dealer-info/6611372-original-17af34ee-587a-413a-9156-0b81ef89f870.jpg/resize/1200x800>/quality/90/"
    }
}

Setting `lightningMode` to `true`

{
    "title": "Porsche 911 GT3 Touring *Liftsystem* *Vollschalensitze*",
    "previewImage": "https://prod.pictures.autoscout24.net/listing-images/2e6a0d1d-0d70-44e8-b94a-bc621b7a1339_ce773023-2442-4022-9361-84f4b530b3cb.jpg/2560x1920.webp",
    "vehicleType": "Car",
    "bodyType": "Coupe",
    "brand": "Porsche",
    "model": "911",
    "modelVersion": "GT3 Touring *Liftsystem* *Vollschalensitze*",
    "url": "https://www.autoscout24.com/offers/porsche-911-gt3-touring-liftsystem-vollschalensitze-gasoline-silver-2e6a0d1d-0d70-44e8-b94a-bc621b7a1339",
    "price": {
        "total": {
            "amount": 299911,
            "currency": "EUR",
            "formatted": "€ 299,911"
        }
    },
    "createdDate": "2025-05-09T13:05:41.041Z",
    "modifiedDate": "2025-06-03T09:10:04.705Z",
    "description": "...",
    "attributes": {
      ...
    },
    "features": [
      ...
    ],
    "images": [
      ...
    ],
    "id": "2e6a0d1d-0d70-44e8-b94a-bc621b7a1339",
    "priceRating": null,
    "dealerDetails": {
        "id": 6611372,
        "name": "Autohaus Peternel GmbH",
        "sellerType": "Dealer",
        "phones": [
            {
                "type": "Mobile",
                "number": "+43 (0)664 - 3448448"
            }
        ],
        "email": "mercedes.benz@peternel.at",
        "url": "https://www.autoscout24.at/haendler/autohaus-peternel-gmbh/ueber-uns",
        "address": "Halbenrainer Straße, 8490 Bad Radkersburg, AT",
        "addressStructured": {
            "street": "Halbenrainer Straße",
            "city": "Bad Radkersburg",
            "countryCode": "AT",
            "zip": "8490"
        },
        "activeSince": "2010-07-01T00:00:00Z",
        "iconUrl": "https://prod.pictures.autoscout24.net/dealer-info/6611372-original-a486b890-19d9-4c5e-b4bd-7d662ca2fdf7.jpg/resize/200x100%3E/quality/90",
        "reviewPageUrl": "https://www.autoscout24.at/haendler/autohaus-peternel-gmbh/bewertungen",
        "rating": {
            "averageScore": 4.9,
            "count": 324
        },
        "reviews": []
    }
}

What will I do next to resolve this issue:

Implement better system to avoid any possible rate-limiting (exponential backoff)
Return the right status (i.e. FAILED) when above 50% request failure rate
Possibly implement some way of storing failed request info in each run (probably into KVS under a certain key) to better communicate each request's status - just an idea - tell me your thoughts on this please

Your questions

Can you implement proper run status that reflects actual success/failure? => Yes (as said earlier, the target will be 50% as per your request - or would you rather see some other behavior?)
Can you provide guidance on configuration to avoid rate limiting? => This Actor does not support custom proxy settings. It is due to the fact that RESIDENTIAL proxies are a lot more expensive, therefore the cost would need to accommodate that. I am thinking of changing the price model of this actor into PPE (Pay per event), which should be better for these scenarios (i.e. customers pay only for the features they use, like residential proxies). But otherwise, your configuration is correct - residential proxies are a lot better for scraping when rate limiting is a factor to consider.
Is there a way to automatically retry failed requests within the same run? => The Actor automatically does that, but without the exponential backoff, it is not working properly (it fails multiple times in a very short time period). Once that is done, it should be much more resilient.

Firstly I would try to use the lightningMode, it is the fastest way of solving this issue (if you don't need the additional dealer detail fields). Also, it works way faster :).

I will keep you informed of my progress in this issue thread.

Best regards, 3x1t

3x1t (3x1t)

Hi again,

Apologies for the delay. I wanted to inform you that following steps were taken to resolve this issue:

Developed a better system for handling rate-limiting, using linear backoff with jitter
A new feature was implemented, which lets the user to decide the failure rate threshold using new input field. It is located under Advanced options section.
- It describes a relative threshold in %. When the amount of failed requests (relatively speaking) exceeds this threshold, the run will be marked as failed.
- The default value is 50, which means that for the run to be considered as failed, the amount of failed requests must be at least 50% of total requests.
- You can set it to 0% to make sure that you detect every failed request that occurs

Once more, I would recommend you to consider using lightningMode, as mentioned above due to avoiding the bottleneck page (which is the most probable culprit).

Please check it out and let me know if you run into any more issues.

Have a great day, 3x1t

Add comment

🔍🚙 AutoScout24 Scraper

3x1t/autoscout24-scraper

3x1t

Autoscout24 Scraper

inquisitive_sarangi/autoscout24-com-scraper

Autoscout24 scraper is a straightforward yet powerful tool designed to extract listings from AutoScout24, an online marketplace for buying and selling new and used vehicles, motorbikes, and accessories.

API Master

Autoscout24 Scraper

ivanvs/autoscout-scraper

Extract data from autoscout24.com for data on thousands of car listings. Scrape car listings, extract descriptions, images, prices, mileage, contact number, addresses, names, engine information, the transmission of the car, and all other listing details.

Ivan Vasiljević

283

4.0

Autoscout24 Scraper Lite

real_spidery/autoscout24-scraper-lite

Fast and lightweight Autoscout24.com scraper allows you to deep dive in the largest pan-European online car market for buyers and sellers. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools. Custom solution is available, just drop us an email

Real Spidery

🔍🚗 Mobile.de Scraper

3x1t/mobile-de-scraper

Effortlessly scrape car data from Germany's largest vehicle marketplace, Mobile.de. Get access to millions of entries of cars, motorbikes, etc. across Europe. Fast, cheap & reliable. Rental version for larger use cases.

3x1t

🔍🚗 Mobile.de Scraper (PPR)

3x1t/mobile-de-scraper-ppr

3x1t

Mobile.de Auto Scraper

lexis-solutions/mobile-de-auto-scraper

Scrape car data from mobile.de. Simple and powerful CSV/JSON car data extraction. Scrape new and used cars for sale on mobile.de.

Lexis Solutions

5.0

Autoscout24 Germany / Deutschland - Scraper

glaswasser/autoscout24-germany-deutschland---scraper

Autoscout24-Scraper für letzte Listungen in ganz Deutschland mit Leasing-Daten, Preisdaten, weitere Fahrzeugdaten, Adressdaten, google maps links...

Alex

Mobile.de Scraper

ivanvs/mobile-de-scraper

Extract data from mobile.de for data on thousands of car listings. Scrape car listings, extract descriptions, images, prices, mileage, contact number, addresses, names, engine information, the transmission of the car, and all other listing details.

Ivan Vasiljević