🔍🚗 Mobile.de Scraper
3 days trial then $9.99/month - No credit card required now
🔍🚗 Mobile.de Scraper
3 days trial then $9.99/month - No credit card required now
Effortlessly scrape car data from Germany's largest vehicle marketplace, Mobile.de. Get access to millions of entries of cars, motorbikes, etc. across Europe. Fast, cheap & reliable. Rental version for larger use cases.
Scraping stops after first page of results.
I have entered a 50 urls, like:
changing page number from "&pageNumber=2" to "&pageNumber=50"
Tried with and without setting the max limits. Actor stops after first page of results, getting just 20 vehicles, instead of 1000.
Em I doing something wrong or the actor has a bug?
How can I scrap all 50 pages of results, to get 1000 vehicles data?
Actors
Hey,
the actor currently works without the possibility of manually setting the pageNumber URL parameter. It automatically starts from the first page and runs until the set amount of results is reached.
If you want to scrape all 1000 results, put just one URL in the input, (e.g. "https://suchen.mobile.de/fahrzeuge/search.html?dam=false&ft=ELECTRICITY&isSearchRequest=true&pageNumber=2&s=Car&sb=rel&vc=Car&lang=en") and set the 📈 Maximum result count per search URL (or searchPageURLMaxItems in JSON) to 1000. That should give you the required amount of results.
We will be adding the possibility of setting the first page index manually. When that happens, we will notify you in this thread.
Hello again,
just wanted to let you know that we implemented a feature called "Automatic paging". When enabled, the actor works as described above (Starts at page 0 and continues forward until limit). When disabled, it ONLY scrapes the exact search URL (while taking the pageNumber URL param into account, e.g. "https://suchen.mobile.de/fahrzeuge/search.html?dam=false&ft=ELECTRICITY&isSearchRequest=true&pageNumber=2&s=Car&sb=rel&vc=Car&lang=en") and nothing more (no further enqueuing of next search page happens).
That could help you for example if you wanted to scrape only the second or third page, or if you wanted to provide an exact list of URLs (just like you did in this run's issue).
Hope that helps. If you have further questions or queries, do not hesitate to either create a new issue, or reopen this one.
Have a great day,
3x1t
3x1tActor Metrics
10 monthly users
-
2 bookmarks
>99% runs succeeded
11 hours response time
Created in Dec 2024
Modified 4 days ago
