Indeed Scraper
Pay $5.00 for 1,000 results
Indeed Scraper
Pay $5.00 for 1,000 results
Scrape jobs posted on Indeed. Get detailed information from this job portal about saved and sponsored jobs. Specify the search based on location with the output attributes position, location, and description.
Do you want to learn more about this Actor?
Get a demoHow can i ensure all data is being saved in one dataset & Company URL Question
Open
How can i ensure all data is being saved in one dataset and only getting charged for unique jobs across all runs vs each individual run?
Also I cant seem to find the company URL info anywhere in the scrape. How do I get that?
Hi, thanks for opening this issue!
-
Unfortunately, web-scraping comes with duplicates and there is no pretty way to mitigate them, especially on Indeed. The scraper can only do as much as is possible on Indeed, and without being logged-in (which we can't due to blocking), we have to crawl through all the pages "blindly". Basically, you will just have to filter them out across multiple runs and pay for the results, because otherwise the crawler would be crawling for essentially free, which is not possible.
-
Thanks for reporting this. We will make sure to add it in :) In the meantime, just enable the
parseCompanyDetailsoption. Just FYI it will make the runs take longer, so if you don't need the info, then you don't have to use it.
I will keep you updated here with the 2nd one. Thanks!
Misceres
