Upwork Scraper Without Stale Job Posts
2 hours trial then $9.99/month - No credit card required now
Upwork Scraper Without Stale Job Posts
2 hours trial then $9.99/month - No credit card required now
Comes without any stale data like many other scrapers do. You only need to insert url(s) and its good to go, low cost and efficient.
Number of results and duplicates
Closed
Hi, I like your Upwork scraper the best because it allows for multiple search URLs. Every run only pulls 20 results (I ran two search URLs and it seems to give 10 for each). I want to run the scraper twice per day and add the results to Airtable through a Make scenario. Is it possible to filter duplicates or only pull results that are new since the last run? Also, how does the scraper choose which results to pull? I assumed it was only the most recent (search filter is sorted by 'newest') but some results are from a week ago. Thanks for your help. Again, I think this has the potential to be the best Upwork scraper on Apify,
Hey. Thank you for the kind comments. As for the amount of results - it is specified via the url you provide. E.g: https://www.upwork.com/nx/search/jobs/?per_page=50&q=python%20developer&sort=recency -- provides 50 results (per_page=50) and sorts by recency (&sort=recency). 10,20 and 50 are the options for the per_page= parameter. I'll see what I can do in regards to the duplicates. As for only pulling new results, this logic you would have to implement on your end. Hope this helped.
alligator.crawl
Great, that's clear, thanks for explaining!
version 0.0, build 0.0.25 (latest) now has the option to remove duplicates
ArturActor Metrics
13 monthly users
-
5 stars
>99% runs succeeded
17 days response time
Created in Oct 2024
Modified 23 days ago
