Wellfound Premium Jobs Scraper avatar
Wellfound Premium Jobs Scraper

Pricing

$24.99/month + usage

Go to Store
Wellfound Premium Jobs Scraper

Wellfound Premium Jobs Scraper

Developed by

Radeance

Radeance

Maintained by Community

Extract data like job title, description, salary range, location, job link and company socials. Specify the search based on job title, location or customized via URL. Supports search and URL lists. Download your data as HTML Table, JSON, JSON-L, CSV, Excel, XML, RSS

4.7 (5)

Pricing

$24.99/month + usage

23

Total users

422

Monthly users

98

Runs succeeded

>99%

Issues response

3.9 hours

Last modified

14 days ago

SG

Monitoring Mode question

Closed

s.gigel opened this issue
19 days ago

Hi!

I have a question regarding "Monitoring Mode - Only new jobs" feature. I'd like to understand how this function works. Does the crawler use the job ID to determine whether it has already scraped the job or not? If so, what happens when a company publishes a job, then closes it, and later reactivates it from the archive with minimal changes - which is what usually happens? In that case, the ID is likely to stay the same, and the job wouldn't show up in the results, right?

Thanks!

radeance avatar

Hi @s.gigel, thanks for reaching out!

Yes you’re right — at the moment, monitoring mode doesn’t index jobs. If a job was fetched once (even in an aborted run), it won’t be picked up again based on the job url. We’re currently working on improving monitoring mode to make it smarter, including handling aborted runs better and adding an option to reset its state, among other enhancements.

The job_listing_posted datetime could be an appropriate indicator to verify a change, but we don't know if it does reflect republishing of a job listing. If you have a sample job where this happened, feel free to share it — we’d be happy to look into it as we refine the feature.

Best regards!

radeance avatar

Hi @s.gigel,

just wanted to update you, we’ve just released significant improvements to monitoring mode in the latest update.

Highlights include: • Monitoring mode now considers job ID, published date, and job description, rather than just the URL, when determining whether a job is new or updated. • Jobs previously fetched can now be reclaimed if they’re republished or updated. • We’ve also added a new reset_monitoring parameter, so you can clear the monitoring state and rescrape all jobs if needed.

Best regards!