Hi @s.gigel,

Thanks for pointing this out!

You’re absolutely right — this was a logic bug where the page_offset value was unintentionally being overwritten and not respected, which caused the run to stop prematurely. We’ve just rolled out a fix, and this should now be working as intended.

To clarify: The max_pages limit of 25 is in place for fair usage reasons (for free users its 3). However, this limit can be bypassed by paying users using the page_offset parameter and running multiple batches. Our intention has always been to allow paying users to scrape as many pages as are available.

Thanks again for catching this, and for your continued feedback — it’s been super valuable!

Also, if you find the scraper useful and have a moment to leave a review, we’d really appreciate it — it helps us a lot :)

Best,

Mario – Radeance

Works smoothly! Thanks I'll definitely leave a 5-star review.

BTW do you plan to increase max_pages limit in future? This would be a real time-saver when using automations, since new job postings are often buried beyond page 25. And to get regular updates, I have to run multiple batches, which increases manual work and makes automation more complicated.

Hi @s.gigel,

Thank you so much again for your awesome review — really appreciate your support and feedback! ✌🏼

Based on your suggestion, we’ve now increased the max_pages limit to 100 for paying users. This should make automations much smoother and more scalable for you.

We’ve tested the update and it’s working well, but we’d love to hear your feedback once you’ve run it on your end.

Important note: With higher page loads, it’s expected that you might occasionally see a brief “session rotation” message or waiting message. This is normal and helps keep the scraping infrastructure stable under higher load.

Thanks again and as always don’t hesitate to reach out if you run into anything else!

Best regards,

Mario - Radeance

Sounds great! Thanks!

Thanks again for such a quick response — I'm a fan!

Let me ask one more question right away:

Right now, the supported URL format requires a location (or “remote”) in the link — for example: https://wellfound.com/role/l/backend-engineer/new-york

At the same time, the scraper doesn’t support links without a specific location — for example: https://wellfound.com/role/software-developer Was this done intentionally?

Thanks!

Hey @s.gigel,

Glad you like our work, it means a lot!

That's a good question. At the moment, we support URL formats that include a specific location or the remote keyword, like: https://wellfound.com/role/l/backend-engineer/new-york , https://wellfound.com/role/r/backend-engineer or https://wellfound.com/location/new-york.

A couple of days ago we added support for location-only URLs but we do plan to add full support for role-only URL formats as well.

It will require a bit more than just changing a line of code, but we’re already working on it and we’re aiming to have it available probably tomorrow or Friday. I’ll make sure to let you know right here in this thread once it’s live!

Thanks again for the great feedback — keep it coming! 🙏🏽

Best regards,

Mario – Radeance