Hey Otis! Thank you for this, will check it out and come back to you asap!

I see! To make this work, I'll need to implement the ability to pass cookies in the request, since the "user_location_match" parameter within the URL refers to the relevant location of the "user" of the request, which is fetched through cookies. Right now, this is ignored (by upwork itself) because the scraper has no cookies, meaning no user is logged in.

Interesting. So you're saying that the US only option is unique to whoever is signed in? I thought that was a universal filtering option.

For some reason the screenshot is not properly loading, but yes, The user_location_match parameter is specific to each user's country. (For example someone from japan would get japan only jobs if user_location_match is "=1") If not logged in, upwork removes the parameter from the url completely. Thats why the scraper still returns jobs.

Dang.. that sucks. So the only workaround would be to feed in my cookies? That's fine with me, I've just never done something like that. Is it something that is set once and it's done? Granted that it's the same URL being scraped each time?

Secondly, I'd imagine another potential workaround would be if your scraper could identify whether the job listing is available worldwide or is specifcally looking for people in a specified country (my case being the U.S.).

(In case the images don't work, here they are as URLs: https://paste.pics/e76b6d4f32efc2e7c903fab59ea11164, https://paste.pics/361b3014d7fc5a4876bc62f78544429b)

Yeah, sucks that its not as simple as it should be. So 2 workarounds. 1 - With cookies. and yes you set it once and you're done. 2 - Once I update the scraper to include job details, you'll be able to filter them directly, so no cookies will be needed

Either way I'll include cookies tomorrow and the job details hopefully within next week!

Awesome man! Thanks so much. Looking forward to those updates. Keep me posted if you can

Hey man, another thing I am trying to figure out is if there is a limit to how many jobs it can scrape with each run? It seems to always cap at 10, but I I am unsure if that simple because it's a coincidence being that there have only been up to 10 jobs found that matched my criteria, or if it's capping the search at 10.

The following is an example of my input:

{ "maxJobAge": { "value": 70, "unit": "minutes" }, "rawUrl": "https://www.upwork.com/nx/search/jobs/?amount=300-&contractor_tier=1,2,3&from_recent_search=true&hourly_rate=30-&payment_verified=1&q=%28automation,%20OR%20make.com%29&sort=recency&t=0,1 ", "sort": "newest" }

You can set the "perPage" property to 10, 20, or 50 to get more results. for example { "perPage": 50, ... }

I’ll keep you posted here once the cookies and details are sorted out!

Btw, if you could drop a nice review, it would mean a lot, it really helps since I just got started on Apify! Thanks!

Ah actually since you're using raw url just add this param in your url "&per_page=50"

I totally intend on giving you a 5 star review, just as soon as these upcoming updates are live and I have been able to test them out. Great work so far, especially given that you are new to Apify.

No with the added parameter, where exactly do I add it? I don't want the URLs to break.

Also, I am specifying that I want perPage to be 100, but it doesn't seem to make a difference. I guess, like you said, it needs to be added to the URL. Please let me know exactly where and how to add that so it doesn't break anything.

Here is one of my URLs: https://www.upwork.com/nx/search/jobs/?amount=300-&contractor_tier=1,2,3&from_recent_search=true&hourly_rate=30-&payment_verified=1&q=%28automation,%20OR%20make.com%29&sort=recency&t=0,1&user_location_match=1

Thank you!

Upwork allows up to 50 per page, so the maximum you can set is 50. You can add it anywhere in the url, for example https://www.upwork.com/nx/search/jobs/?amount=300-&contractor_tier=1,2,3&from_recent_search=true&hourly_rate=30-&payment_verified=1&q=%28automation,%20OR%20make.com%29&sort=recency&t=0,1&user_location_match=1&per_page=50

Got it, thanks. Let me know once the cookies update is live so I can test it out.

Any updates?

Sorry for the delay! Will have this done within the day and update you here!