Does this only extract US jobs?

Thanks for reaching out . We will check on this and get back to you.

Hi humorous_kilobyte,

We've added a new feature to the Upwork job scraper that allows you to input cookies from your browser. This should help the scraper mimic your browsing session and capture all jobs, not just US-based ones.

To use this feature, you'll need to install the EditThisCookie Chrome extension and follow these instructions to export cookies from your browser.

Try the updated scraper here: Upwork Job Explorer with Cookies.

If you need any assistance, feel free to reach out.

Perfect thanks so much for getting this sorted so quick. Can i ask if any of the cookies compromise my upwork account? id not want them to know im using any automation? Please let me know if you are not sure. Or if any particular cookies should be removed?

You're welcome! I'm glad we could get this sorted quickly.

Regarding your question about cookies, it's important to be cautious. When you export cookies, they include information about your current session, which helps the scraper mimic your logged-in environment. However, this also means that Upwork could potentially detect automation if the cookies contain sensitive session data.

To minimize risk:

Use Session-Specific Cookies: The scraper mainly needs cookies related to search preferences and location. Avoid using cookies tied directly to login credentials or session tokens.

Test Without Login: You can try running the scraper with cookies that don’t include login details, like those for search preferences and location. This way, it won't rely on your logged-in session and reduces the risk of detection.

Use Incognito Mode: Consider using an incognito window to export cookies with fewer session details.

If you're unsure about which cookies to include or exclude, feel free to reach out, and I can guide you further.

Thank you! Will try this out and let you know how it goes.

Just one more thing to clarify please. Do you login into upwork in one way or another in order to obtain job listings? It seems to me like you do because amount of jobs is more than when i use search in just incognito mode (but less than what i see in my account which is not locked to US only as mentioned before).

Reason im asking is that i have an impression that user hast to be logged in to get access to certain jobs.

You're correct—the scraper doesn't log in itself, but when you provide cookies from your logged-in session, it can access job listings that require being logged in. This is why you're seeing more jobs compared to incognito mode.

To minimize the risk of being detected or banned by Upwork, I recommend removing sensitive cookies before using them in the scraper. Specifically, look for and remove:

• session_id: This cookie maintains your logged-in session. Removing it will make the scraper operate as if it's not logged in.
• auth_token: This cookie is used for authenticating your session. Without it, the scraper won't carry your login credentials.
• csrf_token or xsrf_token: These are security tokens that protect against certain types of attacks but also indicate an active session.

You can do this by exporting your cookies using the EditThisCookie extension, opening the exported file in a text editor, and deleting these specific cookie entries.

By removing these cookies, the scraper will still access job listings but won't include your login details, reducing the risk of Upwork detecting and blocking your account.

Let me know how it goes!

hmm didnt work, im still getting same US only jobs. I used the extension to export session cookies only. Did you try it yourself by the chance?

When i experiment with it in UI when im logged in i can see that this parameter in URL &user_location_match=1 reflects whether search should be locked to the user country. But according to the logs when i run your actor you dont really have this parameter in url. so not sure if its relevant but thought i mention it to you.

Thanks for sharing the info. let me check and get back to you. I actually tried the scraper myself using cookies and i found lot of jobs when compared to previous version but didn't try to check the difference.

Hi humorous_kilobyte,

I was able to retrieve jobs from all countries using the same cookies extension and instructions i provided . Did you login when you copied the cookies?

thanks for confirming this. Yep i was logged in but i only took the session cookies. I tried it with all the cookies i got from extension apart from the following:

• __zlcmid - Often used for maintaining session state across pages.
• _cfuvid - Contains a unique identifier combined with a timestamp.
• _cq_duid - Directly implies a unique device/user ID.
• _upw_id.5831 - Appears to be a unique identifier for a user session or account.
• console_user - Could contain a user-specific identifier or user settings.
• current_organization_uid - Directly relates to a unique organizational identifier that could be traced back to a user context.
• visitor_id - Explicitly includes an identifier that seems to be used to track visitor data.
• _sp_id.2a16 - Typically used by tracking and analytics tools to store a unique session ID.

Will try again

Btw if this worked for you, any chance you could do upwork apify actor that is not locked to any country so no cookies needed?

Hi humorous_kilobyte,

I made few tests creating apify actors without being locked to any country but it doesn't work that way without cookies. But I found that you can login to any account to scrape the job using the cookies and the results would be same but you must be logged in for sure. This is beyond i can help you on this . Why don't you try scraping using new account with following the instructions i provided before and let me know .

Thanks for understanding. Closing the ticket and let me know if you need any other help .

thanks for the suggestion. I will test and get back to you asap. Im just super cautious about upwork catching this usage and linking it to my acc. So i probably will have to run a virtual machine and make an account from there just to be 100% safe. Appreciate you getting back to me on all the qs!

Just wanted to confirm that this worked for me.

Do you know how often i can use this? i.e. can i run this every minute?