Indeed Scraper avatar

Indeed Scraper

Try for free

Pay $5.00 for 1,000 results

View all Actors
Indeed Scraper

Indeed Scraper

misceres/indeed-scraper
Try for free

Pay $5.00 for 1,000 results

Scrape jobs posted on Indeed. Get detailed information from this job portal about saved and sponsored jobs. Specify the search based on location with the output attributes position, location, and description.

Do you want to learn more about this Actor?

Get a demo
XE

not working

Closed

xylonic_expansion opened this issue
2 months ago

i tried scraping all remote jobs in the philippines and it got no results https://console.apify.com/actors/runs/UuMpzdXBOLTBeQpWq#output.

lukas.prusa avatar

Hi, thanks for opening this issue!

Seems like Indeed has once again started experimenting with blocking on specific domains. This time the ph. domain. There is nothing we can do about this, you will just have to scrape a different domain in the meantime. Last time this has happened, they were blocking a few Oceanian domains for about 3 weeks.

We will figure out all the affected domains and disable them from the input for the time being.

I will keep you updated here, thanks!

XE

xylonic_expansion

a month ago

any update on this?

XE

xylonic_expansion

a month ago

I recently tried using some alternative scrapers to pull job listing data from ph.indeed and was able to do so successfully. Given this, I don't believe the issue lies with the source platform itself, as your team previously suggested.

It seems more likely that the problem is stemming from your software. Could you please look into this again and provide a solution?

XE

xylonic_expansion

a month ago

?

XE

xylonic_expansion

a month ago

?

lukas.prusa avatar

Hi again, we've taken a closer look at this domain, and it seems like the extracted links are a little different that what is found on other Indeed domains.

We will debug this further and fix this :)

XE

xylonic_expansion

a month ago

When do you think it will be fixed?

lukas.prusa avatar

I will try to prioritize it more, as it's our issue, so hopefully in the next few days :)

XE

xylonic_expansion

a month ago

Awesome, thank you so much!

lukas.prusa avatar

Hi, thanks for your patience!

I'm happy to inform you that we've just updated the scraper with the fix :)

Try it out and let me know how it works now, thanks and happy scraping!

XE

xylonic_expansion

a month ago

Thanks! I tried it To scrape 6000 job results but it stopped after page 23 and when I put in the URL for page 24 and then page 25 it scrapes Only the page and stops. It won’t go onto the next page. Please help!

XE

xylonic_expansion

a month ago

hello?

lukas.prusa avatar

Hi, could you please share the run's link?

Pagination and offsets on Indeed are kind of stupid anyway, because the results are randomized for anonymous users (which is how we scrape the page). So the pages won't give you the results you might expect.

Still, it shouldn't just fail on the 25 page for whatever the reason be. What was the listing for? Could that have been all the pages there were?

Thanks!

XE

xylonic_expansion

24 days ago
lukas.prusa avatar

Thanks for sharing the run. It stopped at the 29th page, where it didn't get any new results after 5 retries, so it stopped.

The only thing that I can think of, is to just simply increase the number of retries, but that would be costly on our end, meaning we would be losing money. So we would rather not do that.

I hope this helps, thank!

XE

xylonic_expansion

24 days ago

This didn’t help at all. You didn’t provide any solution

lukas.prusa avatar

Buy their API if you want all results: https://partners.indeed.com

Thanks!

Developer
Maintained by Apify

Actor Metrics

  • 816 monthly users

  • 118 stars

  • >99% runs succeeded

  • 2.1 days response time

  • Created in Mar 2023

  • Modified a day ago

Categories