Github Search Scraper avatar
Github Search Scraper

Pricing

$25.00/month + usage

Go to Store
Github Search Scraper

Github Search Scraper

Developed by

SASWAVE

SASWAVE

Maintained by Community

Github search scraper. Get all data from search results list

0.0 (0)

Pricing

$25.00/month + usage

4

Total users

25

Monthly users

5

Runs succeeded

>99%

Last modified

2 months ago

pooky avatar

Fails on page 10 every time

Closed

Alex (pooky) opened this issue
7 months ago

Please see the actor run. I made a resurrect attempt, but it fails at the same page.

saswave avatar

SASWAVE (saswave)

7 months ago

We are looking into it, probably anti bot protections

pooky avatar

Alex (pooky)

7 months ago

I successfully added my cookies, which improved the process, but there were still some errors. In the end, I managed to run 100 pages. I think it would be helpful to include a line in the README stating that the antibot sometimes fails and that it's better to add a cookie.

saswave avatar

SASWAVE (saswave)

7 months ago

We found the issue, nothing related to anti bot protections:

When not providing cookies, github sometimes returns 429 responses to slow the requests made to the server

Initially, we ask cookies only if you need to search a url with type=code (mandatory for the extraction to work)

Now you should be able to handle the 100 pages without cookies