GPT Scraper avatar
GPT Scraper
Try for free

Pay $9.00 for 1,000 pages

View all Actors
GPT Scraper

GPT Scraper

drobnikj/gpt-scraper
Try for free

Pay $9.00 for 1,000 pages

Extract data from any website and feed it into GPT via the OpenAI API. Use ChatGPT to proofread content, analyze sentiment, summarize reviews, extract contact details, and much more.

User avatar

Not crawling?

Closed

David Haddad (davhad) opened this issue
7 months ago

No matter the entered depth (1 or 2), ithe nput URL's with multiple pages are only having their main page crawled and nothing else.

User avatar

Hey David,

I checked your runs, and you are missing glob patterns for enqueueing URLs. You can check the doc for glob patterns https://apify.com/drobnikj/gpt-scraper#glob-patterns . The scraper works in a way that enqueues URLs found under Link Selector and follows glob patterns.

I hope it helps.

Have a nice day!

User avatar

Hi Jakub,

Was under the impression that this part was optional and with it being empty all links matching the css class a[href] would be followed.

Thanks for clarifying.

User avatar

No problem,

I got your point. It is not intuitive this way. I would put this one into our backlog to improve.

Have a nice day!

Developer
Maintained by Apify
Actor metrics
  • 332 monthly users
  • 98.3% runs succeeded
  • 9.8 days response time
  • Created in Mar 2023
  • Modified 6 days ago