Extended GPT Scraper
Pricing
Pay per usage
Extended GPT Scraper
Extract data from any website and feed it into GPT via the OpenAI API. Use ChatGPT to proofread content, analyze sentiment, summarize reviews, extract contact details, and much more.
4.6 (4)
Pricing
Pay per usage
72
Monthly users
89
Runs succeeded
98%
Response time
1.4 days
Last modified
3 months ago
Some domains are disappearing from the results
Hi I am uploading a list of domains to process, after the task completes I am only uploading results for a part of the domains and the rest are “lost”. Moreover, I load the unprocessed ones again with the next task and they are processed.
Here is an example with one domain that was not included in the report twice and the third time it was processed https://console.apify.com/actors/tasks/t12CcmeQfXMszMbIZ/runs/wtoXCHjOZ1aNSAbAq#output
Thanks!
cooperative_bureau
Is it possible to receive in the results records about domains that could not be processed? Personally, it would solve my problem =)
cooperative_bureau
cooperative_bureau
cooperative_bureau
If I now re-load these 182 failed domains into the job, half of them will be successfully processed.
cooperative_bureau
That is 1600 domains I am processing for the third time and need at least one more time
Hi, thanks for opening this issue!
Yes, I agree adding this would make sense :) From what I understand, two things here would improve this situation:
- Increasing or allow changing the max request retries setting (currently it's just 3)
- Outputting failed items to output
Both of these would make sense :) We will investigate and discuss it with the team. I will keep you updated here, thanks!
cooperative_bureau
Lukáš, thank you. Then I'm not active for now, waiting for your message =)
Hi again, just a little update - this Actor is not really our top priority at the moment, but your suggestions are really good so we will implement them eventually. Just be patient with us, please :) We'll let you know once it's done!
Hi again, I'm happy to inform you that we've just updated the scraper with the update :)
- All failed pages will now be pushed to output
- We've also improved handling of the GPT requests, though don't expect anything substantial. I've noticed that you were having some rate limiting problems in your run from OpenAI, which this should help with a little, although it's still not any rate limit management solution.
Let me know how it works now, thanks and happy scraping!
Pricing
Pricing model
Pay per usageThis Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage.
