No credit card required
Website Content Crawler
No credit card required
Automatically crawl and extract text content from websites with documentation, knowledge bases, help centers, or blogs. This Actor is designed to provide data to feed, fine-tune, or train large language models such as ChatGPT or LLaMA.
long loading of docker (about 10 sec)
Closed
Is there any way to get rid of waiting for the docker image of crawler to load and start crawling process right after sending urls? Now we are satisfied with the parsing itself, but the long loading of the “apify/web-scraper” parser image (about 10 seconds) is critical for us. Perhaps there are ready-made recipes and examples that will allow you to completely get rid of the wait for loading?
Hello and thank you for your interest in this Actor.
The Docker image load is an essential part of the Actor startup - in fact, any Apify Actor is a Docker image; you cannot run it on the Apify Platform, if it doesn't load first.
Our backend devs are constantly working on lowering the latency, though - the latest p90 was around 2 seconds for the Actor start (i.e. 90% of all Actor runs started under 2 seconds). It's more than possible that your long loading was caused by some momentary coincidence.
Closing this issue for now, but feel free to ask additional questions regarding this. Cheers!
Apify- 1.9k monthly users
- 99.9% runs succeeded
- 2.9 days response time
- Created in Mar 2023
- Modified 3 days ago
