Website Content Crawler avatar
Website Content Crawler

Pricing

Pay per usage

Go to Store
Website Content Crawler

Website Content Crawler

Developed by

Apify

Apify

Maintained by Apify

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.

4.5 (39)

Pricing

Pay per usage

1327

Total users

49.9k

Monthly users

7k

Runs succeeded

>99%

Issue response

5 days

Last modified

17 hours ago

JO

Cookie Banner is not removed

Open

Joe11 opened this issue
14 days ago

Cookies banner/warnings are still not removed and block event details from getting fetched by this actor.

jakub.kopecky avatar

Hey,

For this site, you can try adding div.bg-background\/80.backdrop-blur-sm to the removeElementsCssSelector input (in the HTML processing section). See my test run where the cookie banner was successfully removed: https://console.apify.com/view/runs/BxtZXutBadL9KsI4L