Website Content Crawler avatar
Website Content Crawler

Pricing

Pay per usage

Go to Store
Website Content Crawler

Website Content Crawler

apify/website-content-crawler

Developed by

Apify

Maintained by Apify

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.

4.6 (38)

Pricing

Pay per usage

1.1k

Monthly users

5.9k

Runs succeeded

>99%

Response time

2.3 days

Last modified

6 days ago

FT

cannot download pdfs

Open
ftballguy45 opened this issue
7 days ago

I am getting an API error for downloading pdf content. You can see the error here https://api.apify.com/v2/key-value-stores/srmZlRxPw0egV2LOY/records/aec7ca4-Send.httpRq.pdf

jakub.kopecky avatar

Hi, thank you for using Website Content Crawler. The issue is not with the API, it is returning the file as expected. But the returned file with .pdf extension is actually a malformed XML file. Could you please share the input or what pages are you trying to crawl?

Pricing

Pricing model

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage.