Website Content Crawler

apify/website-content-crawler

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.

Apify

34.3k

949

Google Search Results Scraper

apify/google-search-scraper

Scrape Google Search Engine Results Pages (SERPs). Select the country or language and extract organic and paid results, AI overviews, ads, queries, People Also Ask, prices, reviews, like a Google SERP API. Export scraped data, run the scraper via API, schedule runs, or integrate with other tools.

Apify

53.2k

329

Reddit Scraper Lite

trudax/reddit-scraper-lite

Pay Per Result, unlimited Reddit web scraper to crawl posts, comments, communities, and users without login. Limit web scraping by number of posts or items and extract all data in a dataset in multiple formats.

Gustavo Rudiger

4.5k

80

Smart Article Extractor

lukaskrivka/article-extractor-smart

📰 Smart Article Extractor extracts articles from any scientific, academic, or news website with just one click. The extractor crawls the whole website and automatically distinguishes articles from other web pages. Download your data as HTML table, JSON, Excel, RSS feed, and more.

Lukáš Křivka

4.4k

87

Extended GPT Scraper

drobnikj/extended-gpt-scraper

Extract data from any website and feed it into GPT via the OpenAI API. Use ChatGPT to proofread content, analyze sentiment, summarize reviews, extract contact details, and much more.

Jakub Drobník

1.3k

60

Articles Extractor

web.harvester/articles-extractor

A fast and powerful tool for extracting article data from URLs. It uses advanced scraping techniques to parse HTML and extract relevant information such as the article title, author, publication date, and content. Download your data in any format (JSON, CSV, XML, RSS, HTML Table).

Web Harvester

375

13

Fast News Scraper

timgreen/fast-news-scraper

Extract full article text and metadata from popular news sites like The New York Times, Bloomberg, Reuters, BBC, CNBC, and Wired. Scrape thousands of articles in just a few minutes. Scape a single site or provide a list of article URLs to scrape.

Tim Green

213

13

RAG Web Browser

apify/rag-web-browser

Web browser for OpenAI Assistants API and RAG pipelines, similar to a web browser in ChatGPT. It queries Google Search, scrapes the top N pages from the results, and returns their cleaned content as Markdown for further processing by an LLM. It can also scrape individual URLs.

Apify

456

45

Social Media Sentiment Analysis Tool

tri_angle/social-media-sentiment-analysis-tool

Add a profile name and find this social profile on Facebook, Instagram, and TikTok, scrape its recent posts and comments, and perform sentiment analysis for each comment. All in one go. Export results in JSON, CSV, HTML, use API, schedule runs, integrate with other tools.

Tri⟁angle

442

27

GPT Scraper

drobnikj/gpt-scraper

Extract data from any website and feed it into GPT via the OpenAI API. Use ChatGPT to proofread content, analyze sentiment, summarize reviews, extract contact details, and much more.

Jakub Drobník

5.5k

77