Pricing

from $1.50 / 1,000 crawled pages

Try for free

Go to Apify Store

RAG Web Crawler: Markdown & Text Chunks

Try for free

Crawl public web pages into clean Markdown, plain text, and ready-to-embed text chunks for RAG pipelines and LLM apps. Every chunk keeps its source URL, title, and character offsets. Fast HTTP crawling that respects robots.txt and caps pages, requests, and output size.

Pricing

from $1.50 / 1,000 crawled pages

Rating

0.0

(0)

Developer

Richard Feng

Actor stats

Bookmarked

Total users

Monthly active users

18 minutes ago

Last modified

Categories

Developer tools

schemaVersion

Required

Type:string

Page URL

url

Required

The page's final public URL, after redirects. It is never a fetch endpoint and never contains credentials.

Type:string

Min. length:1

Title

title

Required

The page's title as extracted. An empty string when none is found.

Type:string

description

Required

The page's meta description, or null when unknown.

Type:string | null

languageCode

Required

The language the page declares, or null when unknown.

Type:string | null

canonicalUrl

Required

The canonical link the page declares on its own site, kept only when it passes the public URL check; otherwise null. It is what the page claims, not an independently verified identity.

Type:string | null

Markdown content

markdown

Required

The content extracted from the page, as Markdown. It is not a model-generated answer.

Type:string

Min. length:1

Plain text

text

Required

The plain-text version of the content. Chunks are cut from this field.

Type:string

Min. length:1

chunks

Required

Type:object[]

Min. items:1

contentTruncated

Required

Type:boolean

sources

Required

How search found the page. Pages reached by following links inherit the query and rank of the search result they started from, which is not their own search rank. Empty when no search led to the page.

Type:object[]

crawl

Required

Type:object

Instagram Scraper

apify/instagram-scraper

Extract Instagram posts, reels, profiles, places, hashtags, carousels, and comments. Get data from Instagram using one or more Instagram URLs or search queries: content, context, metrics, metadata. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

438K

4.7

(602)

Instagram Profile Scraper

apify/instagram-profile-scraper

Scrape all Instagram profile info. Just add Instagram usernames, IDs or URLs and extract name, join date, number of followers, location, bio, website, related profiles, video&post count, latest posts. Export scraped data, schedule scraper via API, and integrate with other tools or AI workflows.

Apify

255K

4.7

(175)

Google Maps Scraper

compass/crawler-google-places

Extract data from thousands of Google Maps locations and businesses, including reviews, reviewer details, images, contact info, including full name, email, and job title, opening hours, prices & more. Export data, run via API, schedule and monitor runs, or integrate with other tools.

Compass

656K

4.7

(1,818)

TikTok Scraper

clockworks/tiktok-scraper

Extract data from TikTok videos, hashtags, and users. Use URLs or search queries to scrape TikTok profiles, hashtags, posts, URLs, shares, followers, hearts, names, video, and music-related data. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Clockworks

346K

4.7

(370)

LinkedIn Profile Scraper + Email (No Cookies)

harvestapi/linkedin-profile-scraper

Extract detailed information from LinkedIn profiles in bulk, including complete work experience, education history, skills and more. No cookies or account required.

HarvestAPI

82K

4.6

(93)

Facebook Posts Scraper

apify/facebook-posts-scraper

Extract posts, videos, and engagement metrics from Facebook pages. Get text captions, reactions, video transcripts, images, external links, collaborators, and more from Facebook pages and profiles. Export ad data, schedule runs via API, and integrate with other tools or AI workflows.

Apify

127K

4.7

(236)

Google Search Results Scraper

apify/google-search-scraper

Scrape Google Search Engine Results Pages (SERPs). Select the country or language and extract organic and paid results, AI Mode, AI overviews, ads, queries, People Also Ask, prices, reviews, like a Google SERP API. Export data, run the scraper via API, schedule runs, or integrate with other tools.

Apify

211K

4.6

(220)

Web Fetch

apify/web-fetch

Download any web page by URL and convert it to clean Markdown, plain text, HTML, and other formats. Automatically renders JavaScript and works around bot blocking. The Actor provides a real-time API for quick response times.

Apify

33K

5.0

(3)

RAG Web Browser

apify/rag-web-browser

Web search and fetch tool for AI agents and RAG pipelines. It queries Google Search, scrapes the top N pages using a full web browser, and returns their content as clean Markdown for further processing by an LLM. Can also fetch individual URLs.

Apify

198K

4.7

(35)

Instagram Post Scraper

apify/instagram-post-scraper

Scrape Instagram posts. Just add one or more Instagram usernames and get your data in seconds including caption, metrics, images, mentions, coauthors, recent comments, sponsored status, video duration, views. Export scraped data, schedule scraper via API, integrate with other tools or AI workflows.

Apify

143K

4.3

(138)