Pricing

from $0.0024 / page scraped

Website Content Crawler — Text, Titles & Metadata

Extract clean, readable content from any list of websites: page title, meta description, headings, main body text, word count and link/image counts. Optional same-domain crawl. Bulk-ready, no browser, no login. Great for LLM/RAG ingestion, content audits and research.

Pricing

from $0.0024 / page scraped

Rating

0.0

(0)

Developer

hiper soft

Actor stats

Bookmarked

Total users

Monthly active users

21 hours ago

Last modified

Website Content Crawler — Text, Titles & Metadata API through CLI

The Apify CLI is the official tool that allows you to use Website Content Crawler — Text, Titles & Metadata locally, providing convenience functions and automatic retries on errors.

Install Apify CLI

Using installation script (macOS/Linux):

$curl -fsSL https://apify.com/install-cli.sh | bash

Using installation script (Windows):

$irm https://apify.com/install-cli.ps1 | iex

Using Homebrew:

$brew install apify-cli

Using npm:

$npm install -g apify-cli

Other API clients include:

Website Content Crawler — Text, Titles & Metadata API in Python

Website Content Crawler — Text, Titles & Metadata API in JavaScript

Website Content Crawler — Text, Titles & Metadata OpenAPI definition

Website Content Crawler — Text, Titles & Metadata API

Website Content Extractor

tropical_prune/website-content-extractor

CAPABILITIES: extract_content, convert_to_markdown, batch_urls, extract_metadata. INPUT: URLs (single or array), with optional selectors and output format. OUTPUT: structured JSON with title, text, metadata, word_count. FORMATS: json, markdown, text. PRICING: PPE $0.001/page.

Bado

Web Content Crawler — Generic Site Text Extractor

agency-shift/web-content-crawler

Generic web content crawler. Extract text content from any URL. Lightweight alternative for quick page scraping and data collection for AI training and research.

Valdeir Lima

Pinecone Integration

apify/pinecone-integration

This integration transfers data from Apify Actors to a Pinecone and is a good starting point for a question-answering, search, or RAG use case.

Apify

557

3.2

(6)

OpenAI Vector Store Integration

jiri.spilka/openai-vector-store-integration

This integration uploads data from Apify Actors to the OpenAI Vector Store linked to OpenAI Assistant.

Jiří Spilka

224

4.8

(5)

Qdrant Integration

apify/qdrant-integration

Transfer data from Apify Actors to a Qdrant vector database.

Apify

4.7

(2)

PGVector Integration

apify/pgvector-integration

This integration transfers data from Apify Actors to a Postgres SQL database (with PGVector extension).

Apify

4.6

(5)

Milvus Integration

apify/milvus-integration

This integration transfers data from Apify Actors to a Milvus/Zilliz database and is a good starting point for a question-answering, search, or RAG use case.

Apify

4.5

(2)

Weaviate Integration

apify/weaviate-integration

This integration transfers data from Apify Actors to a Weaviate and is a good starting point for a question-answering, search, or RAG use case.

Apify

4.6

(3)

Chroma Integration

apify/chroma-integration

This integration transfers data from Apify Actors to a Chroma and is a good starting point for a question-answering, search, or RAG use case.

Apify

4.5

(2)

Website Content Crawler

parseforge/website-content-crawler

Crawl any website and pull clean Markdown content ready for AI! Follow links across a whole domain and extract page text, titles, headings, images, and metadata. Perfect for building RAG pipelines, training datasets, knowledge bases, and vector databases. Start crawling content in minutes!