AI Website & PDF Extractor — RAG-Ready JSON

🚀 What this Actor does

Turns any webpage or PDF into structured, AI-ready JSON for LLMs, RAG, and embeddings in one call.

One-line description

Extract clean text, metadata, structured webpage elements, schema, and AI-ready semantic chunks from public webpages and PDFs using one consistent JSON format.

---

Overview

Analyze public webpages and PDFs and return structured, AI-ready JSON. Extract clean text, metadata, structured DOM elements, schema, semantic sections, and AI-ready chunks for LLMs, RAG, embeddings, vector databases, semantic search, and website intelligence.

---

Features

• Extract clean text from webpages and PDFs
• Metadata extraction (title, emails, phones, addresses)
• Structured DOM extraction (webpages)
• JSON-LD schema extraction (when available)
• Internal and external link extraction
• Social media link extraction
• CTA detection (buttons, contact links)
• Automatic semantic section detection
• AI-ready chunk generation
• Page statistics
• Consistent JSON structure across webpages and PDFs

---

Output Structure

data
├── url
├── meta
├── schema
├── stats
├── dom
└── text
    ├── raw
    ├── clean
    ├── sections
    └── chunks

Field Descriptions

• url – Final processed URL after redirects
• meta – Metadata (title, emails, phones, addresses)
• schema – JSON-LD structured data
• stats – Word count, headings, links, images, reading time
• dom – Structured elements (headings, images, links, social links, CTAs)
• text – Raw text, cleaned text, semantic sections, AI-ready chunks

Perfect For

• LLM applications
• Retrieval-Augmented Generation (RAG)
• Vector databases
• Embedding pipelines
• AI chatbots
• Knowledge bases
• Semantic search
• Website intelligence
• Content indexing
• Document ingestion

---

Why this Actor?

Most scrapers return raw HTML or inconsistent JSON. This Actor normalizes webpages and PDFs into one consistent AI-ready structure optimized for LLMs, RAG pipelines, embeddings, vector databases, and semantic search.

Input Example

{
  "urls": [
    "https://example.com",
    "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf"
  ]
}

Output Example

{
  "url": "...",
  "meta": {},
  "schema": [],
  "stats": {},
  "dom": {
    "headings": [],
    "images": [],
    "internalLinks": [],
    "externalLinks": [],
    "social": [],
    "cta": []
  },
  "text": {
    "raw": "",
    "clean": "",
    "sections": [],
    "chunks": []
  }
}