Website Intelligence API

Turn any public website into structured, AI-ready intelligence.

Website Intelligence API crawls and renders public pages with Playwright, then returns clean website data for research, RAG pipelines, sales workflows, compliance checks, audits, screenshots, and automation.

Quick start

Use a high-level intent instead of configuring dozens of crawler settings:

{
  "url": "https://example.com",
  "intent": "sales_research"
}

For advanced control, override the preset:

{
  "url": "https://example.com",
  "intent": "rag",
  "advanced": {
    "maxPages": 25,
    "includeScreenshots": false
  }
}

Supported intents

Intent	Use it for
website_snapshot	Screenshots, HTML, Markdown, metadata, PDFs, and page assets
rag	Markdown, chunks, headings, metadata, and internal links
sales_research	Company info, products, pricing pages, contact details, and socials
documentation	Docs pages, navigation, tutorials, code-oriented content, and search data
ecommerce	Product, category, pricing, review, and image discovery
compliance	Privacy, Terms, Cookie Policy, DPA, Subprocessors, Security, and Trust Center evidence
website_clone	Visual structure, colors, fonts, screenshots, layouts, navigation, and assets

What you get

• Structured extract-result.json
• Per-page dataset rows
• Markdown and optional rendered HTML
• Metadata, headings, links, CTAs, and page classifications
• Desktop, mobile, light mode, dark mode, and thumbnail screenshots
• Optional PDFs
• Optional RAG chunks
• Compliance evidence reports
• Snapshot report HTML
• Optional change detection and visual diff reports

Example output

{
  "company": {},
  "products": [],
  "pricing": [],
  "navigation": [],
  "pages": [],
  "faq": [],
  "contact": {},
  "socials": {},
  "metadata": {},
  "markdown": "",
  "screenshots": {},
  "screenshot": "",
  "pdf": "",
  "html": ""
}

Common use cases

• Build a website knowledge base for RAG
• Research a company before a sales call
• Capture compliance evidence from public pages
• Export screenshots, PDFs, Markdown, and structured JSON
• Monitor public website changes
• Analyze site structure for redesign, migration, or clone workflows

Advanced options

Most users only need url and intent.

Advanced users can control crawl size, screenshots, PDFs, Markdown, HTML, wait strategy, concurrency, resource profile, asset loading, visual diff, and change detection through the advanced object.

Notes

This Actor is designed for public websites. It respects robots.txt by default and is intended for website research, public web intelligence, compliance preparation, and AI-ready data extraction workflows.