Universal · Vibe Scraping

Universal · Vibe Scraping

---

One actor, any website

Part of our Vibe Scraping series:
AI-built crawlers for universal data extraction.

---

⚠️ Beta Release

This actor is currently in beta. We're actively improving based on feedback. Report issues on the issues tab.

🔥 Why This Actor

The last scraping actor you'll need. One actor, any website, your schema.

• Any website: Works on most sites out of the box — no hunting for site-specific actors
• Your schema: Describe the data you want, or let the AI figure it out
• Portable: Reuse the same schema across websites for consistent output
• Fast & cheap: Custom Rust engine — no AI during extraction, orders of magnitude faster than LLM-based scraping

✨ Use Cases

• Multi-source Aggregation: Combine data from dozens of sites into a single consistent schema
• Change Monitoring: Track updates across any set of pages with structured alerts
• Dataset Building: Turn unstructured web pages into clean, structured datasets
• Workflow Automation: Replace manual copy-paste with reliable, scheduled extraction

💡 About Extralt

We're rethinking web scraping. Our crawlers are generated by AI but run as compiled code — giving you enterprise-scale performance without the brittleness of traditional scrapers or the cost of pure AI solutions.

🪙 Pricing

This actor uses a pay-per-event pricing model — you only pay for successfully extracted pages:

Subscription	Discount	Price per dataset item
Starter	Bronze	$0.002
Scale	Silver	$0.0015
Business	Gold	$0.001

Example: Extracting 1,000 pages on a Business plan costs: 1,000 × $0.001 = $1.00

All-inclusive pricing: We only use premium residential proxies, with no hidden costs or add-ons.

Why paid plans only? Apify excludes free plan users from revenue calculations (see docs), so we restrict this actor to paying customers only.

Concurrent runs: You can run up to 3 Extralt actors simultaneously. If you need more concurrent runs, please wait for one to finish before starting a new one. This number will increase as we scale up our infrastructure.

⬇️ Input

Parameter	Required	Description
Start URLs	Yes	One or more URLs to begin crawling
Country	Yes	Proxy location for regional content
Extraction Prompt	No	Describe the data you want — leave empty for auto-detection
Budget	No	Maximum number of items to extract
Graph Mode	No	Also extract non-item pages and generate site structure

Start URLs

The crawler adapts based on URL type:

• Catalog/listing page: Follows pagination, extracts items until budget is reached
• Any other page: Parses sitemap or follows internal links

Constraints:

• All URLs must be from the same host
• URLs should match your target country (e.g., example.fr or example.com/fr for France)

Extraction Prompt

Describe the data you want to extract in plain language:

"Extract the article title, author name, publication date, and full text"

"Get property listings with price, address, bedrooms, and square footage"

Leave empty to let the AI analyze the page and generate an appropriate schema automatically.

Graph Mode

Extracts all page types (Home, List, Item, Other) and generates a site structure graph showing page relationships.

When to use: You need to understand the website's architecture, visualize page hierarchy, or analyze how pages link to each other.

When to skip: You just want item data. Non-graph mode is faster, cheaper, and better leverages sitemaps to find items directly.

For more details about graph mode, see the Graph · Vibe Scraping actor.

⬆️ Output

1. Dataset

Each extracted item contains metadata and extracted data:

Field	Description
extracted_at	Unix timestamp of extraction
url	Page URL
page_kind	Page type: home, list, item, or other
title	Page title
data	Extracted fields based on your prompt (item pages only)
outbound_links	Links found on the page

Use the Overview view in Apify to browse results as a formatted table, or download in JSON, CSV, HTML, or Excel.

Page Kind

The crawler classifies each page into one of four kinds:

Page Kind	Description	Example
home	Main entry point of the site	https://www.nike.com
list	Category or collection pages listing multiple items	https://www.nike.com/w/womens-running-shoes-37v7jz5e1x6zy7ok
item	Pages containing 1 main item	https://www.nike.com/t/vomero-plus-womens-road-running-shoes-8AH6updi/HV8154-501
other	Landing, navigation, help, or non-content pages	https://www.nike.com/women

Data

The data field is only present for item pages. Its structure depends on your extraction prompt — if you didn't provide one, the AI generates an appropriate schema based on the page content.

Outbound Links

The crawler extracts different links depending on the page type:

Page Kind	Graph Mode Only	Links Extracted
home	Yes	Header navigation links only
list	Yes	Item pages in the catalog (ignores navigation, header, footer)
item	No	Related variants only (e.g., same product in different colors)
other	Yes	All content links (excludes header/footer navigation)

Example Output

{
    "extracted_at": 1764773403088,
    "url": "https://example.com/article/123",
    "page_kind": "item",
    "title": "How to Build a Web Scraper",
    "data": {
        "article_title": "How to Build a Web Scraper",
        "author": "Jane Smith",
        "publication_date": "2025-01-15",
        "content": "Web scraping is the process of..."
    },
    "outbound_links": []
}

2. Key-Value Store

Only in Graph Mode, see our Graph Actor for more details.

⚙️ Under the Hood

How It Works

1. First run — AI analyzes the site and generates a custom crawler based on your prompt (3-5 minutes)
2. Subsequent runs — Crawler is reused, extraction starts immediately

The crawler is regenerated when you change the website, country, or extraction prompt.

Why It's Fast

Unlike LLM-based scrapers that call AI for every page, we use AI once to generate a compiled Rust extractor. This means:

• No per-page AI costs — extraction runs as pure code
• High throughput — up to 50 pages/second (3,000/minute)
• Consistent results — same extractor, deterministic output

Infrastructure

Extraction runs on our dedicated infrastructure, not Apify's platform. There may be a brief delay (~15-20s) while provisioning resources before the crawl starts.

Stealth

Our Rust engine includes custom HTTP and browser implementations built specifically for web scraping:

• Smart request routing (Chrome rendering, fast HTTP, direct API calls)
• Anti-detection measures to avoid blocks
• Premium residential proxies included

🛠️ Troubleshooting

Extraction taking longer than expected?

• First run: AI is generating your custom crawler (3-5 minutes). Subsequent runs start immediately.
• Provisioning: Brief delay (~15-20s) while infrastructure spins up.

Results missing expected fields?

• Refine your extraction prompt to be more specific about the data you need
• Some fields may not exist on the target website

Getting blocked or no results?

• Verify the start URL is accessible in your browser
• Ensure the selected country matches the website's region
• Try a smaller budget to reduce request volume
• Some sites have aggressive bot protection — report persistent issues

🎙️ Feedback & Support

We're actively improving extraction quality based on your feedback.

• Bugs, questions, feature requests: Issues tab