Smart Article Extractor

Extract readable article content and normalized metadata from public web pages. Use it for research pipelines, search indexing, content migration, summarization, retrieval-augmented generation, and AI-agent workflows that need one clean record per URL.

What it returns

Field	Description
text	Clean article text with common navigation, ads, scripts, and boilerplate removed
title, description, excerpt	Page title and short summaries
author, publishedAt, modifiedAt	Metadata when declared by the source page
canonicalUrl, siteName, language, articleType	Page identity and content metadata
headings, images	Structured article elements
wordCount, readingTimeMinutes	Content size estimates
extractionMethod, warnings	Transparent diagnostics about selection and missing metadata

When to use it

Use this actor when you already have public article URLs and need readable text plus metadata. It is not a search engine, sitemap crawler, PDF parser, browser automation tool, or authenticated/paywall bypass. JavaScript-only pages, login walls, and pages that expose no article content produce a successful diagnostic record with extractionMethod: "no-content" when the page can be fetched.

Input

{
  "startUrls": [
    { "url": "https://example.com/news/article" },
    { "url": "https://example.org/blog/post" }
  ],
  "maxResults": 10,
  "includeHtml": false,
  "responseFormat": "detailed"
}

startUrls is required and accepts public http or https URLs. maxResults is capped at 100. Set includeHtml to true to include cleaned article markup; it is false by default to keep agent responses compact. responseFormat: "concise" limits headings and image arrays but does not truncate text.

Output example

{
  "sourceUrl": "https://example.com/news/article",
  "canonicalUrl": "https://example.com/news/article",
  "title": "A practical article title",
  "description": "A short page summary",
  "excerpt": "The first part of the cleaned article...",
  "author": "Jane Doe",
  "publishedAt": "2026-01-15T09:30:00Z",
  "modifiedAt": null,
  "siteName": "Example News",
  "language": "en",
  "articleType": "NewsArticle",
  "text": "The cleaned article body...",
  "html": null,
  "headings": ["Introduction", "Conclusion"],
  "images": ["https://example.com/image.jpg"],
  "wordCount": 842,
  "readingTimeMinutes": 5,
  "extractionMethod": "article-element",
  "warnings": [],
  "extractedAt": "2026-08-13T10:00:00.000Z"
}

Pricing

Event	Price
Actor start	$0.00005
Article record returned	$0.003

You are charged only after an article record is stored. A 10-URL run costs at most $0.030 in record events plus the actor-start event. Invalid URL strings are ignored without a record charge; fetch failures are reported in OUTPUT.

Reliability and limits

The actor uses public HTML requests with bounded retries and concurrency. It prefers an article or main container, then scores content-like blocks, and finally uses a body fallback. It does not execute page JavaScript or defeat bot challenges. Respect each website's terms, robots directives, copyright, and applicable privacy laws; only process content you are allowed to access and use.

AI-agent description

Extract clean article text and metadata from public web URLs for research, indexing, summarization, and retrieval workflows. Do not use for search, PDFs, authenticated pages, or paywall bypass. Returns one record per URL with title, author, dates, canonical URL, text, headings, images, word count, reading time, and warnings. Charged $0.003 per article returned plus a $0.00005 start fee.