Article Text Extractor | News and Blog API

Collect readable article text and available metadata from public news and blog URLs. Export titles, authors, dates, images, summaries and keywords for research or downstream processing. Metadata and summaries can be missing, and protected pages may not be extractable. Start with the published one-article example, check the output on your target sites, and then increase the URL list.

Best first run: open the Article Text Extractor, run the content-research example with one URL and a $0.03 cap, then add more URLs only after checking the returned text and metadata.

Start with one article

Open the public content-research example to extract one known blog article with a $0.03 run cap. Review the text and available metadata before adding more URLs. The current base rate is $0.02 per successful article ($20 per 1,000), plus $0.00005 per start event per GB of memory (minimum one). The example uses 2 GB, so one successful article plus start events is $0.0201 at this rate. Failed extraction diagnostics are stored separately and do not create a paid article result. Check the Pricing tab for current rates.

For API clients, use the same bounded first run and pass the charge cap as a run parameter:

curl -X POST "https://api.apify.com/v2/acts/xtech~article-extractor/runs?token=$APIFY_TOKEN&maxTotalChargeUsd=0.03" \
  -H "Content-Type: application/json" \
  -d '{"startUrls":[{"url":"https://blog.apify.com/what-is-web-scraping/"}],"language":"en","requestTimeout":30,"maxConcurrency":1,"maxRetries":1,"fetchImages":false}'

Features

• Extract full article text from article and blog post pages
• Return titles, authors, publication dates, source URLs, and language
• Collect summaries, keywords, meta descriptions, and meta keywords when available
• Capture the main image URL, discovered image URLs, and embedded video URLs when supported by the extractor
• Process multiple article URLs in one Actor run
• Select article language, request timeout, and image metadata options
• Save successfully extracted articles in the default dataset and failed-attempt diagnostics separately in errorResults

Input

Provide one or more article URLs in startUrls.

{
  "startUrls": [
    { "url": "https://blog.apify.com/what-is-web-scraping/" }
  ],
  "language": "en",
  "requestTimeout": 30,
  "fetchImages": false
}

Input fields

Field	Type	Description
startUrls	array	Direct article or blog post URLs to extract.
language	string	Optional two-letter language hint such as en, es, de, or fr. Leave empty for automatic detection where possible.
requestTimeout	integer	Maximum time in seconds to wait for each article page. Defaults to 30.
fetchImages	boolean	Include available article image URLs. The Actor stores image URLs, not image files.

The form manages concurrency, retries, User-Agent, and proxy routing automatically. Existing API clients can still supply maxConcurrency, maxRetries, browserUserAgent, and proxyConfiguration; their defaults and explicit proxy choices remain supported.

Output

The Actor returns a JSON dataset with the following fields for each article:

Field	Description
articleURL	Input or resolved article URL.
sourceURL	Source website URL when detected.
articleLanguage	Detected article language, for example en or es.
articleTitle	Article title.
articleAuthors	Article authors as an array.
articlePublishDate	Publication date when detected.
articleText	Clean extracted article text.
articleTopImage	Main article image URL when detected.
articleAllImages	Image URLs as an array when available.
articleVideos	Embedded video URLs as an array when available.
articleKeywords	Keywords or categories as an array when available.
articleSummary	Extracted or generated article summary.
articleMetaDescription	Page meta description when available.
articleMetaKeywords	Page meta keywords as an array when available.
wordCount	Number of words in the extracted article text.
characterCount	Number of characters in the extracted article text.
scrapeSuccess	true when extraction succeeded, otherwise false.
scrapeErrorMessage	Error details for failed extractions.
scrapedAt	UTC timestamp when the URL was processed.

Example Output

[
  {
    "articleURL": "https://www.example.com/news/article1",
    "sourceURL": "https://www.example.com",
    "articleLanguage": "en",
    "articleTitle": "Example News Article",
    "articleAuthors": ["John Doe", "Jane Smith"],
    "articlePublishDate": "2024-07-27T10:00:00Z",
    "articleText": "This is the full text of the example news article...",
    "articleTopImage": "https://www.example.com/images/article1.jpg",
    "articleAllImages": ["https://www.example.com/images/article1.jpg", "https://www.example.com/images/article2.png"],
    "articleVideos": [],
    "articleKeywords": ["news", "example", "article"],
    "articleSummary": "A brief summary of the example news article.",
    "articleMetaDescription": "An example article for demonstration.",
    "articleMetaKeywords": ["example", "article", "news", "demo"],
    "wordCount": 825,
    "characterCount": 4920,
    "scrapeMethod": "newspaper3k",
    "scrapeSuccess": true,
    "scrapedAt": "2024-07-27T12:34:56Z"
  }
]

With current pay-per-event pricing, failed URLs go to the separate errorResults dataset with articleURL, scrapeSuccess: false, scrapeErrorMessage, and scrapedAt. If at least one URL succeeds, the run can finish SUCCEEDED with useful articles saved and failures reported separately. If every valid URL fails, the Actor marks the run FAILED after writing diagnostic rows. Each run also writes RUN_SUMMARY with requested, extracted, saved, failed, all-failed, billing-mode, and spending-limit fields.

Only successfully extracted article rows enter the billable default dataset. Failed URLs and spending-limit diagnostics in errorResults are not charged as article results. The run-start event still applies; check the Pricing tab for current rates.

On Apify cloud, the residential-proxy fallback applies only when no proxy configuration is supplied. Explicitly disabled proxies, selected Apify groups, and custom proxy URLs are respected.

Use cases

• Monitor news articles, press mentions, and competitor content
• Build article datasets for research, analysis, or machine learning
• Enrich URLs with titles, authors, dates, summaries, and clean text
• Collect SEO metadata from article and blog pages
• Feed extracted article text into downstream AI, analytics, or database workflows

Related Actors on Apify Store

Pair this Actor with others from the same author for full content pipelines:

• YouTube Transcript Scraper Pro — video transcripts for the same research workflow
• News Source Crawler — discover article URLs from entire news sites, then extract with this Actor
• RSS Feed Scraper — monitor feeds and pass new article URLs here

Tips

• Use direct article URLs instead of homepages, category pages, or search result pages.
• If all URLs fail because of access restrictions, enable Apify Proxy and retry.
• Increase requestTimeout for slow publishers or long-form pages.
• Some JavaScript-heavy or paywalled pages may return partial text or fail if the article content is not present in the initial HTML.

Proxy settings

An explicit proxy choice is respected: disable proxies with {"proxyConfiguration":{"useApifyProxy":false}}, select the Apify groups available to your account, or supply your custom proxy URLs. If no proxy configuration is supplied on the cloud, the existing residential-proxy fallback applies. Start with a public page that works directly before using proxies for sites that need them. Proxy settings cannot guarantee complete extraction from protected or dynamic pages.