Article Extractor: News & Blog Article Text Scraper
Pricing
Pay per event
Article scraper for news sites and blogs: headline, publish and update dates, language, lead image and full body text without menus, ads or comments. Paste article URLs, or a section page and it finds the articles. Markdown option. Fast HTTP, robots.txt respected. $0.50 per 1,000 articles.
null or list) crashed the run with exit 91. It now ends with exit 0 and one free status row: Invalid input, nothing was fetched and no results were charged: the input must be a JSON object, got <type>.charged: false, no RUN_SUMMARY, no charged event) with the reason <field> has the wrong type (<error>). Covered: articleUrls and sectionUrls (a number, boolean or a list with numbers), maxArticlesPerSection, minWords, maxArticles, maxConcurrency.requirements.txt: apify pinned to >= 4.1.0, < 4.2.0 (was >= 4.0.0, < 5.0.0).tests/test_invalid_input.py.failed, logged as
not fetched: private, loopback and link-local addresses are not allowed. A section page with such an address is
skipped with a warning. When no article is extracted, the free status row's reason now reads "could not be read
(HTTP error, robots.txt, a private address or not an HTML page)".my_actor/common.py (guard helpers; Http._robots_for checks the site first), my_actor/main.py (hook
on the httpx client, Blocked handled), tests/test_ssrf_guard_1102t.py (new), README Limits (one line).tags on sites that declare tags in a parsely-tags meta (for example The Verge). trafilatura 2.x returns tags as a list whose element is the whole comma-separated meta value, and the old code only split plain strings, so the output was one long string with internal tracking tokens (theverge,pagetype:story,...,stream-…). Every tag and category value is now split on commas and semicolons, trimmed and de-duplicated (case-insensitive), and tokens containing :, starting with stream- or equal to the site name are dropped. Categories given alone (without tags) are no longer lost. Regression test: tests/test_tags.py with a Verge-style fixture.statusRow: true.recordType: "status", status: "warning", message starting "Not charged: no article was extracted…") instead of leaving the dataset empty. It is not charged and is not written when at least one article was saved.