🌐 Article Record Extractor

Turns a list of article URLs into normalized records: title, author, dates, canonical, heading outline, main text, word count, language, an extraction-confidence score, and per-field provenance.

Output is one article_record row per result; billing is pay-per-event, the value event being one article extraction check (a $0.02 start fee per run, then $0.05 per article extraction check).

📊 Sample Output

Real rows from run fhyQsHelLVPYzSwsY on build 0.1.4 (2026-09-17), the same input as the Quick start below — every value is as the source published it (emails masked, long text shortened):

url	final_url	title	outcome	input
https://blog.python.org/	https://blog.python.org/	Python Insider	answer	https://blog.python.org/
https://www.iana.org/help/example-domains	https://www.iana.org/help/example-domains	Example Domains	answer	https://www.iana.org/help/example-domains
http://169.254.169.254/			blocked	http://169.254.169.254/

The run finished with the status message: PARTIAL: 2 billable verdict(s), 1 blocked (unbilled status), 0 unreachable, 0 unparseable of 3 target(s) | article_extraction_check billable=2 | article_extraction_check charged=2

✅ What you get

Each row is flat JSON with these fields (from the dataset schema and the sample run; a field the source does not publish for a given row is null):

• url (string/null) — e.g. https://blog.python.org/
• final_url (string/null) — e.g. https://blog.python.org/
• http_status (integer/string/null) — e.g. 200
• title (string/null) — e.g. Python Insider
• author (string/null) — null in every sample row
• published_date (string/null) — null in every sample row
• modified_date (string/null) — null in every sample row
• canonical_url (string/null) — e.g. https://blog.python.org/
• headings (string/null) — e.g. [{"level":1,"text":"Python 3.15.0 candidate 2 is here!"},{"level":2,"text":"Rece
• main_text (string/null) — e.g. News & updates from the Python core team Last chance to test before the big day!
• word_count (integer/string/null) — e.g. 109
• language (string/null) — e.g. en
• extraction_confidence (string/null) — e.g. 0.5
• partial (boolean/null) — e.g. True
• provenance (string/null) — e.g. {"title":"og:title","canonical_url":"link/og:url","main_text":"<article>/<main>
• faults (string/null) — e.g. missing author, published_date
• record_type (string) — e.g. article_record
• source (string/null) — e.g. buyer-submitted article URLs
• outcome (string/null) — e.g. answer
• input (string/null) — e.g. https://blog.python.org/
• outcome_reason (string/null) — e.g. robots.txt for http://169.254.169.254 unavailable/forbidden — cannot confirm per
• observed_at (string/null) — null in every sample row
• terminal (string/null) — null in every sample row

Output & billing

One row per target. A complete evidenced verdict carries outcome: answer and is the only billable outcome; blocked / unreachable / unparseable / rejected targets carry that status and are not billed (push-then-charge — failures can only undercharge).

⚙️ Sample inputs

1. Quick start — the Store example (this is what the sample above came from)

{
  "urls": [
    "https://blog.python.org/",
    "https://www.iana.org/help/example-domains",
    "http://169.254.169.254/"
  ],
  "userAgent": "Mozilla/5.0 (compatible; NexGenWatchBot/1.0; +https://apify.com/nexgenwatch)"
}

The sample run charged exactly: 1 × $0.02 apify-actor-start + 2 × $0.05 article_extraction_check = $0.12 on the Free tier — every delivered row was billed.

2. A smaller, narrowed run

{
  "urls": [
    "https://blog.python.org/",
    "https://www.iana.org/help/example-domains",
    "http://169.254.169.254/"
  ],
  "userAgent": "Mozilla/5.0 (compatible; NexGenWatchBot/1.0; +https://apify.com/nexgenwatch)"
}

A bounded run for checking the field shape before scheduling. userAgent narrows what the source is asked for.

3. A full-size run

{
  "urls": [
    "https://blog.python.org/",
    "https://www.iana.org/help/example-domains",
    "http://169.254.169.254/"
  ],
  "userAgent": "Mozilla/5.0 (compatible; NexGenWatchBot/1.0; +https://apify.com/nexgenwatch)"
}

The same input run on a schedule; each run is billed only for what it delivers.

🧾 JSON sample record

One real record from run fhyQsHelLVPYzSwsY, exactly as it lands in the dataset (emails masked, long text shortened):

{
  "url": "https://blog.python.org/",
  "final_url": "https://blog.python.org/",
  "http_status": 200,
  "title": "Python Insider",
  "author": null,
  "published_date": null,
  "modified_date": null,
  "canonical_url": "https://blog.python.org/",
  "headings": "[{\"level\":1,\"text\":\"Python 3.15.0 candidate 2 is here!\"},{\"level\":2,\"text\":\"Recent\"},{\"level\":3,\"text\":\"RISC-V is now officially supported by CPython!\"},{\"level\":3,\"text\":\"The Python documentation is now available in Russian!\"},{\"level\":3,\"",
  "main_text": "News & updates from the Python core team Last chance to test before the big day! I’m happy to announce that RISC-V platforms are now officially supported by CPython. 🎉 🚀 Announcing the 17 nominees for the inaugural Python Packaging Council ",
  "word_count": 109,
  "language": "en",
  "extraction_confidence": "0.5",
  "partial": true,
  "provenance": "{\"title\":\"og:title\",\"canonical_url\":\"link/og:url\",\"main_text\":\"<article>/<main> paragraphs\"}",
  "faults": "missing author, published_date",
  "record_type": "article_record",
  "source": "buyer-submitted article URLs",
  "outcome": "answer",
  "input": "https://blog.python.org/",
  "outcome_reason": null
}

🔧 How it works

Transport. Plain HTTPS from the Apify platform, no proxy. robots.txt is read first and a disallowed path is never fetched. Every request carries an identified contact User-Agent.

Terminal states. A run ends NORMAL, CAPPED (your cap was reached), PARTIAL (something was withheld and the message says what), GENUINE_EMPTY (the source was read and truly had nothing in scope) or BLOCKED (the source refused or changed shape — the run FAILS loud and bills nothing). A zero-row run is never reported as a silent success.

Charging. Each article extraction check is charged at the moment it is pushed (article_extraction_check); a row that fails to charge is not delivered, so the dataset count always equals the charged count.

What you submit

urls — article urls. Buyer-supplied article URLs. Per URL: one normalized record — title, author, published/modified dates, canonical URL, heading outline, main text, word count, language, an extraction-confidence score, and provenance. Blocked / paywalled / login-walled / bodyless pages never bill; an honest partial (some fields missing) is labeled as such.

The runtime source gate (per submitted target)

You choose the targets, so the source contract is enforced at run time, per origin, before any page is read:

• robots.txt fetched once per origin and honored; no robots / 404 = permitted, a disallowed path or unavailable robots file = BLOCKED.
• SSRF guard — every host is resolved and must be public; private, loopback, link-local, reserved and cloud-metadata addresses are refused before a socket opens.
• HTTP — 403 / 429 / 5xx = BLOCKED; DNS / timeout / connection faults = UNREACHABLE.

A BLOCKED or UNREACHABLE target is delivered as an unbilled status row — never a broken-site verdict, and never charged.

Why this and not the obvious alternative

single-article normalized records with authorship + date provenance receipts — not fleet-1's whole-site crawler; one URL in, one evidenced article record out.

What is not done. No login, no cookie or CAPTCHA bypass, no private or personal-account data, no browser automation.

💰 Pricing example

Event	Free	Bronze	Silver	Gold
Actor Start (apify-actor-start)	$0.02	$0.02	$0.02	$0.02
Article Extraction Check (article_extraction_check)	$0.05	$0.04	$0.04	$0.03

Worked at the live Free-tier price:

• 3 article extraction checks: $0.02 start + 3 × $0.05 = $0.17
• 25 article extraction checks: $0.02 start + 25 × $0.05 = $1.27
• 100 article extraction checks: $0.02 start + 100 × $0.05 = $5.02

A run that delivers zero rows charges the $0.02 start fee only. A BLOCKED run (source refused) fails loud and charges no value event. The start fee is charged once per GB of run memory; the default run memory is 1024 MB.

Yield on the sample run: PARTIAL: 2 billable verdict(s), 1 blocked (unbilled status), 0 unreachable, 0 unparseable of 3 target(s) | article_extraction_check billable=2 | article_extraction_check charged=2.

⚖️ Legal & ToS

This actor reads public data only. It collects only what the source publishes to any visitor, keeps to the source's robots rules (checked on every run), identifies itself with a contact User-Agent, and does not access accounts, private data or anything behind authentication. Use the output in line with the source's terms and your local law; the intended use is B2B research and monitoring.

❓ FAQ

Q: Do I need an API key or a login?
A: No. the input schema has no key field and the actor carries no secrets.

Q: Why did my run return 0 rows?
A: Read the run's status message. GENUINE_EMPTY means the source was read and had nothing in scope for your input; BLOCKED means the source refused and the run failed without billing a value event — retry later or narrow the input. A zero-row run bills the start fee only.

Q: How fresh is the data?
A: Every run reads the source live at run time; nothing is cached between runs. Put it on a schedule for a continuous feed.

Q: What formats can I export?
A: The dataset downloads as JSON, CSV, Excel, XML or RSS from the run's Dataset tab or the Apify API, and any run can push to a webhook or integration.

Q: How is this different from the other web QA and enrichment tools actors?
A: Same output shape and billing model; this one covers Article Record Extractor. The siblings under Related Actors cover the other sources or slices — run several on one schedule for a combined feed.

Q: Are there rate limits?
A: The actor paces itself against the source and honours its robots rules; there is no per-buyer limit beyond your Apify plan's concurrency.

🆘 Troubleshooting

• Run FAILED with BLOCKED → the source refused the request or changed its page shape → nothing was billed beyond the start fee; retry after a while, and if it persists open an Issue with the run id.
• Fewer rows than expected → the source had fewer items in scope → widen the input.
• Input validation error on start → a field is outside the schema's allowed values → start from the Quick start block and change one field at a time.
• Run TIMED-OUT → a very wide request on a slow day → raise the run timeout in Run options or narrow the input; what was delivered before the timeout is still in the dataset.

🔗 Related Actors

• Brand Asset Pack — Give it a list of company domains; it returns one CRM-ready record per domain describing
• Contact Evidence Enricher — Give it a list of company domains; it returns one CRM-ready record per domain describing
• Crossref DOI Metadata QA — A per-DOI QA verdict from official Crossref metadata — each DOI returns valid, not_found, or metadata_incomplete with every missing required field na…
• HTML to PDF Evidence Renderer — Renders each HTML document or permitted public URL to a hashed, print-pinned PDF evidence artifact: page count, resource-failure ledger, input+output…
• 🏢 About NexGenData — NexGen Watch is NexGenData's fleet of 256 public monitoring and lookup actors built on official sources, pay-per-result. Browse the catalog at apify.com/nexgenwatch.

⭐ Found this useful?

If this actor saved you a manual check, a quick review on the Apify Store helps other teams find it. Feature request or a source that changed? Open it from the Issues tab — every one is read.