Article Extractor — Clean Text, Title & Date, $0.001 an Article

Extract just the article from news, blog and documentation pages, with the title, publish date, language and lead image, as plain text and Markdown ready for summaries, RAG and agent pipelines; where URL to Markdown converts the whole page, this keeps the article and drops menus, footers and sidebars. AI agents can call and pay for it through Apify's MCP server and agentic payments. It costs $0.001 per article extracted, and a page that fails is never charged.

Example result

Example record for the minimal input under How to use it, as this Actor returned it on 2026-10-27 in a run of its production image; the record is kept unchanged with the release.

{
  "url": "https://docs.apify.com/academy/scraping-basics-javascript",
  "ok": true,
  "finalUrl": "https://docs.apify.com/academy/scraping-basics-javascript",
  "title": "Web scraping basics for JavaScript devs | Academy | Apify Documentation",
  "publishedAt": null,
  "language": "en",
  "siteName": "docs.apify.com",
  "leadImage": "https://apify.com/og-image/docs-article?title=Web+scraping+basics+for+JavaScript+devs",
  "text": "Web scraping basics for JavaScript devs\nLearn how to use JavaScript to extract information from websites in this practical course, starting from the absolute basics.\nIn this course we'll use JavaScript to create an application for watching prices. It'll be able to scrape all product pages of an e-commerce website and record prices. Data from several runs of such program would be useful for seeing trends in price changes, detecting discounts, etc.\n- Inspect pages using browser DevTools.\n- Download web pages using the Fetch API.\n- Extract data from web pages using the Cheerio library.\n- Save extracted data in various formats (e.g. CSV which MS Excel or Google Sheets can open) using the json2csv library.\n- Follow links programmatically (crawling).\n- Save time and effort with frameworks, such as Crawlee, and scraping platforms, such as Apify.\nAnyone with basic knowledge of developing programs in JavaScript who wants to start with web scraping can take this course. The course does not expect you to have any prior knowledge of other web technologies or scraping.\n- A macOS, Linux, or Windows machine with a web browser and Node.js installed.\n- Familiarity with JavaScript basics: variables, conditions, loops, functions, strings, arrays, objects, files, classes, promises, imports, and exceptions.\n- Comfort with building a Node.js package and installing dependencies with npm.\n- Familiarity with running commands in Terminal (macOS/Linux) or Command Prompt (Windows).\nLet's explore the key reasons to take this course. What is web scraping good for, and what career opportunities does it enable for you?\nThe internet is full of useful data, but most of it isn't offered in a structured way that's easy to process programmatically. That's why you need scraping, a set of approaches to download websites and extract data from them.\nScraper development is also a fun and challenging way to learn web development, web technologies, and understand the internet. You'll reverse-engineer websites, understand how they work internally, discover what technologies they use, and learn how they communicate with servers. You'll also master your chosen programming language and core programming concepts. Understanding web scraping gives you a head start in learning web technologies such as HTML, CSS, JavaScript, frontend frameworks (like React or Next.js), HTTP, REST APIs, GraphQL APIs, and more.\nScrapers are programs specifically designed to mine data from the internet. Point-and-click or no-code scraping solutions do exist, but they only take you so far. While simple to use, they lack the flexibility and optimization needed to handle advanced cases. Only custom-built scrapers can tackle more difficult challenges. And unlike ready-made solutions, they can be fine-tuned to perform tasks more efficiently, at a lower cost, or with greater precision.\nAs a scraper developer, you are not limited by whether certain data is available programmatically through an official API - the entire web becomes your API! Here are some things you can do if you understand scraping:\n- Improve your productivity by building personal tools, such as your own real estate or rare sneakers watchdog.\n- Companies can hire you to build custom scrapers mining data important for their business.\n- Become an invaluable asset to data journalism, data science, or nonprofit teams working to make the world a better place.\n- You can publish your scrapers on platforms like the Apify Store and earn money by letting others use them.\nWe are Apify, a web scraping and automation platform. We do our best to build this course on top of open source technologies. That means what you learn applies to any scraping project, and you'll be able to run your scrapers on any computer. We will show you how a scraping platform can simplify your life, but that lesson is optional and designed to fit within our free tier.\n📄️ DevTools: Inspecting\nLesson about using the browser tools for developers to inspect and manipulate the structure of a website.\n📄️ DevTools: Locating HTML elements\nLesson about using the browser tools for developers to manually find products on an e-commerce website.\n📄️ DevTools: Extracting data\nLesson about using the browser tools for developers to manually extract product data from an e-commerce website.\n📄️ Downloading HTML\nLesson about building a Node.js application for watching prices. Using the Fetch API to download HTML code of a product listing page.\n📄️ Parsing HTML\nLesson about building a Node.js application for watching prices. Using the Cheerio library to parse HTML code of a product listing page.\n📄️ Locating HTML elements\nLesson about building a Node.js application for watching prices. Using the Cheerio library to locate products on the product listing page.\n📄️ Extracting data from HTML\nLesson about building a Node.js application for watching prices. Using string manipulation to extract and clean data scraped from the product listing page.\n📄️ Saving data\nLesson about building a Node.js application for watching prices. Using the json2csv library to save data scraped from product listing pages in both JSON and CSV.\n📄️ Getting links from HTML\nLesson about building a Node.js application for watching prices. Using the Cheerio library to locate links to individual product pages.\n📄️ Crawling websites\nLesson about building a Node.js application for watching prices. Using the Fetch API to follow links to individual product pages.\n📄️ Scraping product variants\nLesson about building a Node.js application for watching prices. Using browser DevTools to figure out how to extract product variants and exporting them as separate items.\n📄️ Using a framework\nLesson about building a Node.js application for watching prices. Using the Crawlee framework to simplify creating a scraper.\n📄️ Using a platform\nLesson about building a Node.js application for watching prices. Using the Apify platform to deploy a scraper.",
  "markdown": "# Web scraping basics for JavaScript devs\n\n**Learn how to use JavaScript to extract information from websites in this practical course, starting from the absolute basics.**\n\nIn this course we'll use JavaScript to create an application for watching prices. It'll be able to scrape all product pages of an e-commerce website and record prices. Data from several runs of such program would be useful for seeing trends in price changes, detecting discounts, etc.\n\n- Inspect pages using browser DevTools.\n- Download web pages using the Fetch API.\n- Extract data from web pages using the Cheerio library.\n- Save extracted data in various formats (e.g. CSV which MS Excel or Google Sheets can open) using the json2csv library.\n- Follow links programmatically (crawling).\n- Save time and effort with frameworks, such as Crawlee, and scraping platforms, such as Apify.\n\nAnyone with basic knowledge of developing programs in JavaScript who wants to start with web scraping can take this course. The course does not expect you to have any prior knowledge of other web technologies or scraping.\n\n- A macOS, Linux, or Windows machine with a web browser and Node.js installed.\n- Familiarity with JavaScript basics: variables, conditions, loops, functions, strings, arrays, objects, files, classes, promises, imports, and exceptions.\n- Comfort with building a Node.js package and installing dependencies with `npm`.\n- Familiarity with running commands in Terminal (macOS/Linux) or Command Prompt (Windows).\n\nLet's explore the key reasons to take this course. What is web scraping good for, and what career opportunities does it enable for you?\n\nThe internet is full of useful data, but most of it isn't offered in a structured way that's easy to process programmatically. That's why you need scraping, a set of approaches to download websites and extract data from them.\n\nScraper development is also a fun and challenging way to learn web development, web technologies, and understand the internet. You'll reverse-engineer websites, understand how they work internally, discover what technologies they use, and learn how they communicate with servers. You'll also master your chosen programming language and core programming concepts. Understanding web scraping gives you a head start in learning web technologies such as HTML, CSS, JavaScript, frontend frameworks (like React or Next.js), HTTP, REST APIs, GraphQL APIs, and more.\n\nScrapers are programs specifically designed to mine data from the internet. Point-and-click or no-code scraping solutions do exist, but they only take you so far. While simple to use, they lack the flexibility and optimization needed to handle advanced cases. Only custom-built scrapers can tackle more difficult challenges. And unlike ready-made solutions, they can be fine-tuned to perform tasks more efficiently, at a lower cost, or with greater precision.\n\nAs a scraper developer, you are not limited by whether certain data is available programmatically through an official API - the entire web becomes your API! Here are some things you can do if you understand scraping:\n\n- Improve your productivity by building personal tools, such as your own real estate or rare sneakers watchdog.\n- Companies can hire you to build custom scrapers mining data important for their business.\n- Become an invaluable asset to data journalism, data science, or nonprofit teams working to make the world a better place.\n- You can publish your scrapers on platforms like the [Apify Store](https://apify.com/store) and earn money by letting others use them.\n\nWe are [Apify](https://apify.com), a web scraping and automation platform. We do our best to build this course on top of open source technologies. That means what you learn applies to any scraping project, and you'll be able to run your scrapers on any computer. We will show you how a scraping platform can simplify your life, but that lesson is optional and designed to fit within our [free tier](https://apify.com/pricing).\n\n[📄️ DevTools: InspectingLesson about using the browser tools for developers to inspect and manipulate the structure of a website.](https://docs.apify.com/academy/scraping-basics-javascript/devtools-inspecting)\n\n[📄️ DevTools: Locating HTML elementsLesson about using the browser tools for developers to manually find products on an e-commerce website.](https://docs.apify.com/academy/scraping-basics-javascript/devtools-locating-elements)\n\n[📄️ DevTools: Extracting dataLesson about using the browser tools for developers to manually extract product data from an e-commerce website.](https://docs.apify.com/academy/scraping-basics-javascript/devtools-extracting-data)\n\n[📄️ Downloading HTMLLesson about building a Node.js application for watching prices. Using the Fetch API to download HTML code of a product listing page.](https://docs.apify.com/academy/scraping-basics-javascript/downloading-html)\n\n[📄️ Parsing HTMLLesson about building a Node.js application for watching prices. Using the Cheerio library to parse HTML code of a product listing page.](https://docs.apify.com/academy/scraping-basics-javascript/parsing-html)\n\n[📄️ Locating HTML elementsLesson about building a Node.js application for watching prices. Using the Cheerio library to locate products on the product listing page.](https://docs.apify.com/academy/scraping-basics-javascript/locating-elements)\n\n[📄️ Extracting data from HTMLLesson about building a Node.js application for watching prices. Using string manipulation to extract and clean data scraped from the product listing page.](https://docs.apify.com/academy/scraping-basics-javascript/extracting-data)\n\n[📄️ Saving dataLesson about building a Node.js application for watching prices. Using the json2csv library to save data scraped from product listing pages in both JSON and CSV.](https://docs.apify.com/academy/scraping-basics-javascript/saving-data)\n\n[📄️ Getting links from HTMLLesson about building a Node.js application for watching prices. Using the Cheerio library to locate links to individual product pages.](https://docs.apify.com/academy/scraping-basics-javascript/getting-links)\n\n[📄️ Crawling websitesLesson about building a Node.js application for watching prices. Using the Fetch API to follow links to individual product pages.](https://docs.apify.com/academy/scraping-basics-javascript/crawling)\n\n[📄️ Scraping product variantsLesson about building a Node.js application for watching prices. Using browser DevTools to figure out how to extract product variants and exporting them as separate items.](https://docs.apify.com/academy/scraping-basics-javascript/scraping-variants)\n\n[📄️ Using a frameworkLesson about building a Node.js application for watching prices. Using the Crawlee framework to simplify creating a scraper.](https://docs.apify.com/academy/scraping-basics-javascript/framework)\n\n[📄️ Using a platformLesson about building a Node.js application for watching prices. Using the Apify platform to deploy a scraper.](https://docs.apify.com/academy/scraping-basics-javascript/platform)",
  "wordCount": 944
}

How to use it

Start a run in Apify Console, by API or from an agent with an input like this one — it is the input Apify runs every day to test this Actor:

{"urls": ["https://docs.apify.com/academy/scraping-basics-javascript"]}

Field	Type	Meaning
urls	array	Public article pages to read; one record each.
includeMarkdown	boolean	Also return the article as Markdown with headings, lists and links.

Events and prices

Event	Title	Charged when	FREE / BRONZE	SILVER	GOLD / PLATINUM / DIAMOND
article-extracted	Article extracted	One page fetched and its main text returned.	$0.001	$0.0009	$0.0008

Platform usage is not passed through. Apify's automatic actor-start event remains $0.00005 per started GB (one GB or less counts once). The synthetic dataset-item event is absent. Charge limits stop further paying work; unprocessed inputs receive uncharged errors. Completed readable pending events are settled on abort.

Worked cost example: ten successful article extracted events cost $0.01 at list, plus actor start. Failures are not included in that count.

Call it from an agent

Through Apify's MCP server (https://mcp.apify.com) use call-actor:

{
  "actor": "hermeskd/article-extractor",
  "input": {
    "urls": [
      "https://docs.apify.com/academy/scraping-basics-javascript"
    ]
  },
  "callOptions": {
    "maxTotalChargeUsd": 0.01,
    "timeout": 600
  },
  "waitSecs": 30
}

Then call get-dataset-items with the returned defaultDatasetId. Dedicated tool URL: https://mcp.apify.com?tools=hermeskd/article-extractor

Output fields

Field	Meaning
url	The input URL.
ok	True for a returned result; a failure is never charged.
finalUrl	The URL after redirects.
title	The article's headline.
publishedAt	The publish date the page declares, ISO 8601.
language	The page's declared or detected language.
siteName	The publishing site's name.
leadImage	The article's lead image URL.
text	The main text, boilerplate removed.
markdown	The main text as Markdown, when includeMarkdown is on.
wordCount	Words in the main text.
error	Failure, always uncharged.

Limits and run time

1–100 URLs a run. Each page is fetched once, up to 5 MB, under the 30-second per-request deadline; only HTML pages are read, and a page with under 20 words of main text is no_article_text. Bylines and author names are not returned. How it differs from URL to Markdown: that Actor converts the whole page, sections, tables and navigation text included; this one keeps only the article and adds its publish date, site name and lead image — use URL to Markdown for a full page, this for news, blog and documentation articles.

The default run timeout is 600 seconds. Typical run time: not yet measured; written from the first deploy.

Errors

Code	Meaning
http_error	The source answered an unsuccessful HTTP status.
download_failed	The request or connection failed within its bounded deadline.
unsupported_content_type	The URL did not return an HTML page.
no_article_text	No main text was found on the page.
robots_disallowed	robots.txt forbids fetching the URL; no request was made to it.
blocked_host	The host is on the deny-list or resolves to a private or local address; no request was made.
too_large	The response exceeded the size cap.
timeout	The input did not finish within the run's time budget.
invalid_input	The run input passed the input schema but cannot be used; the message names the field.
internal_error	Unexpected failure; the message carries the error class.
charge_limit_reached	The run reached its maximum total charge; this input was not processed.

No failure record is charged. A run fails only when all input outcomes are Actor-side failures; caller errors and a mixture of caller/Actor failures do not fail the run.

Sources and terms

Only process content you have the right to process. URL fetches obey HermesBot robots-group selection, the shared deny-list, public-address checks on connections and redirects, honest identity, deadlines and rate limits. No caller credentials, cookies, CAPTCHA solving, residential proxies or evasion. Public pages the caller names are fetched under robots.txt; text extraction by trafilatura (Apache-2.0). Process only content you have the right to process.

Not affiliated

This Actor is not affiliated with Apify or the owners of the pages it reads. Third-party names describe compatibility and source material.

Related Hermes Actors

• Career Site Job Search — Verified Greenhouse & Lever Boards — search open jobs in the Hermes directory of Greenhouse and Lever boards, each confirmed live.
• Speech to Text — Transcribe & Translate to English, SRT/VTT — speech to text with SRT and VTT subtitles.
• Video Converter & MP4 to MP3 — Trim, GIF; Audio $0.001 a Minute — audio and video conversion: MP4 to MP3, trim, GIF.
• Broken Link Checker That Never Charges for the Broken Links — broken links, 404s and redirects on a site.
• Company ATS Finder — Which Hiring System & Job Board It Uses — which hiring system and job board a company uses.
• Crossref DOI Metadata — Titles, Dates & Citation Counts — crossref doi metadata.
• JSON Dataset Cleaner — Merge, Dedup & Export Rows — json dataset cleaner.
• DNS Lookup & SPF, DKIM, DMARC Check — Findings by Severity — DNS records and SPF, DKIM and DMARC checks.
• Word, Excel & PowerPoint to Markdown — $0.002 per Document — Word, Excel, PowerPoint and other office files to Markdown.
• Domain Availability Checker — Registry RDAP, No Contact Data — whether domain names are available, from registry RDAP.
• File Downloader — Save Files & Images from URLs, $0.001 a File — files and images from URLs, saved with a public link each.
• Greenhouse Job Board Scraper — Open Jobs, Live, Any Company — every open job on a company's Greenhouse board.
• Public HTTP GET — Response Body, Status & Hash — public http get.
• Image Format Converter — Resize & Save WebP, PNG or JPEG — image format converter.
• Image Metadata Remover — Strip EXIF, GPS & Serials, Lossless — GPS location, camera serial numbers and other private metadata removed from photos, losslessly.
• Image to Text OCR — Lines & Words with Boxes, $0.002 an Image — OCR: the text in images.
• Image Upscaler & Photo Restorer — AI 2x/4x, Agent-Payable — image upscaling 2x or 4x, and photo restoration.
• Lever Jobs Scraper — Postings from Any Lever Site, US & EU — every published job on a company's Lever site, global or EU.
• Lighthouse Audit, No API Key — Lab Core Web Vitals & SEO Scores — Lighthouse scores and Core Web Vitals.
• Page Change Monitor — Detect Website Changes, $0.001 a Check — what changed on a page since the last check.
• PDF to Markdown & PDF to Text — PDF Text Extractor with OCR — PDF to Markdown and text, with OCR for scanned pages.
• Remote Jobs Scraper — Remote OK Listings, Salary Filter, Links — current remote jobs from Remote OK, with salary and listing links.
• RSS Feed Reader — RSS, Atom & JSON Feeds to JSON, $0.001 a Feed — every item of an RSS, Atom or JSON feed.
• SEO Audit — On-Page Checks, Schema & Hreflang, $0.004 a Page — an on-page SEO audit: meta tags, schema, headings and links.
• Sitemap URL Extractor — Indexes, Gzip, RSS; $0.0003 per URL — the URLs in a site's sitemaps.
• URL to Markdown for LLMs — Selectors, Text & Provenance — a public job page as clean Markdown where the site allows it (deny-listed job sites are refused).
• Video Frame Extractor — by Scene, Interval or Timestamp — frames, thumbnails and scenes from a video.
• Website Icon Finder — Favicons, Touch Icons & Theme Colors — declared website icons, theme colors and manifest links.
• Website Screenshot API — Elements, HTML, Dark Mode & PDF Print — screenshots and PDF prints of web pages.
• Website Tech Stack Lookup — Live Detection, $0.001 a Site — the technologies a website runs, detected live.
• Website to Markdown — Bounded Crawls for Agents — bounded whole-site Markdown with robots and crawl-delay enforced

Changelog

• 1.0 (2026-10-27) — first release.