Dev.to Article Scraper — Tags, Authors & Full Text | from $1/1K No Proxy

Used by developer marketing teams tracking content trends, AI training dataset builders, and content intelligence platforms.

Scrape articles from Dev.to (the world's largest Forem developer community) via the official public Forem REST API. No proxy, no authentication, no fragile DOM parsing — just clean, reliable JSON data at scale.

Features

• 5 scraping modes: by tag, by author username, by article ID, full-text search, top articles
• 14 structured fields per article: title, description, URL, tags (array), author, reactions, comments, reading time, published date, optional body markdown
• includeBody option: fetch full Markdown source via the detail endpoint
• Batch input: pass multiple tags/usernames/queries in a single run
• parse_confidence field in every record — machine-readable data quality signal; 1.0 = perfect, deductions logged to warnings
• Zero proxy cost — official API, plain HTTPS, no access friction
• Pay-per-result pricing — you only pay for what you get

Use Cases

• Content research & trend analysis across the dev community
• Author portfolio scraping for lead generation or outreach
• Building training datasets (tagged articles with Markdown source)
• Monitoring publication activity for specific tags or users
• Competitive content intelligence (reactions, comments, reading time)

Input

Field	Type	Default	Description
mode	string	byTag	byTag / byUsername / byArticleIds / search / top
tags	string[]	["python"]	Tag names to scrape (mode=byTag)
usernames	string[]	[]	Author usernames (mode=byUsername)
articleIds	string[]	[]	Numeric article IDs (mode=byArticleIds)
searchQueries	string[]	[]	Search keywords (mode=search)
topDays	integer	30	Top articles from last N days (mode=top)
maxItems	integer	100	Max total articles (0 = unlimited)
includeBody	boolean	false	Fetch body_markdown via detail endpoint

Output Schema

Field	Type	Description
article_id	integer	Numeric Dev.to article ID
title	string	Article title
description	string	Subtitle / teaser
url	string	Canonical article URL
tags	string[]	Tag list
author_name	string	Author display name
author_username	string	Author @handle
positive_reactions	integer	Hearts / reactions count
comments_count	integer	Number of comments
reading_time_min	integer	Estimated reading time (minutes)
published_at	string	ISO 8601 UTC publish date
body_markdown	string|null	Full Markdown body (requires includeBody=true)
parse_confidence	float	Data quality score (1.0 = perfect)
warnings	string[]	Machine-readable quality warnings

Example Output

{
  "article_id": 3745852,
  "title": "Pick a better video thumbnail automatically with FFmpeg, PySceneDetect, and CLIP",
  "description": "We'll build a pipeline that takes any video file...",
  "url": "https://dev.to/masonwritescode/pick-a-better-video-thumbnail-automatically-4gpk",
  "tags": ["video", "python", "ai", "tutorial"],
  "author_name": "Mason K",
  "author_username": "masonwritescode",
  "positive_reactions": 42,
  "comments_count": 5,
  "reading_time_min": 6,
  "published_at": "2026-05-31T09:14:33Z",
  "body_markdown": null,
  "parse_confidence": 1.0,
  "warnings": []
}

API & Reliability

Uses the official Forem REST API (dev.to/api). Forem is open-source and the API is designed for programmatic access — schema stability is much higher than scraping HTML.

The parse_confidence field provides a data-quality signal in every record. A score below 0.8 indicates missing critical fields and is logged to warnings. This lets you filter or alert on data-quality degradation without manual inspection.

Use with AI agents (MCP)

This scraper is callable as a tool by AI agents (Claude Desktop, Cursor, VS Code, n8n, LangGraph, CrewAI, or any MCP-compatible client) via Apify's hosted Model Context Protocol server. An agent uses it to fetch live Dev.to articles by tag, author, or keyword mid-conversation — e.g. "find the top Python articles on Dev.to this month" or "what has @ben published recently?".

Point your MCP client at this single tool:

{
  "mcpServers": {
    "apify": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://mcp.apify.com/?tools=bovi/devto-scraper",
        "--header",
        "Authorization: Bearer <YOUR_APIFY_TOKEN>"
      ]
    }
  }
}

Minimal call an agent can make:

{ "mode": "byTag", "tags": ["python"], "maxItems": 20 }

Returns clean, flat rows the agent can reason over directly:

{
  "article_id": 3745852,
  "title": "Pick a better video thumbnail automatically with FFmpeg, PySceneDetect, and CLIP",
  "url": "https://dev.to/masonwritescode/pick-a-better-video-thumbnail-automatically-4gpk",
  "tags": ["video", "python", "ai", "tutorial"],
  "author_username": "masonwritescode",
  "positive_reactions": 42,
  "comments_count": 5,
  "reading_time_min": 6,
  "published_at": "2026-05-31T09:14:33Z",
  "parse_confidence": 1.0,
  "warnings": []
}

Reliability for agents: data comes from the official Forem REST API (not HTML scraping), so rows don't break on UI redesigns. Every record includes a parse_confidence score (1.0 = all critical fields present) and a warnings array for machine-readable quality signals — useful for agent pipelines that need to filter or alert on data quality. Keep maxItems to 50–200 and includeBody=false (default) to keep responses token-lean. No API key needed inside the tool — auth is your Apify token in the client config above.

Pricing examples

Run	Items	Cost
100 articles by tag (metadata)	100	~$0.10
1,000 articles with body Markdown	1,000	~$1.50
Top 200 articles last 30 days	200	~$0.20
Daily tag monitoring, 30 days	3,000/mo	~$3.00/mo

FAQ

Do I need a proxy or API key? No. The Forem/Dev.to public API requires no authentication and no proxy. Zero extra cost for buyers.

What formats can I export results in? JSON, CSV, JSONL, Excel — all via Apify dataset export. The output drops straight into Google Sheets, Airtable, Notion, or any BI tool.

Can I schedule it to track new articles? Yes. Use Apify Schedules to run daily on a tag or author and catch new publications automatically. Pair with a webhook to post new articles to Slack or a database.

What if the actor returns empty results? Check that your tag slug is lowercase and matches Dev.to's tag list (e.g. javascript not JavaScript). For search mode, try a broader query — Dev.to search indexes titles and descriptions. Errors are reported in the OUTPUT key-value store with a reason code.

Pricing

Pricing: $1.00 per 1,000 articles (metadata). Enable includeBody for full Markdown source — additional $0.50/1k for body content.

The includeBody=true option performs one additional API call per article and triggers the article-body premium event. Full Markdown source means clean, structured text — no HTML post-processing needed for LLM training pipelines.

Competitor comparison

	This actor	epctex/dev-to-scraper	Any alternative
Data source	Official Forem API	HTML scraping	?
Proxy needed	No	Yes	Unknown
Full body Markdown	Yes (toggle)	Partial	No
parse_confidence	Yes	No	No
Batch input	Yes	No	No

Dev.to runs on Forem — this actor works on any public Forem instance.

AI training datasets

Full Markdown source (includeBody=true) enables AI fine-tuning use cases: tagged technical articles, clean structured text, and engagement signals — ready for LLM training pipelines with no HTML post-processing.

---

This actor is not affiliated with Forem or DEV Community.

Integrations

Built for developer-marketing teams and dataset builders tracking content trends and article metadata on Dev.to — the JSON/dataset output drops into the tools you already run, no glue code:

• n8n / Make / Zapier — trigger a run or pipe every new dataset item into 500+ apps (Google Sheets, Airtable, Slack, HubSpot, your database) with no code: n8n, Make, Zapier.
• Webhooks — fire your own endpoint the moment a run finishes, to push results straight into your pipeline (docs).
• MCP server — expose this actor as a tool to Claude, Cursor, or any MCP client so an AI agent can pull this data mid-conversation (guide).
• API & SDKs — fetch the dataset as JSON, CSV, or Excel through the Apify REST API or the Python / JS SDKs.

See all Apify integrations.

Usage statistics

This Actor creates a small, content-free summary at the end of each run. It is used only to monitor reliability and improve this Actor. A copy is saved as USAGE_STATS in your own Apify key-value store, so you can see the exact record created for your run.

Set disableUsageStats to true in the input to opt out. Nothing is sent then; your USAGE_STATS record only says that statistics were disabled.

Only these fields are recorded:

• schema version, Actor name and build number;
• UTC start and finish hour (not a precise timestamp);
• run duration, number of results and time to the first result, each as a coarse range;
• whether the result was empty, the end status, and an error type from a fixed list;
• memory setting and counts of charged events;
• names of the input fields you set, never their values;
• the selected option for input fields that offer a fixed list of choices (for example a sort order).

We do not collect input text, search terms, URLs, domains, usernames, email addresses, names, proxy credentials, tokens, scraped records, output items, raw error messages, stack traces, or hashes of any of those values. Records are kept for no longer than 13 months, used only as aggregated operational statistics, and never sold or shared.

Additional fields (Phase 2)

This Actor also records your Apify user ID, whether Apify marks the account as paying, the size range of list inputs, the selected country when the input offers a fixed list of countries, and one category from a fixed Actor taxonomy. We use these fields only for aggregate reliability, repeat-use and cross-Actor analysis; reports suppress any cell with fewer than five distinct users.

The same disableUsageStats: true input flag turns these fields off too. The user ID is removed after 13 months; we do not export, sell, share, or attempt to re-identify this data.

Run-outcome signals (v2)

To learn whether a run did what it was asked to do, the record also holds a few more coarse ranges and yes/no flags. None of them contains content:

• the result limit you asked for (a range, when the input has one) and what share of it was delivered;
• results delivered per input item you listed (a range);
• output quality as ranges: how fully the result fields were filled, the share of rows that look like errors, the share of duplicate rows, and how many different fields appeared. These are counted in memory while results are saved; no result content is kept;
• how the run was started (console, API, schedule, webhook, another Actor);
• how it ended: stopped by you, timed out, reached the requested limit, stopped by the charge limit, and how many times the platform moved the run;
• if this Actor reports it: how many items to process worked or failed (ranges) and one failure reason from a fixed list;
• a short code made from the names of the input fields you set, never their values.

Repeat-run fingerprint (v2)

When your Apify user ID is recorded (see above), the record also holds an 8-character one-way code made from your input (proxy settings left out) and this Actor's name. It only lets us see that the same account ran the same input again soon after an unsatisfying run; we never see the input itself. It is stored only in the database, never published, and reports use it in aggregate with the same five-user minimum. It is the one exception to the statement above that no hashes are collected, and disableUsageStats: true turns it off.