Dev.to Article Scraper — Tags, Authors & Full Text | from $1/1K No Proxy

Used by developer marketing teams tracking content trends, AI training dataset builders, and content intelligence platforms.

Scrape articles from Dev.to (the world's largest Forem developer community) via the official public Forem REST API. No proxy, no authentication, no fragile DOM parsing — just clean, reliable JSON data at scale.

Features

• 5 scraping modes: by tag, by author username, by article ID, full-text search, top articles
• 14 structured fields per article: title, description, URL, tags (array), author, reactions, comments, reading time, published date, optional body markdown
• includeBody option: fetch full Markdown source via the detail endpoint
• Batch input: pass multiple tags/usernames/queries in a single run
• parse_confidence field in every record — machine-readable data quality signal; 1.0 = perfect, deductions logged to warnings
• Zero proxy cost — official API, plain HTTPS, no access friction
• Pay-per-result pricing — you only pay for what you get

Use Cases

• Content research & trend analysis across the dev community
• Author portfolio scraping for lead generation or outreach
• Building training datasets (tagged articles with Markdown source)
• Monitoring publication activity for specific tags or users
• Competitive content intelligence (reactions, comments, reading time)

Input

Field	Type	Default	Description
mode	string	byTag	byTag / byUsername / byArticleIds / search / top
tags	string[]	["python"]	Tag names to scrape (mode=byTag)
usernames	string[]	[]	Author usernames (mode=byUsername)
articleIds	string[]	[]	Numeric article IDs (mode=byArticleIds)
searchQueries	string[]	[]	Search keywords (mode=search)
topDays	integer	30	Top articles from last N days (mode=top)
maxItems	integer	100	Max total articles (0 = unlimited)
includeBody	boolean	false	Fetch body_markdown via detail endpoint

Output Schema

Field	Type	Description
article_id	integer	Numeric Dev.to article ID
title	string	Article title
description	string	Subtitle / teaser
url	string	Canonical article URL
tags	string[]	Tag list
author_name	string	Author display name
author_username	string	Author @handle
positive_reactions	integer	Hearts / reactions count
comments_count	integer	Number of comments
reading_time_min	integer	Estimated reading time (minutes)
published_at	string	ISO 8601 UTC publish date
body_markdown	string|null	Full Markdown body (requires includeBody=true)
parse_confidence	float	Data quality score (1.0 = perfect)
warnings	string[]	Machine-readable quality warnings

Example Output

{
  "article_id": 3745852,
  "title": "Pick a better video thumbnail automatically with FFmpeg, PySceneDetect, and CLIP",
  "description": "We'll build a pipeline that takes any video file...",
  "url": "https://dev.to/masonwritescode/pick-a-better-video-thumbnail-automatically-4gpk",
  "tags": ["video", "python", "ai", "tutorial"],
  "author_name": "Mason K",
  "author_username": "masonwritescode",
  "positive_reactions": 42,
  "comments_count": 5,
  "reading_time_min": 6,
  "published_at": "2026-05-31T09:14:33Z",
  "body_markdown": null,
  "parse_confidence": 1.0,
  "warnings": []
}

API & Reliability

Uses the official Forem REST API (dev.to/api). Forem is open-source and the API is designed for programmatic access — schema stability is much higher than scraping HTML.

The parse_confidence field provides a data-quality signal in every record. A score below 0.8 indicates missing critical fields and is logged to warnings. This lets you filter or alert on data-quality degradation without manual inspection.

Use with AI agents (MCP)

This scraper is callable as a tool by AI agents (Claude Desktop, Cursor, VS Code, n8n, LangGraph, CrewAI, or any MCP-compatible client) via Apify's hosted Model Context Protocol server. An agent uses it to fetch live Dev.to articles by tag, author, or keyword mid-conversation — e.g. "find the top Python articles on Dev.to this month" or "what has @ben published recently?".

Point your MCP client at this single tool:

{
  "mcpServers": {
    "apify": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://mcp.apify.com/?tools=bovi/devto-scraper",
        "--header",
        "Authorization: Bearer <YOUR_APIFY_TOKEN>"
      ]
    }
  }
}

Minimal call an agent can make:

{ "mode": "byTag", "tags": ["python"], "maxItems": 20 }

Returns clean, flat rows the agent can reason over directly:

{
  "article_id": 3745852,
  "title": "Pick a better video thumbnail automatically with FFmpeg, PySceneDetect, and CLIP",
  "url": "https://dev.to/masonwritescode/pick-a-better-video-thumbnail-automatically-4gpk",
  "tags": ["video", "python", "ai", "tutorial"],
  "author_username": "masonwritescode",
  "positive_reactions": 42,
  "comments_count": 5,
  "reading_time_min": 6,
  "published_at": "2026-05-31T09:14:33Z",
  "parse_confidence": 1.0,
  "warnings": []
}

Reliability for agents: data comes from the official Forem REST API (not HTML scraping), so rows don't break on UI redesigns. Every record includes a parse_confidence score (1.0 = all critical fields present) and a warnings array for machine-readable quality signals — useful for agent pipelines that need to filter or alert on data quality. Keep maxItems to 50–200 and includeBody=false (default) to keep responses token-lean. No API key needed inside the tool — auth is your Apify token in the client config above.

Pricing examples

Run	Items	Cost
100 articles by tag (metadata)	100	~$0.10
1,000 articles with body Markdown	1,000	~$1.50
Top 200 articles last 30 days	200	~$0.20
Daily tag monitoring, 30 days	3,000/mo	~$3.00/mo

FAQ

Do I need a proxy or API key? No. The Forem/Dev.to public API requires no authentication and no proxy. Zero extra cost for buyers.

What formats can I export results in? JSON, CSV, JSONL, Excel — all via Apify dataset export. The output drops straight into Google Sheets, Airtable, Notion, or any BI tool.

Can I schedule it to track new articles? Yes. Use Apify Schedules to run daily on a tag or author and catch new publications automatically. Pair with a webhook to post new articles to Slack or a database.

What if the actor returns empty results? Check that your tag slug is lowercase and matches Dev.to's tag list (e.g. javascript not JavaScript). For search mode, try a broader query — Dev.to search indexes titles and descriptions. Errors are reported in the OUTPUT key-value store with a reason code.

Pricing

Pricing: $1.00 per 1,000 articles (metadata). Enable includeBody for full Markdown source — additional $0.50/1k for body content.

The includeBody=true option performs one additional API call per article and triggers the article-body premium event. Full Markdown source means clean, structured text — no HTML post-processing needed for LLM training pipelines.

Competitor comparison

	This actor	epctex/dev-to-scraper	Any alternative
Data source	Official Forem API	HTML scraping	?
Proxy needed	No	Yes	Unknown
Full body Markdown	Yes (toggle)	Partial	No
parse_confidence	Yes	No	No
Batch input	Yes	No	No

Dev.to runs on Forem — this actor works on any public Forem instance.

AI training datasets

Full Markdown source (includeBody=true) enables AI fine-tuning use cases: tagged technical articles, clean structured text, and engagement signals — ready for LLM training pipelines with no HTML post-processing.

---

This actor is not affiliated with Forem or DEV Community.

Integrations

Built for developer-marketing teams and dataset builders tracking content trends and article metadata on Dev.to — the JSON/dataset output drops into the tools you already run, no glue code:

• n8n / Make / Zapier — trigger a run or pipe every new dataset item into 500+ apps (Google Sheets, Airtable, Slack, HubSpot, your database) with no code: n8n, Make, Zapier.
• Webhooks — fire your own endpoint the moment a run finishes, to push results straight into your pipeline (docs).
• MCP server — expose this actor as a tool to Claude, Cursor, or any MCP client so an AI agent can pull this data mid-conversation (guide).
• API & SDKs — fetch the dataset as JSON, CSV, or Excel through the Apify REST API or the Python / JS SDKs.

See all Apify integrations.