DEV.to Scraper: Articles, Tags & Author Metrics

DEV.to Articles Scraper pulls structured records from dev.to, title, description, tags, reactions, comments, reading time, canonical URL, cover image, author handle and GitHub profile, and optionally the complete markdown body.

DEV.to publishes this data for programmatic use, so the Actor needs nothing configured to reach it: no browser, no proxy, no login.

Agent skill: SKILL.md

https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/devto-articles-scraper.md

What you get

Output field	Meaning
id	DEV.to article ID, the stable key for de-duplication
title, description	Headline and the auto-generated summary line
url, canonicalUrl	The dev.to URL and the author's declared canonical URL (often their own blog)
tags	Every tag on the post, as an array
reactionsCount, commentsCount	Public reactions and comment count
readingTimeMinutes	DEV.to's own estimate
publishedAt	ISO-8601 publish timestamp
author, authorName, authorGithub	Handle, display name, and GitHub username where the author linked one
coverImage	Cover image, falling back to the generated share card
bodyMarkdown	Full article source in markdown, only when includeBody is on
scrapedAt	Run timestamp

A RUN_SUMMARY record in the key-value store holds per-run counts, the filters used, how many bodies were fetched, and any source query that failed.

Common use cases

Find trending developer topics, take the week's best posts across the tags you care about and see what is actually resonating.

{
 "tags": ["ai", "webdev", "python"],
 "topArticles": 7,
 "maxResults": 300
}

Track your team's content performance, pull every post by an author and diff reactions and comments over time.

{
 "username": "ben",
 "maxResults": 0
}

Source technical writers by engagement, sweep a niche tag, sort the dataset by reactionsCount, and you have a ranked shortlist with GitHub handles attached.

{
 "tags": ["rust"],
 "topArticles": 90,
 "includeBody": true,
 "maxResults": 200
}

Quick start

The latest global feed, 100 articles:

{}

Two tags with full article text:

{
 "tags": ["python", "webdev"],
 "includeBody": true,
 "maxResults": 50
}

Input

Field	Type	Default	Notes
tags	array	[]	DEV.to tag names. Each tag runs as its own query. #rust, Rust and rust are all accepted.
username	string	""	A single author's handle, ben from https://dev.to/ben. A full profile URL is accepted too.
topArticles	integer	0	Order by reactions within the last N days instead of by recency. 7 = this week's best. 0 = chronological.
includeBody	boolean	false	Fetch the full markdown body. Costs one extra request per article.
maxResults	integer	100	Total cap across all sources. 0 = no limit.

Which combinations make sense. DEV.to's API ignores username when a tag is also present, the tag wins and you get articles from everyone. So this Actor runs each tag and the username as separate queries and merges the results, de-duplicating on id. Passing tags: ["rust"] plus username: "ben" therefore gives you the Rust feed and everything Ben wrote, not Ben's Rust posts. If you want the intersection, pass just the username and filter the dataset on tags.

topArticles applies to every query in the run. maxResults is a global cap consumed in source order, tags first, then the username query, so put the source you care most about first.

Output example

{
 "id": 4323385,
 "title": "General Challenge Updates Moving Forward",
 "description": "Hey all, as part of our expanding challenge program we want to update and clarify some challenge.",
 "url": "https://dev.to/thepracticaldev/general-challenge-updates-moving-forward-5h39",
 "canonicalUrl": "https://dev.to/thepracticaldev/general-challenge-updates-moving-forward-5h39",
 "tags": ["devchallenge"],
 "reactionsCount": 14,
 "commentsCount": 1,
 "readingTimeMinutes": 2,
 "publishedAt": "2026-08-05T19:27:14Z",
 "author": "thepracticaldev",
 "authorName": "dev.to staff",
 "authorGithub": null,
 "coverImage": "https://media2.dev.to/dynamic/image/width=1200,height=627./mgecj41w91psrct0upxp.png",
 "bodyMarkdown": "---\ntitle: General Challenge Updates Moving Forward\n---\n\nHey all …",
 "scrapedAt": "2026-08-06T12:00:00.000Z"
}

RUN_SUMMARY looks like this:

{
 "sourcesRequested": 2,
 "sourcesFailed": 0,
 "failures": [],
 "articlesSaved": 100,
 "bodiesFetched": 100,
 "bodiesFailed": 0,
 "filters": {
 "tags": ["python", "webdev"],
 "username": null,
 "topArticles": 7,
 "includeBody": true,
 "maxResults": 100
 },
 "finishedAt": "2026-08-06T12:00:04.812Z"
}

Finding a tag or username

URL you see	What to pass
https://dev.to/t/webdev	tags: ["webdev"]
https://dev.to/t/machinelearning/top/week	tags: ["machinelearning"], topArticles: 7
https://dev.to/ben	username: "ben"
https://dev.to/ben/some-post-slug	username: "ben"

Tags are always lowercase and unspaced on DEV.to, machinelearning, not Machine Learning. The Actor normalises what you give it, but a tag that does not exist simply returns zero articles rather than an error.

Limits and behaviour

• The body needs a second request. GET /api/articles never includes article text. bodyMarkdown comes from GET /api/articles/{id}, one call per article, with a short pause between calls. Turning includeBody on roughly doubles run time; a body that fails to fetch is recorded as null and counted in RUN_SUMMARY.bodiesFailed rather than losing the whole article.
• Pagination is automatic. Pages of 100 are requested until a short page comes back or maxResults is hit. Nothing is silently truncated, the per-source log line reports listed / new / saved counts.
• One bad source never kills the run. A failing query is logged into RUN_SUMMARY.failures and the Actor continues; it only throws if every source failed.
• Transient errors are retried. 429 and 5xx get three attempts with linear backoff. A 404 is treated as fatal immediately.
• De-duplication is global. An article that appears under two of your tags is saved once.
• Public data only. No authentication, no private posts, no personal data beyond the public author profile fields DEV.to itself publishes.

API example

curl -X POST "https://api.apify.com/v2/acts/arman-bd~devto-articles-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "tags": ["python", "webdev"],
 "topArticles": 7,
 "maxResults": 50
 }'

JavaScript example

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('arman-bd/devto-articles-scraper').call({
 tags: ['rust'],
 topArticles: 30,
 maxResults: 100,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const a of items) console.log(`${a.reactionsCount}\t${a.author}\t${a.title}`);

Defaults: 1 GB memory, 15 minute timeout.

FAQ

Do I need a proxy? No. Proxy configuration is not required to run this Actor.

Do I need a DEV.to account or API key? No. You supply no credentials.

What happens if a source is unavailable? It is logged, recorded in RUN_SUMMARY.failures, and the run continues with the remaining sources. The Actor only errors out when every source fails.

Can I schedule it? Yes, it is designed for it. Run daily on your tags and diff on id to get only new posts. Leave includeBody off for cheap change-detection sweeps.

Why is canonicalUrl sometimes a different domain? Because the author cross-posted from their own blog and declared it. That is the real value DEV.to reports, and it is often exactly what you want for attribution.

Why is coverImage set when the post has no cover? DEV.to generates a share card for every post. When cover_image is null the Actor falls back to social_image, which is what DEV.to itself renders in feeds.

Can I get comments? Not in this Actor, commentsCount is included, but comment bodies live on a separate endpoint and would change the per-item cost model.

Can I integrate it with something else? Yes, Apify API, client libraries, webhooks, scheduled runs, dataset exports (JSON/CSV/Excel) or MCP. Output is structured JSON.