# DEV.to Scraper: Articles, Tags & Author Metrics (`arman-bd/devto-articles-scraper`) Actor

Scrape DEV.to articles: title, tags, reactions, comments, reading time, canonical URL and author profile. Filter by tag, username or date. Official API, no key.

- **URL**: https://apify.com/arman-bd/devto-articles-scraper.md
- **Developed by:** [Arman Hossain](https://apify.com/arman-bd) (community)
- **Categories:** Developer tools, Automation, MCP servers
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.37 / 1,000 article scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## DEV.to Scraper: Articles, Tags & Author Metrics

![DEV.to Scraper: Articles by tag, author or trending window, reactions, comments, reading time and the author's GitHub handle](https://api.apify.com/v2/key-value-stores/ZQOcNAOHrIgTacAmy/records/devto-articles-scraper.jpg)

**DEV.to Articles Scraper** pulls structured records from **dev.to**, title, description, tags, reactions, comments, reading time, canonical URL, cover image, author handle and GitHub profile, and optionally the complete markdown body.

DEV.to publishes this data for programmatic use, so the Actor needs nothing configured to reach it: no browser, no proxy, no login.

**Agent skill: [SKILL.md](https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/devto-articles-scraper.md)**

```
https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/devto-articles-scraper.md
```

### What you get

| Output field | Meaning |
|---|---|
| `id` | DEV.to article ID, the stable key for de-duplication |
| `title`, `description` | Headline and the auto-generated summary line |
| `url`, `canonicalUrl` | The dev.to URL and the author's declared canonical URL (often their own blog) |
| `tags` | Every tag on the post, as an array |
| `reactionsCount`, `commentsCount` | Public reactions and comment count |
| `readingTimeMinutes` | DEV.to's own estimate |
| `publishedAt` | ISO-8601 publish timestamp |
| `author`, `authorName`, `authorGithub` | Handle, display name, and GitHub username where the author linked one |
| `coverImage` | Cover image, falling back to the generated share card |
| `bodyMarkdown` | Full article source in markdown, only when `includeBody` is on |
| `scrapedAt` | Run timestamp |

A `RUN_SUMMARY` record in the key-value store holds per-run counts, the filters used, how many bodies were fetched, and any source query that failed.

### Common use cases

**Find trending developer topics**, take the week's best posts across the tags you care about and see what is actually resonating.

```json
{
 "tags": ["ai", "webdev", "python"],
 "topArticles": 7,
 "maxResults": 300
}
```

**Track your team's content performance**, pull every post by an author and diff reactions and comments over time.

```json
{
 "username": "ben",
 "maxResults": 0
}
```

**Source technical writers by engagement**, sweep a niche tag, sort the dataset by `reactionsCount`, and you have a ranked shortlist with GitHub handles attached.

```json
{
 "tags": ["rust"],
 "topArticles": 90,
 "includeBody": true,
 "maxResults": 200
}
```

### Quick start

The latest global feed, 100 articles:

```json
{}
```

Two tags with full article text:

```json
{
 "tags": ["python", "webdev"],
 "includeBody": true,
 "maxResults": 50
}
```

### Input

| Field | Type | Default | Notes |
|---|---|---|---|
| `tags` | array | `[]` | DEV.to tag names. Each tag runs as its own query. `#rust`, `Rust` and `rust` are all accepted. |
| `username` | string | `""` | A single author's handle, `ben` from `https://dev.to/ben`. A full profile URL is accepted too. |
| `topArticles` | integer | `0` | Order by reactions within the last N days instead of by recency. `7` = this week's best. `0` = chronological. |
| `includeBody` | boolean | `false` | Fetch the full markdown body. Costs **one extra request per article**. |
| `maxResults` | integer | `100` | Total cap across all sources. `0` = no limit. |

**Which combinations make sense.** DEV.to's API ignores `username` when a `tag` is also present, the tag wins and you get articles from everyone. So this Actor runs each tag and the username as **separate queries** and merges the results, de-duplicating on `id`. Passing `tags: ["rust"]` plus `username: "ben"` therefore gives you *the Rust feed **and** everything Ben wrote*, not *Ben's Rust posts*. If you want the intersection, pass just the username and filter the dataset on `tags`.

`topArticles` applies to every query in the run. `maxResults` is a global cap consumed in source order, tags first, then the username query, so put the source you care most about first.

### Output example

```json
{
 "id": 4323385,
 "title": "General Challenge Updates Moving Forward",
 "description": "Hey all, as part of our expanding challenge program we want to update and clarify some challenge.",
 "url": "https://dev.to/thepracticaldev/general-challenge-updates-moving-forward-5h39",
 "canonicalUrl": "https://dev.to/thepracticaldev/general-challenge-updates-moving-forward-5h39",
 "tags": ["devchallenge"],
 "reactionsCount": 14,
 "commentsCount": 1,
 "readingTimeMinutes": 2,
 "publishedAt": "2026-08-05T19:27:14Z",
 "author": "thepracticaldev",
 "authorName": "dev.to staff",
 "authorGithub": null,
 "coverImage": "https://media2.dev.to/dynamic/image/width=1200,height=627./mgecj41w91psrct0upxp.png",
 "bodyMarkdown": "---\ntitle: General Challenge Updates Moving Forward\n---\n\nHey all …",
 "scrapedAt": "2026-08-06T12:00:00.000Z"
}
```

`RUN_SUMMARY` looks like this:

```json
{
 "sourcesRequested": 2,
 "sourcesFailed": 0,
 "failures": [],
 "articlesSaved": 100,
 "bodiesFetched": 100,
 "bodiesFailed": 0,
 "filters": {
 "tags": ["python", "webdev"],
 "username": null,
 "topArticles": 7,
 "includeBody": true,
 "maxResults": 100
 },
 "finishedAt": "2026-08-06T12:00:04.812Z"
}
```

### Finding a tag or username

| URL you see | What to pass |
|---|---|
| `https://dev.to/t/webdev` | `tags: ["webdev"]` |
| `https://dev.to/t/machinelearning/top/week` | `tags: ["machinelearning"], topArticles: 7` |
| `https://dev.to/ben` | `username: "ben"` |
| `https://dev.to/ben/some-post-slug` | `username: "ben"` |

Tags are always lowercase and unspaced on DEV.to, `machinelearning`, not `Machine Learning`. The Actor normalises what you give it, but a tag that does not exist simply returns zero articles rather than an error.

### Limits and behaviour

- **The body needs a second request.** `GET /api/articles` never includes article text. `bodyMarkdown` comes from `GET /api/articles/{id}`, one call per article, with a short pause between calls. Turning `includeBody` on roughly doubles run time; a body that fails to fetch is recorded as `null` and counted in `RUN_SUMMARY.bodiesFailed` rather than losing the whole article.
- **Pagination is automatic.** Pages of 100 are requested until a short page comes back or `maxResults` is hit. Nothing is silently truncated, the per-source log line reports listed / new / saved counts.
- **One bad source never kills the run.** A failing query is logged into `RUN_SUMMARY.failures` and the Actor continues; it only throws if *every* source failed.
- **Transient errors are retried.** 429 and 5xx get three attempts with linear backoff. A 404 is treated as fatal immediately.
- **De-duplication is global.** An article that appears under two of your tags is saved once.
- **Public data only.** No authentication, no private posts, no personal data beyond the public author profile fields DEV.to itself publishes.

### API example

```bash
curl -X POST "https://api.apify.com/v2/acts/arman-bd~devto-articles-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "tags": ["python", "webdev"],
 "topArticles": 7,
 "maxResults": 50
 }'
```

### JavaScript example

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('arman-bd/devto-articles-scraper').call({
 tags: ['rust'],
 topArticles: 30,
 maxResults: 100,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const a of items) console.log(`${a.reactionsCount}\t${a.author}\t${a.title}`);
```

**Defaults:** 1 GB memory, 15 minute timeout.

### FAQ

**Do I need a proxy?** No. Proxy configuration is not required to run this Actor.

**Do I need a DEV.to account or API key?** No. You supply no credentials.

**What happens if a source is unavailable?** It is logged, recorded in `RUN_SUMMARY.failures`, and the run continues with the remaining sources. The Actor only errors out when every source fails.

**Can I schedule it?** Yes, it is designed for it. Run daily on your tags and diff on `id` to get only new posts. Leave `includeBody` off for cheap change-detection sweeps.

**Why is `canonicalUrl` sometimes a different domain?** Because the author cross-posted from their own blog and declared it. That is the real value DEV.to reports, and it is often exactly what you want for attribution.

**Why is `coverImage` set when the post has no cover?** DEV.to generates a share card for every post. When `cover_image` is null the Actor falls back to `social_image`, which is what DEV.to itself renders in feeds.

**Can I get comments?** Not in this Actor, `commentsCount` is included, but comment bodies live on a separate endpoint and would change the per-item cost model.

**Can I integrate it with something else?** Yes, Apify API, client libraries, webhooks, scheduled runs, dataset exports (JSON/CSV/Excel) or MCP. Output is structured JSON.

# Actor input Schema

## `tags` (type: `array`):

DEV.to tag names, one source query each. The tag is the last path segment of a tag page. for https://dev.to/t/webdev the tag is 'webdev'. A leading '#' and mixed case are accepted and cleaned up.

## `username` (type: `string`):

Fetch one author's articles. The username is the first path segment of their profile URL. for https://dev.to/ben it is 'ben'. Runs as its own query, so it does NOT narrow the tags above; both sets are merged and de-duplicated.

## `topArticles` (type: `integer`):

Order each query by reactions instead of recency, restricted to the last N days. 7 gives the week's best, 30 the month's. Leave at 0 for the normal chronological feed.

## `includeBody` (type: `boolean`):

Fetch the complete article text as markdown. The listing endpoint does not carry it, so this costs one extra request per article and makes the run noticeably slower. Leave off if you only need metrics.

## `maxResults` (type: `integer`):

Cap the total number of articles saved across all tags and the author query combined. Set 0 to keep paging until every source is exhausted.

## Actor input object example

```json
{
  "tags": [
    "javascript",
    "#rust"
  ],
  "username": "ben",
  "topArticles": 7,
  "includeBody": false,
  "maxResults": 100
}
```

# Actor output Schema

## `items` (type: `string`):

Every record the run produced.

## `runsummary` (type: `string`):

The RUN_SUMMARY record from the run's key-value store.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "tags": [
        "python",
        "webdev"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("arman-bd/devto-articles-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "tags": [
        "python",
        "webdev",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("arman-bd/devto-articles-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "tags": [
    "python",
    "webdev"
  ]
}' |
apify call arman-bd/devto-articles-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,arman-bd/devto-articles-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/6QuszpcRGWcCYcRHg/builds/nPnOEwinrtP5EjaWc/openapi.json