# Dev.to Article Scraper — Tags, Authors & Full Text (`bovi/devto-scraper`) Actor

Scrape Dev.to articles by tag, author, keyword search, or top posts via the official Forem public API. No proxy, no auth. Returns title, tags, reactions, comments, reading time, author, published date, and optional body markdown. Pay per article.

- **URL**: https://apify.com/bovi/devto-scraper.md
- **Developed by:** [Vitalii Bondarev](https://apify.com/bovi) (community)
- **Categories:** Developer tools, MCP servers, AI
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.97 / 1,000 dev.to article scraper — tags, authors & full texts

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Dev.to Article Scraper — Tags, Authors & Full Text | from $1/1K No Proxy

Used by developer marketing teams tracking content trends, AI training dataset builders, and content intelligence platforms.

Scrape articles from **Dev.to** (the world's largest Forem developer community) via the **official public Forem REST API**. No proxy, no authentication, no fragile DOM parsing — just clean, reliable JSON data at scale.

### Features

- **5 scraping modes:** by tag, by author username, by article ID, full-text search, top articles
- **14 structured fields** per article: title, description, URL, tags (array), author, reactions, comments, reading time, published date, optional body markdown
- **`includeBody` option:** fetch full Markdown source via the detail endpoint
- **Batch input:** pass multiple tags/usernames/queries in a single run
- **`parse_confidence` field** in every record — machine-readable data quality signal; 1.0 = perfect, deductions logged to `warnings`
- **Zero proxy cost** — official API, plain HTTPS, no access friction
- **Pay-per-result** pricing — you only pay for what you get

### Use Cases

- Content research & trend analysis across the dev community
- Author portfolio scraping for lead generation or outreach
- Building training datasets (tagged articles with Markdown source)
- Monitoring publication activity for specific tags or users
- Competitive content intelligence (reactions, comments, reading time)

### Input

| Field | Type | Default | Description |
|---|---|---|---|
| `mode` | string | `byTag` | `byTag` / `byUsername` / `byArticleIds` / `search` / `top` |
| `tags` | string\[] | `["python"]` | Tag names to scrape (mode=byTag) |
| `usernames` | string\[] | `[]` | Author usernames (mode=byUsername) |
| `articleIds` | string\[] | `[]` | Numeric article IDs (mode=byArticleIds) |
| `searchQueries` | string\[] | `[]` | Search keywords (mode=search) |
| `topDays` | integer | `30` | Top articles from last N days (mode=top) |
| `maxItems` | integer | `100` | Max total articles (0 = unlimited) |
| `includeBody` | boolean | `false` | Fetch body\_markdown via detail endpoint |

### Output Schema

| Field | Type | Description |
|---|---|---|
| `article_id` | integer | Numeric Dev.to article ID |
| `title` | string | Article title |
| `description` | string | Subtitle / teaser |
| `url` | string | Canonical article URL |
| `tags` | string\[] | Tag list |
| `author_name` | string | Author display name |
| `author_username` | string | Author @handle |
| `positive_reactions` | integer | Hearts / reactions count |
| `comments_count` | integer | Number of comments |
| `reading_time_min` | integer | Estimated reading time (minutes) |
| `published_at` | string | ISO 8601 UTC publish date |
| `body_markdown` | string|null | Full Markdown body (requires includeBody=true) |
| `parse_confidence` | float | Data quality score (1.0 = perfect) |
| `warnings` | string\[] | Machine-readable quality warnings |

### Example Output

```json
{
  "article_id": 3745852,
  "title": "Pick a better video thumbnail automatically with FFmpeg, PySceneDetect, and CLIP",
  "description": "We'll build a pipeline that takes any video file...",
  "url": "https://dev.to/masonwritescode/pick-a-better-video-thumbnail-automatically-4gpk",
  "tags": ["video", "python", "ai", "tutorial"],
  "author_name": "Mason K",
  "author_username": "masonwritescode",
  "positive_reactions": 42,
  "comments_count": 5,
  "reading_time_min": 6,
  "published_at": "2026-05-31T09:14:33Z",
  "body_markdown": null,
  "parse_confidence": 1.0,
  "warnings": []
}
```

### API & Reliability

Uses the **official Forem REST API** (`dev.to/api`). Forem is open-source and the API is designed for programmatic access — schema stability is much higher than scraping HTML.

The `parse_confidence` field provides a data-quality signal in every record. A score below 0.8 indicates missing critical fields and is logged to `warnings`. This lets you filter or alert on data-quality degradation without manual inspection.

### Use with AI agents (MCP)

This scraper is callable as a **tool by AI agents** (Claude Desktop, Cursor, VS Code,
n8n, LangGraph, CrewAI, or any MCP-compatible client) via Apify's hosted Model Context
Protocol server. An agent uses it to **fetch live Dev.to articles by tag, author, or keyword
mid-conversation** — e.g. "find the top Python articles on Dev.to this month" or "what has
@ben published recently?".

Point your MCP client at this single tool:

```json
{
  "mcpServers": {
    "apify": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://mcp.apify.com/?tools=bovi/devto-scraper",
        "--header",
        "Authorization: Bearer <YOUR_APIFY_TOKEN>"
      ]
    }
  }
}
```

Minimal call an agent can make:

```json
{ "mode": "byTag", "tags": ["python"], "maxItems": 20 }
```

Returns clean, flat rows the agent can reason over directly:

```json
{
  "article_id": 3745852,
  "title": "Pick a better video thumbnail automatically with FFmpeg, PySceneDetect, and CLIP",
  "url": "https://dev.to/masonwritescode/pick-a-better-video-thumbnail-automatically-4gpk",
  "tags": ["video", "python", "ai", "tutorial"],
  "author_username": "masonwritescode",
  "positive_reactions": 42,
  "comments_count": 5,
  "reading_time_min": 6,
  "published_at": "2026-05-31T09:14:33Z",
  "parse_confidence": 1.0,
  "warnings": []
}
```

**Reliability for agents:** data comes from the **official Forem REST API** (not HTML scraping),
so rows don't break on UI redesigns. Every record includes a `parse_confidence` score
(1.0 = all critical fields present) and a `warnings` array for machine-readable quality
signals — useful for agent pipelines that need to filter or alert on data quality.
Keep `maxItems` to 50–200 and `includeBody=false` (default) to keep responses token-lean.
No API key needed inside the tool — auth is your Apify token in the client config above.

### Pricing examples

| Run | Items | Cost |
|---|---|---|
| 100 articles by tag (metadata) | 100 | ~$0.10 |
| 1,000 articles with body Markdown | 1,000 | ~$1.50 |
| Top 200 articles last 30 days | 200 | ~$0.20 |
| Daily tag monitoring, 30 days | 3,000/mo | ~$3.00/mo |

### FAQ

**Do I need a proxy or API key?**
No. The Forem/Dev.to public API requires no authentication and no proxy. Zero extra cost for buyers.

**What formats can I export results in?**
JSON, CSV, JSONL, Excel — all via Apify dataset export. The output drops straight into Google Sheets, Airtable, Notion, or any BI tool.

**Can I schedule it to track new articles?**
Yes. Use Apify Schedules to run daily on a tag or author and catch new publications automatically. Pair with a webhook to post new articles to Slack or a database.

**What if the actor returns empty results?**
Check that your tag slug is lowercase and matches Dev.to's tag list (e.g. `javascript` not `JavaScript`). For `search` mode, try a broader query — Dev.to search indexes titles and descriptions. Errors are reported in the `OUTPUT` key-value store with a reason code.

### Pricing

**Pricing:** $1.00 per 1,000 articles (metadata). Enable `includeBody` for full Markdown source — additional $0.50/1k for body content.

The `includeBody=true` option performs one additional API call per article and triggers the `article-body` premium event. Full Markdown source means clean, structured text — no HTML post-processing needed for LLM training pipelines.

### Competitor comparison

| | This actor | epctex/dev-to-scraper | Any alternative |
|---|---|---|---|
| Data source | Official Forem API | HTML scraping | ? |
| Proxy needed | No | Yes | Unknown |
| Full body Markdown | Yes (toggle) | Partial | No |
| `parse_confidence` | Yes | No | No |
| Batch input | Yes | No | No |

Dev.to runs on **Forem** — this actor works on any public Forem instance.

### AI training datasets

Full Markdown source (`includeBody=true`) enables AI fine-tuning use cases: tagged technical articles, clean structured text, and engagement signals — ready for LLM training pipelines with no HTML post-processing.

***

*This actor is not affiliated with Forem or DEV Community.*

### Integrations

Built for developer-marketing teams and dataset builders tracking content trends and article metadata on Dev.to — the JSON/dataset output drops into the tools you already run, no glue code:

- **n8n / Make / Zapier** — trigger a run or pipe every new dataset item into 500+ apps (Google Sheets, Airtable, Slack, HubSpot, your database) with no code: [n8n](https://docs.apify.com/platform/integrations/n8n), [Make](https://docs.apify.com/platform/integrations/make), [Zapier](https://docs.apify.com/platform/integrations/zapier).
- **Webhooks** — fire your own endpoint the moment a run finishes, to push results straight into your pipeline ([docs](https://docs.apify.com/platform/integrations/webhooks)).
- **MCP server** — expose this actor as a tool to Claude, Cursor, or any [MCP client](https://mcp.apify.com) so an AI agent can pull this data mid-conversation ([guide](https://blog.apify.com/how-to-use-mcp/)).
- **API & SDKs** — fetch the dataset as JSON, CSV, or Excel through the Apify REST API or the Python / JS SDKs.

See all [Apify integrations](https://apify.com/integrations).

# Actor input Schema

## `mode` (type: `string`):

Scraping mode. Pick one: 'byTag' — fetch articles by tag (requires 'tags'); 'byUsername' — all articles by author (requires 'usernames'); 'byArticleIds' — specific articles (requires 'articleIds'); 'search' — full-text search (requires 'searchQueries'); 'top' — top articles by reactions over last N days (uses 'topDays'). Default 'byTag'.

## `tags` (type: `array`):

One or more Dev.to tag slugs to fetch articles for. Each tag runs separately and results are merged. Example: \["python", "javascript", "webdev"]. Used only when mode='byTag'.

## `usernames` (type: `array`):

Dev.to @handles to fetch all articles from. Example: \["ben", "t\_jallice"]. Used only when mode='byUsername'.

## `articleIds` (type: `array`):

Specific numeric Dev.to article IDs to fetch, e.g. \["3745852", "1234567"]. Each ID is fetched via the detail endpoint — body\_markdown is always available regardless of includeBody. Used only when mode='byArticleIds'.

## `searchQueries` (type: `array`):

Full-text keyword queries against Dev.to's search index. Each query runs separately and results are merged. Example: \["vector databases", "rust async"]. Used only when mode='search'.

## `topDays` (type: `integer`):

Fetch top articles by reactions from the last N days. Common values: 7 (weekly), 30 (monthly), 365 (yearly). Used only when mode='top'.

## `maxItems` (type: `integer`):

Maximum total articles to return across all tags/usernames/queries. Set to 50–200 for agent use to avoid large payloads. 0 = no limit.

## `includeBody` (type: `boolean`):

Fetch full article body in Markdown format (body\_markdown field). Requires one extra API call per article. Set false (default) to keep responses token-lean for agent use.

## Actor input object example

```json
{
  "mode": "byTag",
  "tags": [
    "python"
  ],
  "usernames": [
    "ben"
  ],
  "articleIds": [
    "3745852"
  ],
  "searchQueries": [
    "vector databases"
  ],
  "topDays": 30,
  "maxItems": 100,
  "includeBody": false
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset containing Devto Scraper records (article\_id, title, url, tags, author\_username, positive\_reactions, comments\_count, reading\_time\_min, published\_at, parse\_confidence, description, author\_name).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "mode": "byTag",
    "tags": [
        "python"
    ],
    "usernames": [
        "ben"
    ],
    "articleIds": [
        "3745852"
    ],
    "searchQueries": [
        "vector databases"
    ],
    "topDays": 30,
    "maxItems": 100,
    "includeBody": false
};

// Run the Actor and wait for it to finish
const run = await client.actor("bovi/devto-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "mode": "byTag",
    "tags": ["python"],
    "usernames": ["ben"],
    "articleIds": ["3745852"],
    "searchQueries": ["vector databases"],
    "topDays": 30,
    "maxItems": 100,
    "includeBody": False,
}

# Run the Actor and wait for it to finish
run = client.actor("bovi/devto-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "mode": "byTag",
  "tags": [
    "python"
  ],
  "usernames": [
    "ben"
  ],
  "articleIds": [
    "3745852"
  ],
  "searchQueries": [
    "vector databases"
  ],
  "topDays": 30,
  "maxItems": 100,
  "includeBody": false
}' |
apify call bovi/devto-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,bovi/devto-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/h7J2zxWVftBH6yVZc/builds/KFgoHN3cK4JB72WEa/openapi.json