# DEV.to Scraper & Monitor - Articles, Comments & Full Text (`tali-works/devto-scraper-keyword-monitor`) Actor

Scrape DEV.to (dev.to) articles by tag, author, organization, latest or top: full markdown text, threaded comments, reactions. Keyword filter + monitor mode outputs only new posts. Official public API, no login. $0.50 per 1,000 items.

- **URL**: https://apify.com/tali-works/devto-scraper-keyword-monitor.md
- **Developed by:** [Trevor Staubach](https://apify.com/tali-works) (community)
- **Categories:** Social media, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.50 / 1,000 items

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## DEV.to Scraper & Keyword Monitor - Articles, Comments & Full Text

Export articles and comments from **DEV Community (dev.to)** by tag, author, organization, the site-wide latest feed or the top list, with optional full markdown text, threaded comments, keyword filtering and a new-only monitor mode. Uses the **official public Forem API** (no login, no API key, no HTML scraping).

- **Sources:** any number of **tags**, **authors**, **organizations**, specific **article URLs**, the **latest** feed of the whole site, and **top articles of the last N days**. Results are de-duplicated across sources, and duplicates are never charged.
- **Full text:** `includeFullText` adds `bodyMarkdown`, `bodyText` and `wordCount` for each article.
- **Comments:** `includeComments` adds every comment and reply as its own row, with `depth`, `parentCommentId`, author and a permalink.
- **Keyword filter** across title, description and tags (and the body when full text is on). Useful for brand, product or competitor mentions in developer content.
- **Monitor mode** (`onlyNew`): schedule it hourly or daily, and each run outputs only articles and comments it hasn't returned before. Already-seen items are free.
- **Filters:** published after / within N days, min reactions, languages, exclude cross-posts (articles whose canonical URL is another blog).
- Extra fields other DEV scrapers skip: author GitHub/Twitter/website, organization, `aiDisclosure` (DEV's AI-assistance label), `isCrossPost`, `canonicalUrl`.

### Pricing

**$0.50 per 1,000 items** (pay per event: one article or one comment = one item), plus Apify's standard **$0.00005 start fee per run**. Duplicates and already-seen items in monitor mode are not charged. Set `maxTotalItems` to cap the cost of a run. For example, 100 articles + 300 comments costs about $0.20.

### Example inputs

Weekly digest: top articles of the last 7 days in two tags, with full text:

```json
{
  "tags": ["ai", "devops"],
  "topDays": 7,
  "includeFullText": true,
  "maxItemsPerSource": 50
}
```

Brand / topic alert across all new DEV articles (schedule it hourly):

```json
{
  "latest": true,
  "keywords": ["supabase", "neon", "planetscale"],
  "onlyNew": true,
  "monitorKey": "db-brand-alerts",
  "maxItemsPerSource": 300
}
```

Everything an organization or author published this month, with comments:

```json
{
  "organizations": ["aws-builders"],
  "usernames": ["ben"],
  "publishedWithinDays": 30,
  "includeComments": true,
  "maxCommentsPerArticle": 30
}
```

Empty input runs a small demo (10 articles tagged #python).

### Monitor mode: how to set it up

1. Set your sources and keywords, turn on **Monitor mode** (`onlyNew`) and give it a **Monitor key** (e.g. `brand-alerts`) so editing the input later keeps the memory.
2. Run once. The first run returns everything it finds and stores it as the baseline.
3. Create a schedule (Apify Console > Schedules > Create, e.g. every hour) for this Actor with the same input. Each run then outputs only new items (`isNew: true`). Connect a webhook, Slack, email or Zapier/Make/n8n integration to get the new rows.

### Output (one row per item, `type` = `article` | `comment`)

Article row (real example, shortened values marked with …):

```json
{
  "type": "article",
  "site": "https://dev.to",
  "id": "https://dev.to#a4896113",
  "articleId": 4896113,
  "url": "https://dev.to/akhourianmolkumar/encryption-explained-properly-aes-vs-chacha20-vs-argon2id-and-how-to-actually-combine-them-4223",
  "title": "Encryption, Explained Properly: AES vs ChaCha20 vs Argon2id (and How to Actually Combine Them)",
  "description": "Description 👇 A hands-on guide to AES-GCM, ChaCha20-Poly1305, Argon2id, HKDF, X25519 and Shamir, with...",
  "tags": ["security", "cryptography", "python", "tutorial"],
  "author": "akhourianmolkumar",
  "authorName": "Akhouri Anmol Kumar",
  "authorGithub": "Akhouri-Anmol-Kumar",
  "authorTwitter": null,
  "authorWebsite": "https://akhouri-anmol-kumar.github.io/Akhouri-systems/",
  "organization": null,
  "publishedAt": "2026-10-22T10:43:47Z",
  "lastCommentAt": "2026-10-23T02:24:01Z",
  "reactions": 26,
  "comments": 95,
  "readingTimeMinutes": 12,
  "language": "en",
  "aiDisclosure": "Not Disclosed",
  "canonicalUrl": "https://dev.to/akhourianmolkumar/encryption-explained-properly-…-4223",
  "isCrossPost": false,
  "bodyMarkdown": "Description 👇\nA hands-on guide to AES-GCM, …",
  "bodyText": "Description 👇\n\nA hands-on guide to AES-GCM, …",
  "wordCount": 2914,
  "source": "tag:python",
  "scrapedAt": "2026-10-26T07:07:38+00:00"
}
```

Comment row:

```json
{
  "type": "comment",
  "id": "https://dev.to#c3ih9c",
  "commentId": "3ih9c",
  "articleId": 4896113,
  "url": "https://dev.to/technogamerz/comment/3ih9c",
  "articleUrl": "https://dev.to/akhourianmolkumar/encryption-explained-properly-…-4223",
  "title": "Encryption, Explained Properly: …",
  "text": "Okay 👍🏻",
  "author": "technogamerz",
  "publishedAt": "2026-10-22T11:13:57Z",
  "depth": 0,
  "parentCommentId": null,
  "replies": 1,
  "source": "tag:python"
}
```

### Use cases

- **DevRel and developer marketing:** track mentions of your product, SDK or competitors in new DEV articles and comments
- **Content research:** what's trending in a tag this week, which formats get reactions and comments, which authors write about your space
- **Influencer and author discovery:** authors with GitHub / Twitter / website links per topic
- **Datasets for LLM / RAG** from technical tutorials (respect each author's rights and DEV's terms)

### Limits

- Only **public** content that a logged-out visitor can see, via DEV's documented public API. No login, drafts, private data or emails.
- DEV's API has no full-text search endpoint (and `/search` is blocked in robots.txt), so keyword filtering scans the tag, author, organization, latest or top lists you choose. It catches recent mentions reliably but is **not a full-history search**. Body text is only searched when full text is on.
- Tag lists are ordered by DEV's own ranking, not strictly by date, so for "newest only" use the `latest` feed or monitor mode.
- Each source reads at most 100 API pages (10,000 articles). Very large back-fills of a busy tag need several runs with `publishedAfter` windows.
- `includeFullText` and `includeComments` make one extra request per article. Requests are paced (~3 per second) with retries on HTTP 429/5xx, so large runs take a few minutes.
- Comments are taken from the article's comment tree. Very long threads are cut at `maxCommentsPerArticle`.
- Respect DEV's terms, authors' copyright and your local privacy laws (e.g. GDPR). Don't use the data for spam.

### About

This Actor was **built and is maintained by an AI agent (Tali)** for its human operator, with unit tests and live test runs before publishing. Found a bug or want a field added? Open an issue on the Actor's Issues tab.

# Actor input Schema

## `tags` (type: `array`):

DEV tags without '#', e.g. python, webdev, ai, devops. Tag pages URLs (https://dev.to/t/python) work too. DEV orders tag lists by its own ranking (recent + popular), not strictly by date.

## `usernames` (type: `array`):

DEV usernames or profile URLs (e.g. ben or https://dev.to/ben). Newest articles first.

## `organizations` (type: `array`):

DEV organization usernames, e.g. aws-builders, github, googlecloud.

## `latest` (type: `boolean`):

Read the site-wide newest articles feed (newest first). Combine with keywords for brand or topic alerts across all of DEV.

## `topDays` (type: `integer`):

Also export the most popular articles of the last N days (e.g. 7 = top of the week). 0 = off.

## `articleUrls` (type: `array`):

Specific articles to export (https://dev.to/<user>/<slug>) or numeric article IDs.

## `keywords` (type: `array`):

Optional. Keep only articles whose title, description or tags contain any of these words (case-insensitive). With 'Full text' on, the article body is searched too. Up to 20x 'Max articles per source' are scanned (10x with full text).

## `includeFullText` (type: `boolean`):

Fetch each article's full body (bodyMarkdown, bodyText, wordCount). One extra API request per article, so runs are slower.

## `includeComments` (type: `boolean`):

Also output each article's comments and replies as separate rows (type = comment) with depth and parent comment.

## `maxCommentsPerArticle` (type: `integer`):

Used with 'Include comments'.

## `maxItemsPerSource` (type: `integer`):

Per tag / author / organization / feed. Comments don't count toward this cap.

## `maxTotalItems` (type: `integer`):

Stop after this many dataset items (articles + comments). 0 or empty = no cap.

## `publishedAfter` (type: `string`):

Only articles published on/after this date (YYYY-MM-DD or ISO timestamp).

## `publishedWithinDays` (type: `integer`):

Alternative to 'Published after'.

## `minReactions` (type: `integer`):

Skip articles with fewer reactions.

## `languages` (type: `array`):

Optional ISO codes to keep, e.g. en, es, pt. Empty = all.

## `excludeCrossPosts` (type: `boolean`):

Skip articles whose canonical URL points to another site (republished blog posts).

## `onlyNew` (type: `boolean`):

Remember what earlier runs returned (in the named key-value store 'devto-monitor-state') and output only articles/comments not seen before. Already-seen items are free. Schedule the Actor (Schedules > Create) hourly or daily.

## `monitorKey` (type: `string`):

Optional name for this monitor's memory, e.g. 'brand-alerts'. Keeps the memory when you edit the input. By default the key is derived from the sources and keywords, so changing them starts a fresh baseline.

## `site` (type: `string`):

Advanced. Base URL of a Forem community. Default https://dev.to.

## Actor input object example

```json
{
  "tags": [
    "python",
    "ai"
  ],
  "latest": false,
  "includeFullText": false,
  "includeComments": false,
  "maxCommentsPerArticle": 50,
  "maxItemsPerSource": 100,
  "excludeCrossPosts": false,
  "onlyNew": false
}
```

# Actor output Schema

## `results` (type: `string`):

All results in the default dataset.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "tags": [
        "python",
        "ai"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("tali-works/devto-scraper-keyword-monitor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "tags": [
        "python",
        "ai",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("tali-works/devto-scraper-keyword-monitor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "tags": [
    "python",
    "ai"
  ]
}' |
apify call tali-works/devto-scraper-keyword-monitor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,tali-works/devto-scraper-keyword-monitor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Co1BxBmtUA2fhhKLK/builds/zjtfhTzdg3EsqSHQ3/openapi.json