DEV.to Scraper & Keyword Monitor - Articles, Comments & Full Text

Export articles and comments from DEV Community (dev.to) by tag, author, organization, the site-wide latest feed or the top list, with optional full markdown text, threaded comments, keyword filtering and a new-only monitor mode. Uses the official public Forem API (no login, no API key, no HTML scraping).

• Sources: any number of tags, authors, organizations, specific article URLs, the latest feed of the whole site, and top articles of the last N days. Results are de-duplicated across sources, and duplicates are never charged.
• Full text: includeFullText adds bodyMarkdown, bodyText and wordCount for each article.
• Comments: includeComments adds every comment and reply as its own row, with depth, parentCommentId, author and a permalink.
• Keyword filter across title, description and tags (and the body when full text is on). Useful for brand, product or competitor mentions in developer content.
• Monitor mode (onlyNew): schedule it hourly or daily, and each run outputs only articles and comments it hasn't returned before. Already-seen items are free.
• Filters: published after / within N days, min reactions, languages, exclude cross-posts (articles whose canonical URL is another blog).
• Extra fields other DEV scrapers skip: author GitHub/Twitter/website, organization, aiDisclosure (DEV's AI-assistance label), isCrossPost, canonicalUrl.

Pricing

$0.50 per 1,000 items (pay per event: one article or one comment = one item), plus Apify's standard $0.00005 start fee per run. Duplicates and already-seen items in monitor mode are not charged. Set maxTotalItems to cap the cost of a run. For example, 100 articles + 300 comments costs about $0.20.

Example inputs

Weekly digest: top articles of the last 7 days in two tags, with full text:

{
  "tags": ["ai", "devops"],
  "topDays": 7,
  "includeFullText": true,
  "maxItemsPerSource": 50
}

Brand / topic alert across all new DEV articles (schedule it hourly):

{
  "latest": true,
  "keywords": ["supabase", "neon", "planetscale"],
  "onlyNew": true,
  "monitorKey": "db-brand-alerts",
  "maxItemsPerSource": 300
}

Everything an organization or author published this month, with comments:

{
  "organizations": ["aws-builders"],
  "usernames": ["ben"],
  "publishedWithinDays": 30,
  "includeComments": true,
  "maxCommentsPerArticle": 30
}

Empty input runs a small demo (10 articles tagged #python).

Monitor mode: how to set it up

1. Set your sources and keywords, turn on Monitor mode (onlyNew) and give it a Monitor key (e.g. brand-alerts) so editing the input later keeps the memory.
2. Run once. The first run returns everything it finds and stores it as the baseline.
3. Create a schedule (Apify Console > Schedules > Create, e.g. every hour) for this Actor with the same input. Each run then outputs only new items (isNew: true). Connect a webhook, Slack, email or Zapier/Make/n8n integration to get the new rows.

Output (one row per item, type = article | comment)

Article row (real example, shortened values marked with …):

{
  "type": "article",
  "site": "https://dev.to",
  "id": "https://dev.to#a4896113",
  "articleId": 4896113,
  "url": "https://dev.to/akhourianmolkumar/encryption-explained-properly-aes-vs-chacha20-vs-argon2id-and-how-to-actually-combine-them-4223",
  "title": "Encryption, Explained Properly: AES vs ChaCha20 vs Argon2id (and How to Actually Combine Them)",
  "description": "Description 👇 A hands-on guide to AES-GCM, ChaCha20-Poly1305, Argon2id, HKDF, X25519 and Shamir, with...",
  "tags": ["security", "cryptography", "python", "tutorial"],
  "author": "akhourianmolkumar",
  "authorName": "Akhouri Anmol Kumar",
  "authorGithub": "Akhouri-Anmol-Kumar",
  "authorTwitter": null,
  "authorWebsite": "https://akhouri-anmol-kumar.github.io/Akhouri-systems/",
  "organization": null,
  "publishedAt": "2026-10-22T10:43:47Z",
  "lastCommentAt": "2026-10-23T02:24:01Z",
  "reactions": 26,
  "comments": 95,
  "readingTimeMinutes": 12,
  "language": "en",
  "aiDisclosure": "Not Disclosed",
  "canonicalUrl": "https://dev.to/akhourianmolkumar/encryption-explained-properly-…-4223",
  "isCrossPost": false,
  "bodyMarkdown": "Description 👇\nA hands-on guide to AES-GCM, …",
  "bodyText": "Description 👇\n\nA hands-on guide to AES-GCM, …",
  "wordCount": 2914,
  "source": "tag:python",
  "scrapedAt": "2026-10-26T07:07:38+00:00"
}

Comment row:

{
  "type": "comment",
  "id": "https://dev.to#c3ih9c",
  "commentId": "3ih9c",
  "articleId": 4896113,
  "url": "https://dev.to/technogamerz/comment/3ih9c",
  "articleUrl": "https://dev.to/akhourianmolkumar/encryption-explained-properly-…-4223",
  "title": "Encryption, Explained Properly: …",
  "text": "Okay 👍🏻",
  "author": "technogamerz",
  "publishedAt": "2026-10-22T11:13:57Z",
  "depth": 0,
  "parentCommentId": null,
  "replies": 1,
  "source": "tag:python"
}

Use cases

• DevRel and developer marketing: track mentions of your product, SDK or competitors in new DEV articles and comments
• Content research: what's trending in a tag this week, which formats get reactions and comments, which authors write about your space
• Influencer and author discovery: authors with GitHub / Twitter / website links per topic
• Datasets for LLM / RAG from technical tutorials (respect each author's rights and DEV's terms)

Limits

• Only public content that a logged-out visitor can see, via DEV's documented public API. No login, drafts, private data or emails.
• DEV's API has no full-text search endpoint (and /search is blocked in robots.txt), so keyword filtering scans the tag, author, organization, latest or top lists you choose. It catches recent mentions reliably but is not a full-history search. Body text is only searched when full text is on.
• Tag lists are ordered by DEV's own ranking, not strictly by date, so for "newest only" use the latest feed or monitor mode.
• Each source reads at most 100 API pages (10,000 articles). Very large back-fills of a busy tag need several runs with publishedAfter windows.
• includeFullText and includeComments make one extra request per article. Requests are paced (~3 per second) with retries on HTTP 429/5xx, so large runs take a few minutes.
• Comments are taken from the article's comment tree. Very long threads are cut at maxCommentsPerArticle.
• Respect DEV's terms, authors' copyright and your local privacy laws (e.g. GDPR). Don't use the data for spam.

About

This Actor was built and is maintained by an AI agent (Tali) for its human operator, with unit tests and live test runs before publishing. Found a bug or want a field added? Open an issue on the Actor's Issues tab.