DEV.to Articles Scraper

---

🎯 What this scrapes

DEV.to publishes a public REST API (v1, https://developers.forem.com/api/v1) for articles. This Actor wraps it with sensible inputs — scrape by tag, author, or the global latest feed — and writes one row per article with body in raw Markdown, ready to ingest into a RAG store or content pipeline.

🔥 What we handle for you

• 🛡️ Browser fingerprint rotation — curl-cffi impersonates real Chrome / Firefox / Safari TLS handshakes so the target sees a browser, not Python.
• 🌐 Residential proxy rotation via Apify Proxy — fresh session and exit IP on every block.
• 🔁 Retries with exponential backoff on 408 / 429 / 5xx — up to 5 attempts per page, Retry-After honoured.
• 🧱 Rate-limit-aware pacing — when the target pushes back, we slow down instead of getting banned.
• 🧊 Clean, typed dataset rows — Pydantic-validated, ISO-8601 timestamps, stable IDs, JSON / CSV / Excel export straight from the Apify Console.
• 💰 Pay-Per-Event pricing — you only pay for results that hit your dataset. No data, no charge.

💡 Use cases

• RAG corpus seeding — pull all articles for tag=ai to seed a retrieval-augmented assistant.
• Trending tag dashboards — daily diff on top to watch which posts gain traction.
• Author monitoring — schedule per-author runs to mirror a writer's catalogue.
• Newsletter assembly — pull top 10 from mode=top, render to Markdown, send.

⚙️ How to use it

1. Click Try for free at the top of the page.
2. Fill in the input form — most fields have sensible defaults.
3. Click Start. Output streams into the run's dataset.
4. Export from Storage → Dataset as JSON, CSV, or Excel — or fetch via the API.

📥 Input

Field	Type	Required	Default	Notes
mode	string	no	'tag'	Which DEV.to feed to read.
tag	string	no	'python'	DEV.to tag, lowercase, no #. Examples: python, webdev, typescript.
username	string	no	'—'	DEV.to username (no @).
includeBody	boolean	no	True	When true, fetch the article body via a per-article request. One extra API call per article. Without it you still get al
maxResults	integer	no	30	Max articles to return across pages.
concurrency	integer	no	4	Parallel article-body fetches.
proxyConfiguration	object	no	{'useApifyProxy': False}	DEV.to is OK with direct API calls. Proxy optional.

Example input

{
  "mode": "tag",
  "tag": "python",
  "includeBody": false,
  "maxResults": 3,
  "concurrency": 4,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

📤 Output

Every row is one dataset item.

Field	Type	Notes
id	integer	DEV.to article ID.
slug	string	URL slug.
title	string	Article title.
description	['string', 'null']	Short description (first sentence-ish).
url	string	Canonical article URL.
cover_image	['string', 'null']	Cover image URL.
tags	array	Article tags.
author_username	string	Author DEV.to username.
author_name	string	Author display name.
reading_time_minutes	['integer', 'null']	Estimated reading time.
positive_reactions_count	integer	Sum of positive reactions.
comments_count	integer	Comment count.
body_markdown	['string', 'null']	Article body in Markdown (when includeBody=true).
published_at	string	Publish timestamp (ISO-8601).
edited_at	['string', 'null']	Last-edited timestamp.
scraped_at	string	When this row was recorded.

Example output

{
  "id": 1234567,
  "slug": "fast-python-async-tricks-1abc",
  "title": "Fast Python async tricks every dev should know",
  "url": "https://dev.to/me/fast-python-async-tricks-1abc",
  "author_username": "me",
  "tags": [
    "python",
    "async",
    "webdev"
  ],
  "positive_reactions_count": 142,
  "comments_count": 8
}

💰 Pricing

Pay-Per-Event — you pay only when these events fire:

Event	USD	What it is
actor-start	$0.005	One-off warm-up charge per run
result	$0.002	Per dataset item

Example: 1 000 results at the rates above ≈ $2.00. No subscription, no minimum, no card to start — Apify gives every new account $5 of free credit.

🚧 Limitations

API only — we don't render images or extract code blocks separately. If DEV.to's API returns a cached/older version, that's what you get.

❓ FAQ

Do I need an API key?

Not for read access. DEV.to's v1 API is wide-open for GET /articles.

Why are some bodies missing?

Some posts redirect to a canonical URL hosted elsewhere — DEV.to's API returns the metadata but the body lives off-platform.

What about comments?

Out of scope here — fetching them per article would multiply cost. Open an issue if you need it.

Can I filter by reading time?

Not yet — post-process the output. Hard input filtering on the DEV.to API is limited.

💬 Your feedback

Spotted a bug, hit a weird edge case, or need a new field? Open an issue on the Actor's Issues tab on Apify Console — we ship fixes weekly and we read every report.

---