Dev.to Articles Scraper

Extract articles from Dev.to by tag, username, or sort order — no HTML parsing, no browser overhead, just clean JSON from the free Dev.to REST API. Filter by reactions, paginate up to 100 results, and push structured data straight to an Apify dataset. First 10 results per run are free; pay-per-event pricing applies thereafter.

Dev.to (also known as The DEV Community) is a massive platform where software developers share articles, tutorials, and discussions. With this Actor, you can programmatically search, filter, and collect that content for analysis, curation, or integration into your own tools.

Why use Dev.to Articles Scraper?

• Content research — Discover trending articles in your tech stack (Python, Rust, AI, webdev, etc.) and track what topics are resonating with the community
• Community monitoring — Follow a specific author's output, or monitor a tag for new content as it's published
• Data analysis — Gather reaction counts, comment volumes, and posting patterns to analyze engagement trends across the Dev.to ecosystem
• Content curation — Build automated feeds, newsletters, or recommendation engines from filtered Dev.to content
• Integration — Pipe article data into your own dashboards, databases, or AI pipelines via Apify's API and webhook integrations

Because the Actor uses Dev.to's official REST API (no scraping, no browser), it's fast, reliable, and respectful of the platform. You get clean structured data without worrying about HTML changes, selectors, or anti-bot measures.

How to use Dev.to Articles Scraper

1. Install dependencies — Run pip install -r requirements.txt or use the pre-configured Docker image
2. Configure input — Edit test-input.json or use the Apify Console input form. Set a tag (e.g. "python"), choose a sort order (trending, newest, or most_reactions), and pick how many results you want (up to 100)
3. Run locally — apify run --input-file=test-input.json will execute the Actor and save results to storage/datasets/default/
4. Deploy to the cloud — apify push builds and deploys your Actor to Apify Console, where you can schedule it, trigger it via API, or connect it to Zapier/Make

Input

Configure the Actor through the Input tab in Apify Console, or via a JSON file:

{
    "tag": "python",
    "sort": "trending",
    "max_results": 25,
    "min_reactions": 10,
    "username": "",
    "api_key": ""
}

Field	Type	Default	Description
tag	string	"python"	Filter articles by tag (e.g., "ai", "webdev", "javascript", "rust")
username	string	""	Fetch articles by a specific Dev.to username. Overrides tag and sort.
sort	string	"trending"	One of trending (last 30 days), newest, or most_reactions (all time)
max_results	integer	25	Max articles to return (1–100). First 10 free, then $0.002 each.
min_reactions	integer	0	Minimum total reactions an article must have to be included
api_key	string	""	Optional Dev.to API key from dev.to/settings/account for higher rate limits

Output

Each article is pushed as a separate dataset item. You can download the dataset in JSON, HTML, CSV, or Excel from Apify Console.

{
    "title": "Building a REST API with FastAPI and PostgreSQL",
    "url": "https://dev.to/johndoe/building-a-rest-api-with-fastapi-and-postgresql-123",
    "description": "A comprehensive guide to building production-ready APIs using FastAPI...",
    "author": "johndoe",
    "author_image": "https://dev.to/user_images/johndoe/profile.jpg",
    "tags": ["python", "fastapi", "postgresql", "tutorial"],
    "published_at": "2025-06-15T10:30:00Z",
    "reading_time_minutes": 12,
    "reactions_count": 87,
    "comments_count": 14,
    "cover_image": "https://dev.to/social_previews/article/12345.png"
}

Data fields

Field	Type	Description
title	string	Article headline
url	string	Full Dev.to article URL
description	string	Excerpt or intro paragraph
author	string	Dev.to username
author_image	string	Profile image URL
tags	array of strings	Tag list (e.g., ["python", "tutorial"])
published_at	string	ISO 8601 publish date
reading_time_minutes	integer	Estimated reading time
reactions_count	integer	Total reactions (❤️ 🦄 🔖, etc.)
comments_count	integer	Total comments
cover_image	string	Cover image URL (nullable)

Use Cases

Content discovery for tech newsletters

Collect the most-reacted articles in "ai" or "webdev" every week. Schedule the Actor in Apify Console to run once a week, pipe the dataset into a spreadsheet, and curate your newsletter from the results. The min_reactions filter ensures only high-quality articles make the cut.

Author portfolio monitoring

Track a specific author's publishing frequency and engagement. Set username to follow a thought leader or competitor, run daily, and build a time-series dataset of their articles, reactions, and comments.

Trend analysis across tags

Compare engagement metrics across tags. Run the Actor with tag: "python" and sort: "most_reactions", then repeat for "rust", "javascript", and "go". Merge the datasets to see which ecosystem generates the most community buzz.

Automated content pipeline

Trigger the Actor via Apify webhook after a new blog post is published. Feed the resulting dataset into an AI summarization pipeline, a Slack bot, or a personal dashboard — all without writing any scraping infrastructure.

Community health dashboards

Use the Actor to sample Dev.to activity over time. Track how reaction counts, comment volumes, and posting frequency evolve across tags. Pair with Apify's scheduled runs and the Data API to build live dashboards in Grafana, Metabase, or a custom frontend.

Pricing / Cost estimation

How much does it cost to scrape Dev.to articles?

• First 10 results per run are free
• Each additional result costs $0.002 (charged via Actor.charge(event_name='result'))
• The Actor start event costs $0.00005
• Example: scraping 25 articles = 10 free + 15 charged = $0.03 per run
• Example: scraping 100 articles = 10 free + 90 charged = $0.18 per run

The Actor uses the Dev.to public API with no authentication required for basic usage. If you need higher rate limits, add your free Dev.to API key from Account Settings.

Advanced options

• Rate limiting — The Dev.to API has generous rate limits. For production workloads, set an api_key to avoid throttling
• Large datasets — To collect more than 100 articles, run the Actor multiple times with different tags or sort orders and merge the datasets
• Local development — Use apify run --purge to clear previous storage before a fresh test run

FAQ, disclaimers, and support

Is this legal?

Yes. This Actor uses the official Dev.to REST API (dev.to/api/articles). It does not scrape HTML, bypass rate limits, or violate the Dev.to Terms of Service. The API is free and publicly documented at docs.dev.to/api.

Why use an API instead of scraping HTML?

The Dev.to API returns structured JSON directly. No HTML parsing, no fragile CSS selectors, no browser overhead. Requests are 10–100x faster than browser-based scraping, and the data is guaranteed to be correct because it comes from the platform's own data layer.

Known limitations

• Max 100 results per API call (Dev.to API limit)
• The trending sort covers the last 30 days
• The most_reactions sort covers the last 365 days
• No pagination is currently implemented (single-page fetch only)

Support

Found a bug or have a feature request? Open an issue on the GitHub repository or submit feedback via the Apify Console Issues tab. For custom scraping solutions or Enterprise needs, contact the Apify team.

---

Built with Apify SDK for Python and httpx.