DEV.to Scraper — Extract Articles, Metadata & Engagement Data

Scrape articles from DEV.to tag pages, user profiles, and trending feeds. Extract titles, authors, publication dates, tags, reactions, comments, and estimated reading time in structured JSON format.

Use Cases

• Content Research — Monitor trending topics across developer communities by scraping tag pages like webdev, javascript, or python.
• Influencer Analysis — Collect all articles from specific DEV.to authors to analyze posting frequency, engagement trends, and content themes.
• Trend Tracking — Build datasets of top-performing posts to identify what resonates with the developer audience.
• Competitive Monitoring — Track how often competitors or partners publish on DEV.to and measure their community engagement.

Input

Field	Type	Required	Description
startUrls	Array	Yes	DEV.to URLs to scrape. Supports tag pages (/t/webdev), user profiles (/username), and trending (/top/week).
maxItems	Number	No	Maximum articles to extract (default: 100, max: 10,000).
proxyConfiguration	Object	No	Proxy settings. Uses Apify Proxy by default.

Output

The Actor outputs a dataset with the following fields:

{
  "title": "How to Build a Scraper with Crawlee",
  "url": "https://dev.to/example/build-scraper-crawlee",
  "author": "JaneDev",
  "authorUrl": "https://dev.to/janedev",
  "publishedAt": "2026-04-19T08:30:00Z",
  "tags": ["javascript", "webscraping", "tutorial"],
  "reactionsCount": 42,
  "commentsCount": 7,
  "readingTimeMinutes": 5
}

Pricing

Pay per event: $0.002 per article scraped.

No monthly subscription. You only pay for the articles extracted. Example: scraping 500 articles costs $1.00.

Limitations

• DEV.to pages are server-side rendered (SSR) HTML. No JavaScript rendering required.
• Article engagement counts (reactions, comments) reflect the state at scrape time and may change.
• Private or draft articles are not accessible.
• Rate limiting may apply for very large crawls; the Actor uses polite delays and retries.

FAQ

Q: Can I scrape a specific user's entire article history? A: Yes. Use their profile URL (e.g., https://dev.to/username) as a start URL. The Actor follows pagination automatically.

Q: Does this work with DEV.to organizations? A: Yes. Organization profile URLs follow the same structure as user profiles.

Q: Is proxy required? A: Proxy is included by default via Apify Proxy. You can configure a custom proxy if needed.

Changelog

• v1.0.0 — Initial release. Supports tag pages, user profiles, and trending feeds with full metadata extraction.