DEV.to Scraper ⚡

Fast and reliable DEV.to article scraper that uses the official DEV.to API. Extract articles, user information, tags, reactions, and comments without browser overhead.

Why This Scraper?

• ⚡ Super Fast - Uses official API, no browser needed
• 💰 Cost Efficient - HTTP requests only, minimal compute units
• 🎯 Reliable - Official API means no breakage from UI changes
• 🔍 Flexible Filters - Search by username, tag, keywords, or get top articles
• 📊 Rich Data - Get reactions, comments, user profiles, tags, and more

Features

✅ Scrape articles by username (e.g., all posts from @ben)
✅ Filter by tag (e.g., #javascript, #python, #webdev)
✅ Search within articles (title, description, tags)
✅ Get top articles by day/week/month/year
✅ Extract comments (optional)
✅ Get user profiles with social links
✅ Fetch organization information
✅ Sort by fresh or rising articles

Use Cases

• 📈 Content Research - Find trending topics and popular articles
• 🎯 Competitor Analysis - Track what successful developers write about
• 📚 Content Curation - Build reading lists or newsletters
• 🤖 AI Training Data - Collect developer content for LLMs
• 📊 Analytics - Track engagement metrics (reactions, comments)
• 🔔 Monitoring - Watch for articles with specific keywords/tags

Input Configuration

Basic Filters

• Username - Get articles from a specific user (e.g., ben, jess)
• Tag - Filter by tag (e.g., javascript, webdev, tutorial)
• Search Query - Search within titles, descriptions, and tags
• Max Articles - Limit results (default: 100)

Advanced Options

• Top N - Get top articles (7=week, 30=month, 365=year, 999=all-time)
• State - Choose fresh (recent), rising (trending), or all
• Include Comments - Fetch comments for each article (increases runtime)

Output Data

Each article includes:

{
  "id": 3265532,
  "title": "A New Chapter: DEV is Joining Forces with MLH",
  "description": "Hey everyone, I have some massive news...",
  "url": "https://dev.to/devteam/...",
  "publishedAt": "2026-02-18T14:10:31Z",
  "commentsCount": 86,
  "publicReactionsCount": 343,
  "readingTimeMinutes": 6,
  "tags": ["news", "devto", "hackathon", "discuss"],
  "coverImage": "https://...",
  "user": {
    "name": "Ben Halpern",
    "username": "ben",
    "twitterUsername": "bendhalpern",
    "githubUsername": "benhalpern",
    "websiteUrl": "http://benhalpern.com",
    "profileImage": "https://..."
  },
  "organization": {
    "name": "The DEV Team",
    "username": "devteam"
  }
}

Examples

Get Recent JavaScript Articles

{
  "tag": "javascript",
  "maxArticles": 50
}

Get All Posts from a User

{
  "username": "ben",
  "maxArticles": 200
}

Find Articles About React Hooks

{
  "searchQuery": "react hooks",
  "maxArticles": 100
}

Get Top Weekly Articles

{
  "top": 7,
  "maxArticles": 50
}

Get Trending Python Articles with Comments

{
  "tag": "python",
  "state": "rising",
  "includeComments": true,
  "maxArticles": 30
}

API Limits & Best Practices

• The DEV.to API is rate-limited but generous for reasonable use
• This scraper includes automatic 500ms delays between pages
• Fetching comments increases runtime - use only when needed
• Start with small maxArticles values to test your filters

Cost

This actor is extremely cheap to run because it uses simple HTTP requests instead of browsers:

• ~0.001 compute units per 100 articles (without comments)
• ~0.01 compute units per 100 articles (with comments)

Support

• 📖 DEV.to API Documentation
• 💬 Questions? Open an issue or contact the developer
• ⭐ Like this actor? Leave a review!

License

Apache-2.0