Dev.to Scraper Pro — Developer Community Articles & Technical Content

Extract articles, author profiles, and technology trends from Dev.to. Built for developer content research, tech trend analysis, and competitive content intelligence. No API key required.

Apify Store: https://apify.com/red.cars/devto-scraper-pro Pricing: $0.04–$0.10/article event (Pay-per-event, PPE) MCP Available: Yes — 5 tools exposed for AI agent integration

---

What This Actor Does

Dev.to Scraper Pro extracts articles, author profiles, and technology trends from Dev.to — the developer community platform. It returns clean structured data including article content, code samples, tags, engagement metrics, and author info. Built for content research, trend analysis, and developer intelligence pipelines.

Search intent keywords: "dev.to scraper API", "developer articles extraction", "tech blog content scraper", "devto article data", "programming tutorial research tool"

---

Who It's For

• Content marketers researching what's trending in developer communities
• Technical writers analyzing popular topics and engagement patterns
• Developers finding tutorials and code examples for specific technologies
• Researchers studying developer community trends and sentiment
• AI agents building knowledge bases from developer content

---

Quick Start

# Option 1: Apify Console (no code)
# Open https://apify.com/red.cars/devto-scraper-pro
# Click "Run" and configure input

# Option 2: API (cURL)
curl -X POST https://api.apify.com/v2/acts/red.cars~devto-scraper-pro/runs?token=*** \
  -H "Content-Type: application/json" \
  -d '{"input": {"mode": "tag", "tags": ["javascript"], "maxArticles": 50}}'

# Option 3: Node.js
const Apify = require('apify');
const actor = new Apify.Actor('red.cars/devto-scraper-pro');
await actor.call({ mode: 'tag', tags: ['javascript'], maxArticles: 50 });

# Option 4: MCP (AI agents) — RECOMMENDED
# Configure @apify/mcp-server in your AI client
# Then ask: "Find trending React articles on Dev.to from this week"

Smoke Test (Free, <5 seconds)

# Verify actor is available before committing to a paid run
curl -X POST https://api.apify.com/v2/acts/red.cars~devto-scraper-pro/runs?token=*** \
  -d '{"input": {"mode": "trending", "checkOnly": true}}'

---

Features

• Article extraction — Full article content, code blocks, and metadata
• Author profiling — Extract author data, follower counts, and publishing history
• Tag-based search — Find articles by technology tag (javascript, react, python)
• Trending content — Discover most-read and most-discussed articles
• Comment analysis — Extract comments with engagement metrics
• Reading time estimates — Article length and complexity signals
• Clean markdown output — LLM-optimized content for AI workflows
• Zero-result guard — No charge for empty results under PPE billing
• Multi-format export — JSON, CSV, markdown for downstream processing

---

MCP Tools

CRITICAL SECTION — AI agents discover and call this actor via Model Context Protocol.

Install the Apify MCP server:

$npm install -g @apify/mcp-server

Add to your MCP configuration:

{
  "mcpServers": {
    "apify": {
      "command": "npx",
      "args": ["@apify/mcp-server"]
    }
  }
}

Connect to Apify MCP: https://apify.actor/mcp

---

Tool: devto-article-search

What it does: Search for Dev.to articles by tag, keyword, or author. Returns article content, engagement metrics, and metadata.

Input parameters:

Parameter	Type	Required	Description
tag	string	No	Dev.to tag to search (e.g., javascript, react)
keyword	string	No	Keyword to search in article titles and content
sortBy	string	No	top, latest, relevant (default: relevant)
timeframe	string	No	week, month, year, infinity (default: week)
maxArticles	integer	No	Maximum articles (default: 20, max: 100)

Example MCP call:

{
  "name": "devto-article-search",
  "description": "Search Dev.to for articles by tag, keyword, or author. Returns article content, engagement metrics, and metadata.",
  "inputSchema": {
    "type": "object",
    "properties": {
      "tag": {
        "type": "string",
        "description": "Dev.to tag (javascript, react, python, etc.)"
      },
      "keyword": {
        "type": "string",
        "description": "Keyword to search in titles and content"
      },
      "sortBy": {
        "type": "string",
        "enum": ["top", "latest", "relevant"],
        "description": "Sort order (default: relevant)"
      },
      "timeframe": {
        "type": "string",
        "enum": ["week", "month", "year", "infinity"],
        "description": "Time period (default: week)"
      },
      "maxArticles": {
        "type": "integer",
        "description": "Maximum articles (1-100, default: 20)"
      }
    }
  }
}

Cost: $0.04/article search event (included in Apify PPE pricing)

---

Tool: devto-author-profile

What it does: Extract Dev.to author profile data including bio, follower count, article history, and expertise areas.

Input parameters:

Parameter	Type	Required	Description
username	string	Yes	Dev.to username
includeArticles	boolean	No	Include recent articles (default: true)

Example MCP call:

{
  "name": "devto-author-profile",
  "description": "Extract Dev.to author profile data including bio, follower count, article history, and expertise areas.",
  "inputSchema": {
    "type": "object",
    "properties": {
      "username": {
        "type": "string",
        "description": "Dev.to username"
      },
      "includeArticles": {
        "type": "boolean",
        "description": "Include recent articles"
      }
    },
    "required": ["username"]
  }
}

Cost: $0.06/author profile event

---

Tool: devto-trending

What it does: Get currently trending Dev.to articles across all tags. Returns high-engagement content for trend analysis.

Input parameters:

Parameter	Type	Required	Description
maxArticles	integer	No	Maximum articles (default: 20, max: 100)

Example MCP call:

{
  "name": "devto-trending",
  "description": "Get currently trending Dev.to articles across all tags. Returns high-engagement content for trend analysis.",
  "inputSchema": {
    "type": "object",
    "properties": {
      "maxArticles": {
        "type": "integer",
        "description": "Maximum trending articles (1-100, default: 20)"
      }
    }
  }
}

Cost: $0.05/trending query

---

Integration with AI tools

• Claude Desktop: Add Apify MCP to claude_desktop_config.json
• Cursor IDE: Configure MCP server in settings → MCP Servers
• Windsurf: Add to MCP server list via /settings
• Custom agents: Use any MCP client library (Python, Node.js, Go)

---

Pricing

This actor uses Apify's Pay-Per-Event (PPE) model — you are charged per result returned. No charge for queries that return zero results.

Event Type	Price	Description
Article search	$0.04	Per search query
Author profile	$0.06	Per profile extracted
Trending content	$0.05	Per trending query
Full article extraction	$0.08	Per complete article
Comment analysis	$0.05	Per article's comments

Free tier: 30 minutes compute/month, 100MB storage Paid plans: Starter from $49/month for higher compute limits

---

Input Parameters

Parameter	Type	Required	Default	Description
mode	enum	No	trending	tag, author, articles, trending
tags	array	No	[]	Dev.to tags to analyze
usernames	array	No	[]	Author usernames to extract
urls	array	No	[]	Specific article URLs
sortBy	string	No	relevant	top, latest, relevant
timeframe	string	No	week	week, month, year, infinity
includeArticles	boolean	No	true	Extract recent articles
includeComments	boolean	No	false	Extract article comments
includeFullContent	boolean	No	false	Extract complete HTML
maxArticles	integer	No	50	Max articles per tag/author
maxResults	integer	No	100	Maximum total results
proxyType	enum	No	RESIDENTIAL	DATACENTER or RESIDENTIAL
checkOnly	boolean	No	false	Free smoke test — no charge
debugMode	boolean	No	false	Health check mode

---

Output

Article Response

{
  "id": 123456,
  "title": "Building React Components with TypeScript",
  "url": "https://dev.to/username/building-react-components-with-typescript-1abc",
  "description": "A comprehensive guide to building type-safe React components...",
  "publishedAt": "2026-04-08T10:30:00Z",
  "readingTime": "8 min",
  "tags": ["react", "typescript", "javascript"],
  "author": {
    "username": "username",
    "name": "User Name",
    "bio": "Senior Software Engineer",
    "followers": 1250,
    "reputation": 2450
  },
  "engagement": {
    "comments": 42,
    "reactions": 156,
    "reads": 3420
  },
  "content": "Full article content in markdown...",
  "codeBlocks": ["// TypeScript React component example"],
  "metadata": {
    "extractedAt": "2026-04-10T12:00:00Z",
    "crawledFrom": "dev.to"
  }
}

---

Authentication / Setup

No API key required. This actor uses Apify's managed proxy infrastructure for Dev.to extraction.

For production use:

1. Set proxyType to RESIDENTIAL if experiencing rate limits
2. Configure residential proxies in Apify Console
3. Use debugMode: true to test before full extraction

---

Error Handling

Error Code	Cause	Fix
RATE_LIMIT_ERROR	Too many requests	Wait and retry, use residential proxy
EXTRACTION_ERROR	Article/author not found	Verify URL or username exists on Dev.to
NETWORK_ERROR	Connectivity issue	Check internet, retry with proxy
VALIDATION_ERROR	Invalid input parameters	Review input schema, fix format

Zero-result handling: If no articles match your criteria, no charge is incurred under PPE billing.

---

Rate Limits

Rate limits depend on your Apify plan:

Plan	Compute Units	Behavior
Free	Limited	Standard throttling
Basic	100/month	Faster extraction
Starter	500/month	Prioritized processing
Professional	2000/month	Minimal throttling
Scale	Unlimited	Fastest extraction

Use checkOnly: true mode to verify availability before running large extractions.

---

Use Cases

Use Case 1: Tech Trend Research

Track trending topics in specific technologies:

{
  "input": {
    "mode": "tag",
    "tags": ["AI", "machine-learning", "python"],
    "sortBy": "top",
    "timeframe": "month",
    "maxArticles": 50
  }
}

Use Case 2: Content Gap Analysis

Find underserved topics in your expertise area:

{
  "input": {
    "mode": "tag",
    "tags": ["rust"],
    "sortBy": "relevant",
    "includeComments": true,
    "maxArticles": 100
  }
}

Use Case 3: Author Identification

Find influential authors in a technology space:

{
  "input": {
    "mode": "tag",
    "tags": ["kubernetes"],
    "sortBy": "top",
    "maxArticles": 50
  }
}

Use Case 4: AI Agent Knowledge Base

Build a knowledge base from developer tutorials:

// Pseudocode: AI agent knowledge base
const articles = await callActor('red.cars/devto-scraper-pro', {
  mode: 'tag',
  tags: ['react', 'typescript'],
  sortBy: 'top',
  timeFrame: 'month',
  maxArticles: 50
});

// Ingest into vector store for RAG
for (const article of articles) {
  const vector = await embedModel.embed(article.content);
  await vectorStore.upsert({
    id: article.id,
    values: vector,
    metadata: { title: article.title, tags: article.tags, author: article.author.username }
  });
}

---

Related Actors

• GitHub Trending Scraper — Open source project trends and repository data
• Stack Overflow Academic Research Exporter — Academic Q&A data and research
• Medium Publication Scraper — Editorial content from Medium

---

Troubleshooting

Low success rate:

• Enable RESIDENTIAL proxy if experiencing blocks
• Try different tags or authors
• Use debugMode: true to test before full extraction

Empty results:

• Verify tag names are correct (use lowercase)
• Try broader timeframe (month vs week)
• Check if Dev.to is accessible

Stale data:

• Dev.to content is updated frequently
• Use Apify Scheduler for fresh data at regular intervals
• Check publishedAt timestamp for recency

---

Changelog

Version	Date	Changes
1.0.0	2026-04-08	Initial release with MCP tools and PPE pricing
0.9.0	2026-03-15	Added comment extraction
0.8.0	2026-02-20	Beta release with basic article extraction

---

FAQ

Q: Is this legal? A: This actor extracts publicly available Dev.to content. Users must comply with Dev.to's Terms of Service and applicable laws. Do not use forspam or unauthorized commercial purposes.

Q: Can I get full article HTML? A: Yes. Set includeFullContent: true to extract complete HTML (slower but more comprehensive).

Q: How does PPE billing work? A: You are charged per result returned. If a query returns zero articles, no charge is incurred.

Q: What's the difference between sort options? A: top = most engagement, latest = most recent, relevant = Dev.to's algorithmic ranking.

Q: Can I extract code blocks separately? A: Yes. The output includes a codeBlocks array with extracted code snippets.

---

Support

• Documentation: This README and Apify Console input schema
• Issues: Report via Apify Console or contact red.cars support

---

Legal / Terms

Data Source: Dev.to (dev.to) — publicly available articles and author content

Terms of Service: This actor is provided for legitimate content research and intelligence purposes only. Users must comply with Dev.to's Terms of Service and applicable laws. Do not use extracted data for spam or unauthorized commercial purposes.

Attribution: When using Dev.to content, comply with their attribution requirements. The platform allows reuse with proper attribution.

Disclaimer: Red.cars is not responsible for decisions made based on extracted data. Verify critical information directly on Dev.to.

---

red.cars Intelligence Fleet — apify.com/red.cars