# Dev.to Scraper: Articles, Comments, Users & Tags (`perconey/devto-scraper`) Actor Scrape dev.to (Forem) via the official public REST API. Articles by tag/user/latest/top, comments, user profiles, tags, podcasts, videos, listings. No browser, no proxies, no auth. Pay only per result item. - **URL**: https://apify.com/perconey/devto-scraper.md - **Developed by:** [Perconey](https://apify.com/perconey) (community) - **Categories:** Developer tools, Social media, News - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing $1.00 / 1,000 result items This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ### What does Dev.to Scraper do? **Dev.to Scraper** pulls structured data from [dev.to](https://dev.to) via the **official Forem REST API**. Articles (with full markdown bodies), nested comment threads, user profiles with badges and follower counts, the global tag taxonomy, video posts, podcast episodes, and the classifieds (jobs, courses, events). The actor calls the documented public API directly - no browser, no proxies, no cookies, no anti-bot fight. dev.to is one of the largest developer-focused publishing platforms; this actor exposes its content as clean JSON. Try it instantly: pick **getArticles**, leave everything default, click Start. You get the 30 newest articles on dev.to (title, description, tags, reactions, comment counts, author) in under 3 seconds for $0.03. ### Why use Dev.to Scraper? - **Content marketers**: Track trending topics in your tech niche. Schedule daily `getArticles` with a tag filter (e.g. `tag=rust`) to know what's resonating with developers right now. - **DevRel teams**: Monitor mentions of your product across dev.to. Pull every article tagged with your stack and use comments as a free customer-feedback channel. - **Recruiters**: `getUserProfile` returns a developer's badges, GitHub username, location, and post count. Combined with `getUserArticles` you can quickly judge depth of expertise. - **Trend analysts**: `getTags` gives you the global taxonomy with article counts. Compute month-over-month changes to see which technologies are climbing. - **Conference / podcast scouts**: `getPodcastEpisodes` and `getVideos` surface developer-creator content beyond text articles. - **Job-board aggregators**: `getListings` returns the classifieds (job ads, course launches, events) for downstream filtering. ### How to use Dev.to Scraper 1. Open the **Input** tab. 2. Pick an **action** from the dropdown. `getArticles` is the simplest starting point. 3. (Optional) for `getArticles` set a **tag** filter (e.g. `rust`, `webdev`, `javascript`) or a **topDays** value (e.g. 7 for top of the week). 4. For article-detail / article-comments / user actions, paste an article id or username in **queries**. 5. Tune **maxItems** (default 30). 6. Click **Start**. #### Query format by action Action | Query format --- | --- getArticles | leave empty (use tag / topDays / state filters) getArticleDetail | numeric article id (e.g. `1234567`) getArticleComments | numeric article id getUserProfile | username (e.g. `ben`) getUserArticles | username getTags | leave empty getVideos | leave empty getPodcastEpisodes | leave empty getListings | leave empty ### Input Field | Required | Description --- | --- | --- `action` | yes | Which API call to make. Nine options. `queries` | sometimes | Required for article-detail / comments / user actions. Empty otherwise. `maxItems` | no | Max items per query. Default 30. `tag` | no | getArticles only. Tag name to filter (e.g. `rust`). `topDays` | no | getArticles only. Top articles from last N days (e.g. 7). `state` | no | getArticles only. `all` (default), `fresh`, or `rising`. `apiKey` | no | Forem API key. Not required - all 9 read endpoints work anonymously today. ### Output Every item carries `_type` (`article` / `comment` / `user` / `tag` / `video` / `podcast_episode` / `listing` / `error`) plus `_action` for filtering. ```json { "_type": "article", "_action": "getArticles", "id": 9876543, "title": "Why Rust 1.85 Is My Favorite Release", "description": "The 2024 edition went stable. Here is what changed in practice.", "url": "https://dev.to/rustacean/why-rust-1-85-is-my-favorite-release-1abc", "tag_list": ["rust", "webdev", "programming"], "published_at": "2026-05-12T10:30:00Z", "reading_time_minutes": 6, "public_reactions_count": 542, "comments_count": 89, "cover_image": "https://...", "user": { "name": "Rustacean", "username": "rustacean", "github_username": "rustacean", "profile_image": "https://..." }, "organization": null } ```` You can download the dataset in JSON, CSV, XML, Excel, RSS or HTML format from the Output tab. ### Data fields Type | Key fields \--- | --- `article` | id, title, description, body\_markdown (detail only), body\_html (detail only), url, slug, canonical\_url, cover\_image, tag\_list, published\_at, reading\_time\_minutes, public\_reactions\_count, comments\_count, user, organization `comment` | id\_code, body\_html, created\_at, user, children (count) `user` | id, username, name, summary, twitter\_username, github\_username, website\_url, location, joined\_at, profile\_image, badge\_ids, followers\_count, following\_count, post\_count `tag` | id, name, bg\_color\_hex, text\_color\_hex `video` | inherits article shape + video\_url, video\_source\_url, video\_duration\_in\_minutes `podcast_episode` | id, title, slug, path, image\_url, podcast (channel info) `listing` | id, title, slug, body\_markdown, tag\_list, category, processed\_html, author, organization ### Pricing **Pay-per-result: $0.001 per item.** No flat monthly fee. Cost examples: - Daily newest 30 articles: **$0.03** - 1,000 articles tagged `javascript` for content research: **$1.00** - 100 user profiles + their recent articles (~3000 items): **$3.00** - All 500 active tags: **$0.50** ### Tips - **getArticles + topDays = trend dashboard.** Set `topDays=7` once a week and you get the top of the week reliably. The dev.to feed defaults to chronological - you have to opt into top sorting. - **getArticles vs getArticleDetail.** Feed endpoints return article metadata WITHOUT body\_markdown (lighter payload). Use getArticleDetail to fetch the full content when you actually need it. - **Comments are flat-list with parent\_id** in the response. The actor preserves Forem's depth-first traversal order in the dataset. - **Articles can be syndicated.** Watch `canonical_url` - if it differs from `url`, the author cross-posted the content from elsewhere (their personal blog, Medium, etc). - **Forem powers more than dev.to**: forem.dev and several enterprise instances run the same software. This actor is hardcoded to dev.to today but the API surface is identical. ### FAQ, disclaimers, support **Is this legal?** The actor calls dev.to's official public REST API with documented endpoints. Public read access is the design intent of the open-source Forem software. We send a clear User-Agent identifying the actor and honor rate-limit / Retry-After headers. **Does it need an API key?** No. As of 2026-05, all read endpoints we wrap (articles, comments, users, tags, videos, podcasts, listings) work anonymously. The `apiKey` input is there as a forward-compatible knob in case Forem tightens limits. **Will I get rate-limited?** dev.to's per-IP rate limits are generous for read-only traffic and the actor retries on 429 with exponential backoff. For very heavy scraping consider supplying your own API key (10K req/day). **Why is comment threading flat?** The Forem comments API v1 returns a tree, but it does not expose parent\_id directly on each comment. The actor depth-first flattens the tree preserving order; if you need exact parent pointers, file an issue and we will add parent\_code. **Bug or feature request?** Open an Issue on the actor's Issues tab. I usually respond within a day. **Need a scraper for Hacker News, Stack Overflow, Mastodon, Lemmy?** See my other actors at https://apify.com/perconey, or open an Issue for the platform you need. # Actor input Schema ## `action` (type: `string`): Article-list actions need no queries. getArticleDetail / getArticleComments / getUserArticles need an article id or username. getUserProfile takes a username. ## `queries` (type: `array`): Depends on action. Article-list / tags / videos / podcasts / listings: leave empty (use tag + top filters below if needed). getArticleDetail / getArticleComments: article id (e.g. 123456). getUserProfile / getUserArticles: username (e.g. ben). ## `maxItems` (type: `integer`): Stop after this many items (dev.to API caps per\_page at 1000 - we paginate). ## `tag` (type: `string`): Optional. Used by getArticles only (e.g. 'javascript', 'rust', 'webdev'). ## `topDays` (type: `integer`): Optional. getArticles only. If set, returns highest-reaction articles from last N days (e.g. 7 = top of week). Mutually exclusive with state. ## `state` (type: `string`): Optional. getArticles only. 'fresh' = recently posted, 'rising' = climbing in reactions, 'all' = default chronological. ## `apiKey` (type: `string`): Personal dev.to API key. Anonymous reads work for all 9 actions today; supply a key only if dev.to tightens limits in future. https://dev.to/settings/extensions ## Actor input object example ```json { "action": "getArticles", "queries": [], "maxItems": 30, "state": "all" } ``` # Actor output Schema ## `dataset` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "queries": [] }; // Run the Actor and wait for it to finish const run = await client.actor("perconey/devto-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "queries": [] } # Run the Actor and wait for it to finish run = client.actor("perconey/devto-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "queries": [] }' | apify call perconey/devto-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=perconey/devto-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Dev.to Scraper: Articles, Comments, Users & Tags", "description": "Scrape dev.to (Forem) via the official public REST API. Articles by tag/user/latest/top, comments, user profiles, tags, podcasts, videos, listings. No browser, no proxies, no auth. Pay only per result item.", "version": "0.1", "x-build-id": "oDdZAyV4AOF601Dqh" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/perconey~devto-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-perconey-devto-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/perconey~devto-scraper/runs": { "post": { "operationId": "runs-sync-perconey-devto-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/perconey~devto-scraper/run-sync": { "post": { "operationId": "run-sync-perconey-devto-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "action" ], "properties": { "action": { "title": "What do you want to scrape?", "enum": [ "getArticles", "getArticleDetail", "getArticleComments", "getUserProfile", "getUserArticles", "getTags", "getVideos", "getPodcastEpisodes", "getListings" ], "type": "string", "description": "Article-list actions need no queries. getArticleDetail / getArticleComments / getUserArticles need an article id or username. getUserProfile takes a username.", "default": "getArticles" }, "queries": { "title": "Queries", "type": "array", "description": "Depends on action. Article-list / tags / videos / podcasts / listings: leave empty (use tag + top filters below if needed). getArticleDetail / getArticleComments: article id (e.g. 123456). getUserProfile / getUserArticles: username (e.g. ben).", "items": { "type": "string" } }, "maxItems": { "title": "Max items per query", "minimum": 0, "maximum": 100000, "type": "integer", "description": "Stop after this many items (dev.to API caps per_page at 1000 - we paginate).", "default": 30 }, "tag": { "title": "Filter articles by tag", "type": "string", "description": "Optional. Used by getArticles only (e.g. 'javascript', 'rust', 'webdev')." }, "topDays": { "title": "Top articles from last N days", "minimum": 1, "maximum": 365, "type": "integer", "description": "Optional. getArticles only. If set, returns highest-reaction articles from last N days (e.g. 7 = top of week). Mutually exclusive with state." }, "state": { "title": "Article state filter", "enum": [ "all", "fresh", "rising" ], "type": "string", "description": "Optional. getArticles only. 'fresh' = recently posted, 'rising' = climbing in reactions, 'all' = default chronological.", "default": "all" }, "apiKey": { "title": "Forem API key (optional)", "type": "string", "description": "Personal dev.to API key. Anonymous reads work for all 9 actions today; supply a key only if dev.to tightens limits in future. https://dev.to/settings/extensions" } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```