# Dev.to Articles Scraper (`ef12/devto-articles-scraper`) Actor Scrape Dev.to articles by tag, username, or trending. Get titles, descriptions, authors, tags, reactions, reading time, and comment counts via the free Dev.to API. - **URL**: https://apify.com/ef12/devto-articles-scraper.md - **Developed by:** [Daniel Wilson](https://apify.com/ef12) (community) - **Categories:** Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Dev.to Articles Scraper **Extract articles from Dev.to by tag, username, or sort order** — no HTML parsing, no browser overhead, just clean JSON from the free [Dev.to REST API](https://docs.dev.to/api/). Filter by reactions, paginate up to 100 results, and push structured data straight to an Apify dataset. First 10 results per run are free; pay-per-event pricing applies thereafter. [Dev.to](https://dev.to) (also known as The DEV Community) is a massive platform where software developers share articles, tutorials, and discussions. With this Actor, you can programmatically search, filter, and collect that content for analysis, curation, or integration into your own tools. ### Why use Dev.to Articles Scraper? - **Content research** — Discover trending articles in your tech stack (Python, Rust, AI, webdev, etc.) and track what topics are resonating with the community - **Community monitoring** — Follow a specific author's output, or monitor a tag for new content as it's published - **Data analysis** — Gather reaction counts, comment volumes, and posting patterns to analyze engagement trends across the Dev.to ecosystem - **Content curation** — Build automated feeds, newsletters, or recommendation engines from filtered Dev.to content - **Integration** — Pipe article data into your own dashboards, databases, or AI pipelines via Apify's API and webhook integrations Because the Actor uses Dev.to's **official REST API** (no scraping, no browser), it's fast, reliable, and respectful of the platform. You get clean structured data without worrying about HTML changes, selectors, or anti-bot measures. ### How to use Dev.to Articles Scraper 1. **Install dependencies** — Run `pip install -r requirements.txt` or use the pre-configured Docker image 2. **Configure input** — Edit `test-input.json` or use the Apify Console input form. Set a tag (e.g. `"python"`), choose a sort order (`trending`, `newest`, or `most_reactions`), and pick how many results you want (up to 100) 3. **Run locally** — `apify run --input-file=test-input.json` will execute the Actor and save results to `storage/datasets/default/` 4. **Deploy to the cloud** — `apify push` builds and deploys your Actor to Apify Console, where you can schedule it, trigger it via API, or connect it to Zapier/Make ### Input Configure the Actor through the **Input** tab in Apify Console, or via a JSON file: ```json { "tag": "python", "sort": "trending", "max_results": 25, "min_reactions": 10, "username": "", "api_key": "" } ```` | Field | Type | Default | Description | | ------- | ------ | --------- | ------------- | | `tag` | string | `"python"` | Filter articles by tag (e.g., "ai", "webdev", "javascript", "rust") | | `username` | string | `""` | Fetch articles by a specific Dev.to username. Overrides tag and sort. | | `sort` | string | `"trending"` | One of `trending` (last 30 days), `newest`, or `most_reactions` (all time) | | `max_results` | integer | `25` | Max articles to return (1–100). First 10 free, then $0.002 each. | | `min_reactions` | integer | `0` | Minimum total reactions an article must have to be included | | `api_key` | string | `""` | Optional Dev.to API key from [dev.to/settings/account](https://dev.to/settings/account) for higher rate limits | ### Output Each article is pushed as a separate dataset item. You can download the dataset in JSON, HTML, CSV, or Excel from Apify Console. ```json { "title": "Building a REST API with FastAPI and PostgreSQL", "url": "https://dev.to/johndoe/building-a-rest-api-with-fastapi-and-postgresql-123", "description": "A comprehensive guide to building production-ready APIs using FastAPI...", "author": "johndoe", "author_image": "https://dev.to/user_images/johndoe/profile.jpg", "tags": ["python", "fastapi", "postgresql", "tutorial"], "published_at": "2025-06-15T10:30:00Z", "reading_time_minutes": 12, "reactions_count": 87, "comments_count": 14, "cover_image": "https://dev.to/social_previews/article/12345.png" } ``` #### Data fields | Field | Type | Description | | ------- | ------ | ------------- | | `title` | string | Article headline | | `url` | string | Full Dev.to article URL | | `description` | string | Excerpt or intro paragraph | | `author` | string | Dev.to username | | `author_image` | string | Profile image URL | | `tags` | array of strings | Tag list (e.g., `["python", "tutorial"]`) | | `published_at` | string | ISO 8601 publish date | | `reading_time_minutes` | integer | Estimated reading time | | `reactions_count` | integer | Total reactions (❤️ 🦄 🔖, etc.) | | `comments_count` | integer | Total comments | | `cover_image` | string | Cover image URL (nullable) | ### Use Cases #### Content discovery for tech newsletters Collect the most-reacted articles in `"ai"` or `"webdev"` every week. Schedule the Actor in Apify Console to run once a week, pipe the dataset into a spreadsheet, and curate your newsletter from the results. The `min_reactions` filter ensures only high-quality articles make the cut. #### Author portfolio monitoring Track a specific author's publishing frequency and engagement. Set `username` to follow a thought leader or competitor, run daily, and build a time-series dataset of their articles, reactions, and comments. #### Trend analysis across tags Compare engagement metrics across tags. Run the Actor with `tag: "python"` and `sort: "most_reactions"`, then repeat for `"rust"`, `"javascript"`, and `"go"`. Merge the datasets to see which ecosystem generates the most community buzz. #### Automated content pipeline Trigger the Actor via Apify webhook after a new blog post is published. Feed the resulting dataset into an AI summarization pipeline, a Slack bot, or a personal dashboard — all without writing any scraping infrastructure. #### Community health dashboards Use the Actor to sample Dev.to activity over time. Track how reaction counts, comment volumes, and posting frequency evolve across tags. Pair with Apify's scheduled runs and the Data API to build live dashboards in Grafana, Metabase, or a custom frontend. ### Pricing / Cost estimation How much does it cost to scrape Dev.to articles? - **First 10 results per run are free** - Each additional result costs **$0.002** (charged via `Actor.charge(event_name='result')`) - The Actor start event costs **$0.00005** - Example: scraping **25 articles** = 10 free + 15 charged = **$0.03** per run - Example: scraping **100 articles** = 10 free + 90 charged = **$0.18** per run The Actor uses the Dev.to public API with no authentication required for basic usage. If you need higher rate limits, add your free Dev.to API key from [Account Settings](https://dev.to/settings/account). ### Advanced options - **Rate limiting** — The Dev.to API has generous rate limits. For production workloads, set an `api_key` to avoid throttling - **Large datasets** — To collect more than 100 articles, run the Actor multiple times with different tags or sort orders and merge the datasets - **Local development** — Use `apify run --purge` to clear previous storage before a fresh test run ### FAQ, disclaimers, and support #### Is this legal? Yes. This Actor uses the **official Dev.to REST API** (`dev.to/api/articles`). It does not scrape HTML, bypass rate limits, or violate the Dev.to Terms of Service. The API is free and publicly documented at [docs.dev.to/api](https://docs.dev.to/api/). #### Why use an API instead of scraping HTML? The Dev.to API returns structured JSON directly. No HTML parsing, no fragile CSS selectors, no browser overhead. Requests are 10–100x faster than browser-based scraping, and the data is guaranteed to be correct because it comes from the platform's own data layer. #### Known limitations - Max **100 results** per API call (Dev.to API limit) - The `trending` sort covers the last **30 days** - The `most_reactions` sort covers the last **365 days** - No pagination is currently implemented (single-page fetch only) #### Support Found a bug or have a feature request? Open an issue on the [GitHub repository](https://github.com/apify-fleet/devto-articles-scraper) or submit feedback via the Apify Console Issues tab. For custom scraping solutions or Enterprise needs, contact the Apify team. *** Built with [Apify SDK for Python](https://docs.apify.com/sdk/python/) and [httpx](https://www.python-httpx.org/). # Actor input Schema ## `tag` (type: `string`): Filter articles by tag (e.g., 'python', 'ai', 'webdev', 'javascript') ## `username` (type: `string`): Fetch articles by a specific Dev.to username (overrides tag/sort) ## `sort` (type: `string`): How to sort articles ## `max_results` (type: `integer`): Maximum number of articles to return (max 100). First 10 are free, each additional costs $0.002. ## `min_reactions` (type: `integer`): Only return articles with at least this many reactions ## `api_key` (type: `string`): Optional Dev.to API key for higher rate limits (https://dev.to/settings/account) ## Actor input object example ```json { "tag": "python", "sort": "trending", "max_results": 25, "min_reactions": 0 } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = {}; // Run the Actor and wait for it to finish const run = await client.actor("ef12/devto-articles-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = {} # Run the Actor and wait for it to finish run = client.actor("ef12/devto-articles-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{}' | apify call ef12/devto-articles-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=ef12/devto-articles-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Dev.to Articles Scraper", "description": "Scrape Dev.to articles by tag, username, or trending. Get titles, descriptions, authors, tags, reactions, reading time, and comment counts via the free Dev.to API.", "version": "0.0", "x-build-id": "xcIJdxMlxncSWbO2K" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/ef12~devto-articles-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-ef12-devto-articles-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/ef12~devto-articles-scraper/runs": { "post": { "operationId": "runs-sync-ef12-devto-articles-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/ef12~devto-articles-scraper/run-sync": { "post": { "operationId": "run-sync-ef12-devto-articles-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "tag": { "title": "Tag", "type": "string", "description": "Filter articles by tag (e.g., 'python', 'ai', 'webdev', 'javascript')", "default": "python" }, "username": { "title": "Username", "type": "string", "description": "Fetch articles by a specific Dev.to username (overrides tag/sort)" }, "sort": { "title": "Sort Order", "enum": [ "trending", "newest", "most_reactions" ], "type": "string", "description": "How to sort articles", "default": "trending" }, "max_results": { "title": "Max Results", "minimum": 1, "maximum": 100, "type": "integer", "description": "Maximum number of articles to return (max 100). First 10 are free, each additional costs $0.002.", "default": 25 }, "min_reactions": { "title": "Minimum Reactions", "minimum": 0, "type": "integer", "description": "Only return articles with at least this many reactions", "default": 0 }, "api_key": { "title": "Dev.to API Key", "type": "string", "description": "Optional Dev.to API key for higher rate limits (https://dev.to/settings/account)" } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```