# Hashnode Scraper (`automation-lab/hashnode-scraper`) Actor Scrape public Hashnode article feeds by tag or URL. Export titles, authors, dates, excerpts, cover images, and article links for developer-content research. - **URL**: https://apify.com/automation-lab/hashnode-scraper.md - **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community) - **Categories:** Developer tools, Social media - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Hashnode Scraper Collect public Hashnode articles from tags and feed URLs. Use this actor to monitor developer conversations, discover new technical content, and export article metadata for research, newsletters, competitive intelligence, and developer-relations workflows. ### What does Hashnode Scraper do? Hashnode Scraper extracts public article cards from Hashnode tag and topic feeds such as `https://hashnode.com/n/ai` or `https://hashnode.com/tag/javascript`. It reads the server-rendered page metadata and saves clean dataset rows with article titles, URLs, authors, excerpts, publication dates, cover images, word counts, and estimated reading time. ### Who is it for? - ๐Ÿง‘โ€๐Ÿ’ป Developer relations teams tracking what engineers write about a language, framework, or product category. - ๐Ÿ“ฐ Newsletter writers looking for fresh posts on AI, JavaScript, DevOps, startups, or open-source tools. - ๐Ÿ“ˆ Content marketers researching which technical topics are gaining attention. - ๐Ÿงช Product researchers monitoring developer feedback around libraries, APIs, and engineering workflows. - ๐Ÿงฐ Automation builders who need a reliable Hashnode dataset for downstream enrichment. ### Why use it? Hashnode is a public developer publishing platform. Manual browsing is fine for one tag, but it becomes slow when you need repeatable exports across many topics. This actor turns public Hashnode feeds into structured data that can be filtered, deduplicated, joined with other sources, or sent into BI and AI pipelines. ### What data can you extract? | Field | Description | | --- | --- | | `tag` | Hashnode tag inferred from the input feed | | `feedUrl` | Tag or feed URL that produced the item | | `position` | Position in the current run output | | `title` | Article headline | | `articleUrl` | Public article URL | | `canonicalUrl` | Article URL without tracking parameters | | `authorName` | Author display name when available | | `authorUrl` | Public Hashnode author profile URL | | `authorHandle` | Hashnode handle inferred from author URL | | `publicationName` | Publisher name from JSON-LD metadata | | `publicationUrl` | Publisher URL from JSON-LD metadata | | `excerpt` | Article description or preview text | | `imageUrl` | Cover image URL when present | | `datePublished` | Publication timestamp from Hashnode metadata | | `dateModified` | Modification timestamp when article detail metadata is fetched | | `wordCount` | Word count from Hashnode structured data | | `estimatedReadTimeMinutes` | Estimated read time based on word count | | `tags` | Detail-page keywords when available | | `scrapedAt` | ISO timestamp for the scrape | ### How much does it cost to scrape Hashnode articles? The actor uses pay-per-event pricing: a small start fee plus a per-article result fee. Small monitoring runs are inexpensive because the default input is capped at 25 articles. For large recurring jobs, set `maxItems` to the smallest number that gives your workflow enough fresh content. ### How to use Hashnode Scraper 1. Open the actor on Apify. 2. Enter one or more Hashnode tags such as `ai`, `javascript`, `devops`, or `startup`. 3. Optionally add full Hashnode feed URLs. 4. Set `maxItems` to your desired article limit. 5. Keep detail fetching disabled for fastest runs, or enable it when you need extra article metadata. 6. Start the run and download the dataset as JSON, CSV, Excel, or through the API. ### Input options #### `tags` A list of Hashnode tag slugs without the `#` sign. Example: ```json ["ai", "javascript", "startup"] ```` #### `startUrls` Optional full Hashnode feed URLs. Example: ```json [{ "url": "https://hashnode.com/n/ai" }] ``` #### `maxItems` Maximum number of article records to save across all inputs. The default is intentionally low so first runs are cheap. #### `includeArticleDetails` When enabled, the actor opens each article page and attempts to read more JSON-LD metadata. Disable it for lightweight feed monitoring. ### Output example ```json { "tag": "ai", "feedUrl": "https://hashnode.com/n/ai", "position": 1, "title": "How I Evaluate AI Models for Real Product Workflows", "articleUrl": "https://example.hashnode.dev/how-i-evaluate-ai-models", "canonicalUrl": "https://example.hashnode.dev/how-i-evaluate-ai-models", "authorName": "Jane Developer", "authorUrl": "https://hashnode.com/@janedev", "authorHandle": "@janedev", "excerpt": "Choosing an AI model is not a one-time decision...", "datePublished": "2026-05-29T08:40:44.781Z", "wordCount": 500, "estimatedReadTimeMinutes": 2, "scrapedAt": "2026-05-29T09:00:00.000Z" } ``` ### Tips for better results - โœ… Use multiple tags when you need more volume, for example `ai`, `machine-learning`, and `llm`. - โœ… Keep `includeArticleDetails` off for scheduled trend monitoring. - โœ… Use detail fetching for one-off enrichment runs where extra metadata matters more than speed. - โœ… Deduplicate downstream by `canonicalUrl` if you combine many related tags. - โœ… Schedule the actor daily or weekly to watch new technical content. ### Integrations Hashnode Scraper works well with: - Google Sheets or Airtable for editorial planning. - Slack or Discord notifications for new posts about a product category. - Vector databases for searchable developer-content archives. - CRM enrichment workflows for developer advocates and content teams. - LLM pipelines that summarize public engineering discussions. ### API usage #### Node.js ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('automation-lab/hashnode-scraper').call({ tags: ['ai', 'javascript'], maxItems: 50 }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items); ``` #### Python ```python from apify_client import ApifyClient import os client = ApifyClient(os.environ['APIFY_TOKEN']) run = client.actor('automation-lab/hashnode-scraper').call(run_input={ 'tags': ['ai', 'javascript'], 'maxItems': 50, }) items = client.dataset(run['defaultDatasetId']).list_items().items print(items) ``` #### cURL ```bash curl -X POST "https://api.apify.com/v2/acts/automation-lab~hashnode-scraper/runs?token=$APIFY_TOKEN" \ -H 'Content-Type: application/json' \ -d '{"tags":["ai","javascript"],"maxItems":50}' ``` ### MCP usage Use the actor through Apify MCP with tools scoped to this actor: ```text https://mcp.apify.com/?tools=automation-lab/hashnode-scraper ``` Claude Code setup: ```bash claude mcp add apify-hashnode https://mcp.apify.com/?tools=automation-lab/hashnode-scraper ``` Claude Desktop JSON config: ```json { "mcpServers": { "apify-hashnode": { "url": "https://mcp.apify.com/?tools=automation-lab/hashnode-scraper" } } } ``` Example prompts: - "Run Hashnode Scraper for the `ai` tag and summarize the newest posts." - "Find recent Hashnode articles about JavaScript tooling and export the URLs." - "Monitor Hashnode startup posts weekly and identify repeated pain points." ### Scheduling Create an Apify schedule to run the actor every morning for your target tags. Combine the dataset with a webhook to send new article URLs to Slack, Notion, or an internal dashboard. ### Data quality notes The actor reads public structured metadata embedded in Hashnode pages. Some articles may omit cover images, detail keywords, or modified timestamps. The actor leaves unavailable values as `null` instead of inventing data. ### Performance notes For normal feed monitoring, this is an HTTP-only actor and does not open a browser. Enabling article detail fetching adds one HTTP request per article, so use it only when you need more metadata. ### Legality This actor extracts publicly available Hashnode page metadata. You are responsible for using the data in line with applicable laws, Hashnode terms, and privacy rules. Do not use scraped data for spam, harassment, or unauthorized profiling. ### FAQ #### Can I scrape any Hashnode tag? You can provide public Hashnode tag slugs or public Hashnode feed URLs. If a tag has few recent posts, the actor may return fewer items than requested. #### Why did I get fewer articles than `maxItems`? Hashnode may show fewer public posts for that tag in the initial feed. Add related tags or start URLs to increase volume. #### Does this scrape private drafts or gated content? No. It only reads public pages available without logging in. #### Should I enable article detail fetching? Enable it when you need extra article-page metadata. Leave it disabled for cheaper and faster monitoring. ### Troubleshooting If a run returns no items, check that the tag exists publicly on Hashnode. Try a broad tag such as `ai` or `javascript`. If a specific article field is empty, the page likely did not expose that value in structured metadata. ### Related scrapers You may also find these automation-lab actors useful: - https://apify.com/automation-lab/github-repositories-search-scraper - https://apify.com/automation-lab/product-hunt-scraper - https://apify.com/automation-lab/medium-scraper - https://apify.com/automation-lab/rss-feed-scraper ### Changelog #### 0.1 Initial release with tag inputs, feed URLs, public article metadata extraction, optional article detail enrichment, and structured dataset output. # Actor input Schema ## `tags` (type: `array`): Hashnode tag slugs to scrape. Use values without the # sign, for example ai, javascript, startup, or devops. ## `startUrls` (type: `array`): Optional Hashnode tag/feed URLs such as https://hashnode.com/n/ai or https://hashnode.com/tag/javascript. ## `maxItems` (type: `integer`): Maximum number of article records to save across all tags and start URLs. ## `includeArticleDetails` (type: `boolean`): When enabled, the actor opens each article page to look for extra JSON-LD metadata. Leave disabled for faster feed monitoring. ## Actor input object example ```json { "tags": [ "ai", "javascript" ], "startUrls": [ { "url": "https://hashnode.com/n/ai" } ], "maxItems": 20, "includeArticleDetails": false } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "tags": [ "ai", "javascript" ], "startUrls": [ { "url": "https://hashnode.com/n/ai" } ], "maxItems": 20 }; // Run the Actor and wait for it to finish const run = await client.actor("automation-lab/hashnode-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "tags": [ "ai", "javascript", ], "startUrls": [{ "url": "https://hashnode.com/n/ai" }], "maxItems": 20, } # Run the Actor and wait for it to finish run = client.actor("automation-lab/hashnode-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "tags": [ "ai", "javascript" ], "startUrls": [ { "url": "https://hashnode.com/n/ai" } ], "maxItems": 20 }' | apify call automation-lab/hashnode-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=automation-lab/hashnode-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Hashnode Scraper", "description": "Scrape public Hashnode article feeds by tag or URL. Export titles, authors, dates, excerpts, cover images, and article links for developer-content research.", "version": "0.1", "x-build-id": "v0THD2kkfBKxxnDhu" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/automation-lab~hashnode-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-automation-lab-hashnode-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/automation-lab~hashnode-scraper/runs": { "post": { "operationId": "runs-sync-automation-lab-hashnode-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/automation-lab~hashnode-scraper/run-sync": { "post": { "operationId": "run-sync-automation-lab-hashnode-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "tags": { "title": "Tags", "type": "array", "description": "Hashnode tag slugs to scrape. Use values without the # sign, for example ai, javascript, startup, or devops.", "items": { "type": "string" } }, "startUrls": { "title": "Start URLs", "type": "array", "description": "Optional Hashnode tag/feed URLs such as https://hashnode.com/n/ai or https://hashnode.com/tag/javascript.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "maxItems": { "title": "Maximum articles", "minimum": 1, "maximum": 500, "type": "integer", "description": "Maximum number of article records to save across all tags and start URLs.", "default": 20 }, "includeArticleDetails": { "title": "Fetch article detail pages", "type": "boolean", "description": "When enabled, the actor opens each article page to look for extra JSON-LD metadata. Leave disabled for faster feed monitoring.", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```