# Indie Hackers Discussions Scraper (`automation-lab/indie-hackers-discussions-scraper`) Actor Export public Indie Hackers discussion threads, founder topics, authors, timestamps, and source URLs from IndieHackers.com. - **URL**: https://apify.com/automation-lab/indie-hackers-discussions-scraper.md - **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community) - **Categories:** Social media - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Indie Hackers Discussions Scraper Export public Indie Hackers discussion threads from IndieHackers.com into a clean Apify dataset. This actor reads the public Indie Hackers Firebase JSON feed, so it is fast, inexpensive, and does not need browser automation, login, or proxies. ### What does Indie Hackers Discussions Scraper do? Indie Hackers Discussions Scraper collects public discussion threads from Indie Hackers. It can export recent threads, newly created threads, updated threads, or keyword-filtered conversations. Each result includes the thread id, title, author, timestamps, bump metrics, source URL, and optional body content. You can also enable recent comment scanning to attach public comments that are visible in the public comments feed. ### Who is it for? 🎯 Startup founders use it to monitor market pain points and founder discussions. 📈 Growth marketers use it to find conversations around distribution, launch strategy, SaaS, and product ideas. 🧑‍💻 Indie builders use it to track topics that resonate with the Indie Hackers community. 🔎 Lead researchers use it to discover founders, usernames, and discussion context for manual follow-up. 🤖 AI workflow builders use it to feed recent founder conversations into LLM summaries or alerts. ### Why use this actor? It is HTTP-only and lightweight. It returns structured data instead of browser screenshots or raw HTML. It uses public data only. It supports keyword filtering. It includes both ISO timestamps and raw timestamps for easy downstream processing. It can run cheaply for scheduled monitoring jobs. ### Public data source The actor uses public Indie Hackers discussion data. The primary source is the public Firebase endpoint behind IndieHackers.com. No Indie Hackers account is required. No session cookies are required. No CSRF token is required. No browser automation is required. ### Data you can extract | Field | Description | | --- | --- | | `threadId` | Stable Firebase id of the discussion thread | | `title` | Public discussion title | | `content` | Public discussion body when enabled | | `username` | Public author username | | `userId` | Public author id | | `createdAt` | ISO creation time | | `updatedAt` | ISO update time | | `bumpedAt` | ISO bump time | | `bumpCount` | Public bump count | | `boostPoints` | Public boost points | | `commentCount` | Number of attached comments found by the optional scan | | `comments` | Optional nested recent comment records | | `matchedKeyword` | Keyword filter used for the match | | `sourceUrl` | Indie Hackers source URL | ### How much does it cost to scrape Indie Hackers discussions? This actor uses pay-per-event pricing. The default result event is one saved discussion thread. BRONZE pricing is $0.0000808 per saved discussion thread (about $0.0808 per 1,000 saved discussion threads), plus a $0.005 run-start event. Tier discounts apply automatically based on the user's Apify plan. Small monitoring runs are designed to be inexpensive. ### Input options #### Maximum threads Set `maxItems` to control how many discussion thread records are saved. The default is 50. The prefill is intentionally small so first runs stay cheap. #### Sort public threads by Choose the timestamp field used for ordering. Use `bumpedAt` to monitor recently active discussions. Use `createdAt` to monitor newly created discussions. Use `updatedAt` to monitor edited discussions. #### Sort direction Use `newest` for recent monitoring. Use `oldest` for archive-style exports. #### Keyword filter Use `keyword` to match discussion titles, bodies, and usernames. Leave it empty to export the latest public discussions. #### Include thread content Keep `includeContent` enabled when you want the full discussion body. Disable it when you only need metadata and source URLs. #### Attach recent comments Enable `includeComments` to scan the latest public comments feed. The comment scan is best-effort because Indie Hackers does not expose a complete public thread-specific comment index without restricted paths. ### Example input ```json { "maxItems": 25, "sortBy": "bumpedAt", "sortDirection": "newest", "keyword": "distribution", "includeContent": true, "includeComments": false, "commentScanLimit": 200 } ```` ### Example output ```json { "threadId": "-Ol7IaOMfkxU0TSITgnF", "title": "What's harder: building the thing or getting people to use it?", "content": "4 days into my founder journey...", "username": "ZeroRook", "userId": "6VpcNYNi7nO35UbSifIgs3RMZHB3", "createdAt": "2026-02-10T12:07:03.427Z", "createdAtTimestamp": 1770739623427, "updatedAt": "2026-02-10T12:07:03.427Z", "updatedAtTimestamp": 1770739623427, "bumpedAt": "2026-02-10T12:07:03.427Z", "bumpedAtTimestamp": 1770739623427, "bumpCount": 0, "boostPoints": 0, "commentCount": 0, "matchedKeyword": "distribution", "sourceUrl": "https://www.indiehackers.com/post/-Ol7IaOMfkxU0TSITgnF", "sourceId": "-Ol7IaOMfkxU0TSITgnF" } ``` ### How to run it Open the actor on Apify. Set `maxItems` to the number of discussions you need. Choose `bumpedAt` for active discussions or `createdAt` for new posts. Optionally enter a keyword. Click Start. Download the dataset as JSON, CSV, Excel, or through the API. ### Tips for better results Use broad keywords such as `distribution`, `marketing`, `SaaS`, `founder`, or `launch`. Use scheduled runs to monitor new founder discussions every day. Use `bumpedAt` when you care about active conversations. Use `createdAt` when you care about brand-new posts. Keep comment scanning disabled unless you need comment context. Increase `commentScanLimit` only when comment matches are important. ### Integrations Send new discussions to Slack for founder-signal monitoring. Export keyword-filtered threads to Google Sheets for lead research. Feed discussion content into an LLM summarizer for weekly market intelligence. Trigger a CRM workflow when a thread mentions a relevant pain point. Build a content ideation dashboard from recurring Indie Hackers topics. ### API usage with Node.js ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('automation-lab/indie-hackers-discussions-scraper').call({ maxItems: 25, sortBy: 'bumpedAt', keyword: 'distribution', includeContent: true }); console.log(run.defaultDatasetId); ``` ### API usage with Python ```python from apify_client import ApifyClient client = ApifyClient('MY-APIFY-TOKEN') run = client.actor('automation-lab/indie-hackers-discussions-scraper').call(run_input={ 'maxItems': 25, 'sortBy': 'bumpedAt', 'keyword': 'distribution', 'includeContent': True, }) print(run['defaultDatasetId']) ``` ### API usage with cURL ```bash curl -X POST 'https://api.apify.com/v2/acts/automation-lab~indie-hackers-discussions-scraper/runs?token=MY-APIFY-TOKEN' \ -H 'Content-Type: application/json' \ -d '{"maxItems":25,"sortBy":"bumpedAt","keyword":"distribution","includeContent":true}' ``` ### MCP usage Use this actor from Apify MCP tools in Claude Desktop or Claude Code. MCP URL: ```text https://mcp.apify.com/?tools=automation-lab/indie-hackers-discussions-scraper ``` Claude Code setup: ```bash claude mcp add apify-indie-hackers "https://mcp.apify.com/?tools=automation-lab/indie-hackers-discussions-scraper" ``` Claude Desktop JSON config example: ```json { "mcpServers": { "apify-indie-hackers": { "url": "https://mcp.apify.com/?tools=automation-lab/indie-hackers-discussions-scraper" } } } ``` Example prompt: ```text Run the Indie Hackers Discussions Scraper for 50 recently bumped posts about distribution and summarize the top pain points founders mention. ``` Another example prompt: ```text Find recent Indie Hackers discussions mentioning SaaS onboarding and return the source URLs plus short summaries. ``` ### Scheduling ideas Run every morning for `bumpedAt` discussions. Run weekly for a list of recurring keywords. Send keyword matches to a webhook. Archive monthly exports to a data warehouse. ### Limitations The actor exports public data only. Private or restricted Firebase paths are intentionally avoided. Comment attachment is best-effort and scans recent public comments instead of restricted thread-comment indexes. Indie Hackers may change its public data structure in the future. ### Troubleshooting #### Why did I get fewer results than `maxItems`? A keyword filter may remove records from the fetched public thread window. Try a broader keyword or leave the keyword empty. #### Why are comments empty? The public comments feed is scanned by recency. If no recently scanned comments match the exported thread ids, the actor leaves comments empty. #### Why are some timestamps unusual? The actor preserves the raw public timestamps returned by Indie Hackers and also converts them to ISO strings. ### Data quality checks Each item includes a source id. Each item includes a source URL. Numbers are stored as numbers. Timestamps are stored as both ISO strings and raw numeric values. Optional missing fields are returned as `null` instead of invented values. ### Legality This actor collects publicly available information. You are responsible for using exported data in compliance with applicable laws, platform terms, and privacy rules. Do not use the data for spam. Do not attempt to access private Indie Hackers paths. ### Related scrapers You may also find these automation-lab actors useful: - https://apify.com/automation-lab/product-hunt-scraper - https://apify.com/automation-lab/tiny-startups-scraper - https://apify.com/automation-lab/reddit-scraper - https://apify.com/automation-lab/hacker-news-scraper ### FAQ #### Does this actor require an Indie Hackers account? No. It uses public JSON data only. #### Does it use a browser? No. It is HTTP-only. #### Does it use proxies? No proxy is required for the public Firebase endpoint. #### Can I filter by topic? Yes. Use the `keyword` input. #### Can I export comments? Yes, but comment attachment is best-effort through the recent public comments feed. #### Can I schedule it? Yes. Apify schedules work well for daily or weekly monitoring. ### Changelog Initial version exports public Indie Hackers discussion threads with metadata, content, keyword filtering, and optional recent comment attachment. # Actor input Schema ## `maxItems` (type: `integer`): Maximum number of public Indie Hackers discussion threads to save. ## `sortBy` (type: `string`): Firebase timestamp field used to order discussion threads before export. ## `sortDirection` (type: `string`): Return newest discussions first or oldest discussions first for the selected timestamp field. ## `keyword` (type: `string`): Optional case-insensitive keyword filter applied to thread title, content, and username. Leave empty to export all fetched discussions. ## `includeContent` (type: `boolean`): Include the public discussion body/content in each dataset item. ## `includeComments` (type: `boolean`): Scan the latest public comments feed and attach matching comments for exported threads when available. ## `commentScanLimit` (type: `integer`): When comments are enabled, scan this many latest public comments and attach those whose threadId matches exported threads. ## Actor input object example ```json { "maxItems": 10, "sortBy": "bumpedAt", "sortDirection": "newest", "includeContent": true, "includeComments": false, "commentScanLimit": 20 } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "maxItems": 10, "sortBy": "bumpedAt", "sortDirection": "newest", "includeContent": true, "includeComments": false, "commentScanLimit": 20 }; // Run the Actor and wait for it to finish const run = await client.actor("automation-lab/indie-hackers-discussions-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "maxItems": 10, "sortBy": "bumpedAt", "sortDirection": "newest", "includeContent": True, "includeComments": False, "commentScanLimit": 20, } # Run the Actor and wait for it to finish run = client.actor("automation-lab/indie-hackers-discussions-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "maxItems": 10, "sortBy": "bumpedAt", "sortDirection": "newest", "includeContent": true, "includeComments": false, "commentScanLimit": 20 }' | apify call automation-lab/indie-hackers-discussions-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=automation-lab/indie-hackers-discussions-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Indie Hackers Discussions Scraper", "description": "Export public Indie Hackers discussion threads, founder topics, authors, timestamps, and source URLs from IndieHackers.com.", "version": "0.1", "x-build-id": "ji0LKYUrkr4aXrOgp" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/automation-lab~indie-hackers-discussions-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-automation-lab-indie-hackers-discussions-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/automation-lab~indie-hackers-discussions-scraper/runs": { "post": { "operationId": "runs-sync-automation-lab-indie-hackers-discussions-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/automation-lab~indie-hackers-discussions-scraper/run-sync": { "post": { "operationId": "run-sync-automation-lab-indie-hackers-discussions-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "maxItems": { "title": "Maximum threads", "minimum": 1, "maximum": 1000, "type": "integer", "description": "Maximum number of public Indie Hackers discussion threads to save.", "default": 50 }, "sortBy": { "title": "Sort public threads by", "enum": [ "bumpedAt", "createdAt", "updatedAt" ], "type": "string", "description": "Firebase timestamp field used to order discussion threads before export.", "default": "bumpedAt" }, "sortDirection": { "title": "Sort direction", "enum": [ "newest", "oldest" ], "type": "string", "description": "Return newest discussions first or oldest discussions first for the selected timestamp field.", "default": "newest" }, "keyword": { "title": "Keyword filter", "type": "string", "description": "Optional case-insensitive keyword filter applied to thread title, content, and username. Leave empty to export all fetched discussions." }, "includeContent": { "title": "Include thread content", "type": "boolean", "description": "Include the public discussion body/content in each dataset item.", "default": true }, "includeComments": { "title": "Attach recent comments", "type": "boolean", "description": "Scan the latest public comments feed and attach matching comments for exported threads when available.", "default": false }, "commentScanLimit": { "title": "Recent comments to scan", "minimum": 1, "maximum": 5000, "type": "integer", "description": "When comments are enabled, scan this many latest public comments and attach those whose threadId matches exported threads.", "default": 500 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```