# Product Hunt Comment Scraper for Lead Generation (`danpoletaev/product-hunt-comment-scraper-for-lead-generation`) Actor Scrape commenter profiles from any Product Hunt daily leaderboard. Provide a date and get structured data for every unique commenter: name, avatar, verified status, LinkedIn, Twitter/X, bio, role (Maker, Team, etc.), comment text, and Launching Soon badge. Handles comment pagination automatically. - **URL**: https://apify.com/danpoletaev/product-hunt-comment-scraper-for-lead-generation.md - **Developed by:** [Danil Poletaev](https://apify.com/danpoletaev) (community) - **Categories:** Lead generation, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 1 bookmarks - **User rating**: No ratings yet ## Pricing from $25.00 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Product Hunt Comment Scraper Extract commenter profiles, social links, and comment data from any Product Hunt daily leaderboard. For every unique commenter across all products launched on a given date, this Actor scrapes their Product Hunt profile and returns structured data including LinkedIn, Twitter/X, bio, role (Maker, Team, etc.), and their original comment text. ### What does this Actor do? For a given date, this Actor: 1. Loads the Product Hunt daily leaderboard (e.g., `producthunt.com/leaderboard/daily/2024/7/25/all`) 2. Navigates to the latest launch page for each product 3. Scrapes every comment page (with automatic pagination), deduplicating commenters globally 4. Visits each unique commenter's profile 5. Outputs one structured record per commenter, merged with their comment context Powered by [Playwright](https://playwright.dev/) and [Camoufox](https://camoufox.com/) for reliable, bot-resistant browser automation. ### Why scrape Product Hunt comments? Product Hunt commenters are among the most engaged early adopters and builders on the internet. A targeted list of people actively discussing new products is valuable for outreach, research, and competitive intelligence. - **Lead generation** — find founders, makers, and investors commenting on products in your space - **Launching Soon outreach** — identify users with the "Launching Soon" badge who are about to launch their own product - **Competitive analysis** — see who is engaging with competitor launches and what they are saying - **Community research** — analyze the roles, backgrounds, and social profiles of a product's early audience ### Input | Field | Type | Required | Description | |-------|------|----------|-------------| | `date` | string | ✅ | Date to scrape in `YYYY-MM-DD` format | | `filter.topNProducts` | integer | ❌ | Limit to the top N products from the leaderboard | | `filter.minVotes` | integer | ❌ | Only include products with at least this many upvotes | #### Example input ```json { "date": "2024-07-25", "filter": { "topNProducts": 5, "minVotes": 50 } } ```` ### Output Each record represents one unique commenter. Profile data is merged with comment context from the product they commented on. | Field | Type | Description | |-------|------|-------------| | `name` | string | Commenter's display name | | `avatarUrl` | string | URL to their profile avatar | | `isVerified` | boolean | Whether the account has a verified badge | | `url` | string | Full Product Hunt profile URL | | `slug` | string | Username (e.g., `johndoe` from `/@johndoe`) | | `links` | array | All social and external links listed on the profile | | `linkedIn` | string | LinkedIn URL, if listed | | `twitter` | string | Twitter/X URL, if listed | | `aboutText` | string | Bio from the profile page | | `commentText` | string | Text of their first comment on the product | | `productHref` | string | Product Hunt product URL they commented on | | `role` | string | Role badge on the comment, e.g., `Maker` or `Team` | | `isLaunchingSoon` | boolean | Whether they had a "Launching Soon" badge | | `commentsLength` | integer | Total number of comments on that product | | `requestUrl` | string | Launch page URL where the comment was found | | `scrapedAt` | string | ISO timestamp of when the profile was scraped | #### Example output record ```json { "name": "Jane Smith", "avatarUrl": "https://ph-avatars.imgix.net/123456/avatar.jpeg", "isVerified": true, "url": "https://www.producthunt.com/@janesmith", "slug": "janesmith", "links": [ "https://www.linkedin.com/in/janesmith/", "https://x.com/janesmith" ], "linkedIn": "https://www.linkedin.com/in/janesmith/", "twitter": "https://x.com/janesmith", "aboutText": "Founder @ Acme. Building things people want.", "commentText": "This is exactly what I've been looking for. Congrats on the launch!", "productHref": "https://www.producthunt.com/products/some-product", "role": null, "isLaunchingSoon": false, "commentsLength": 42, "requestUrl": "https://www.producthunt.com/products/some-product/launches/some-product", "scrapedAt": "2024-07-25T14:32:00.000Z" } ``` ### How pagination works Product Hunt paginates comments across multiple pages. This Actor detects all available comment pages from the pagination links (e.g., `?page=2#comments`, `?page=3#comments`) and enqueues each one automatically — no comments are missed. Commenters are deduplicated globally: if the same person comments on multiple pages or products, their profile is only scraped once. ### Proxy and anti-bot This Actor uses Playwright with Firefox and Camoufox for fingerprint-resistant browser automation. When running on the Apify platform with a paid plan, it routes the initial leaderboard request through a US residential proxy and remaining requests through datacenter proxies to balance reliability and cost. ### Export formats Results can be exported from Apify Console in JSON, CSV, XML, Excel, and other formats. You can also access them via the [Apify API](https://docs.apify.com/api/v2) or connect to downstream tools using [Apify integrations](https://apify.com/integrations). ### Tips - Run or schedule this Actor at around 23:00 PST. Product Hunt resets its daily leaderboard at 00:01 PST, so running just before midnight gives you the most complete data for the day. - Use `filter.topNProducts` to limit the scope of a run and reduce compute time when you only need data from the top-ranked products. ### FAQ **How many results can I expect?** It depends on how many products you scrape and how active the comment sections are. A single top product on a popular day can have 50+ unique commenters across multiple comment pages. **Is the data real-time?** Yes. The Actor fetches live data directly from Product Hunt on each run. **Can I scrape multiple dates?** Each Actor run processes one date. To scrape multiple dates, trigger multiple runs via the [Apify API](https://docs.apify.com/api/v2#tag/Actor-runs/operation/act_runs_post) or schedule recurring runs in Apify Console. **Will it capture all comments?** Yes. The Actor handles comment pagination automatically and deduplicates commenters across all pages and products. # Actor input Schema ## `date` (type: `string`): The date to scrape the Product Hunt leaderboard for, in YYYY-MM-DD format. Defaults to yesterday in US/Pacific time (matching Product Hunt's daily reset). ## `filter` (type: `object`): Optional filters for products. Without filters all listings for the given date will be fetched. ## Actor input object example ```json { "date": "2024-07-25", "filter": { "minVotes": 100, "topNProducts": 10 } } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "filter": { "topNProducts": 10 } }; // Run the Actor and wait for it to finish const run = await client.actor("danpoletaev/product-hunt-comment-scraper-for-lead-generation").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "filter": { "topNProducts": 10 } } # Run the Actor and wait for it to finish run = client.actor("danpoletaev/product-hunt-comment-scraper-for-lead-generation").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "filter": { "topNProducts": 10 } }' | apify call danpoletaev/product-hunt-comment-scraper-for-lead-generation --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=danpoletaev/product-hunt-comment-scraper-for-lead-generation", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Product Hunt Comment Scraper for Lead Generation", "description": "Scrape commenter profiles from any Product Hunt daily leaderboard. Provide a date and get structured data for every unique commenter: name, avatar, verified status, LinkedIn, Twitter/X, bio, role (Maker, Team, etc.), comment text, and Launching Soon badge. Handles comment pagination automatically.", "version": "0.0", "x-build-id": "rDffG5m4BKTM2YM2E" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/danpoletaev~product-hunt-comment-scraper-for-lead-generation/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-danpoletaev-product-hunt-comment-scraper-for-lead-generation", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/danpoletaev~product-hunt-comment-scraper-for-lead-generation/runs": { "post": { "operationId": "runs-sync-danpoletaev-product-hunt-comment-scraper-for-lead-generation", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/danpoletaev~product-hunt-comment-scraper-for-lead-generation/run-sync": { "post": { "operationId": "run-sync-danpoletaev-product-hunt-comment-scraper-for-lead-generation", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "date": { "title": "Date", "type": "string", "description": "The date to scrape the Product Hunt leaderboard for, in YYYY-MM-DD format. Defaults to yesterday in US/Pacific time (matching Product Hunt's daily reset)." }, "filter": { "title": "Filter", "type": "object", "description": "Optional filters for products. Without filters all listings for the given date will be fetched." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```