HN/Reddit Pain Point Miner

What does HN/Reddit Pain Point Miner do?

HN/Reddit Pain Point Miner scrapes Hacker News (via the public Algolia API) and Reddit (via the anonymous JSON API across r/programming, r/MachineLearning, r/LocalLLaMA, r/ClaudeAI, r/ChatGPTCoding, r/aipromptprogramming and any subreddits you add) for posts where developers complain about AI coding agents — Claude Code, Cursor, GitHub Copilot, Cline, Aider, Windsurf, Devin, Replit Agent, v0, ChatGPT, GPT-4/5 and friends. Every post is scored 0-100 by a transparent pain heuristic (keyword match + engagement + recency + product mention) and the dataset is sorted from "loudest complaint" to "background noise" so you can use the top rows as validated MVP wedge ideas.

Try it from the Apify Console Try for free button at the top of this page, or call it via API, schedule it as a daily cron, and pipe results into Slack / Notion / Linear via Apify's integrations. The platform handles proxy rotation, retries, monitoring, and dataset export for you.

This Actor is part of Ian Mu's (github.com/ianymu) 100-Actor portfolio. It pairs naturally with claude-verify-before-stop, the open-source hook that keeps Claude Code from declaring victory on broken builds — both ship the same philosophy: turn vague developer pain into auditable signal.

Why use HN/Reddit Pain Point Miner?

Manual Reddit/HN browsing is slow and biased toward whatever's on the front page today. This Actor:

• Saves hours of manual research per idea cycle. What used to take an afternoon of tab-juggling now takes 60-120 seconds.
• Validates startup ideas with real complaints, not hypotheses. Every row is a public post with upvotes and comments — actual humans expressing actual frustration.
• Surfaces under-the-radar pain. The scoring rewards engagement density, not just upvotes, so high-comment-count "me too" threads with low scores still float to the top.
• Builds a longitudinal feedback database. Schedule it weekly; over 6 months you'll see which complaints persist (real product gaps) vs which fade (one-off rants).
• Feeds product teams, founders, devrel, and content marketers. Same dataset, three different read patterns: PMs find roadmap items, founders find wedges, devrel finds talking points, marketers find headline hooks.

How to use HN/Reddit Pain Point Miner to find AI coding agent complaints

1. Click Try for free at the top of this page (or hit Start if you're already on the Actor page).
2. Open the Input tab. Default queries already cover the major AI coding agents — you can leave them as-is.
3. (Optional) Adjust subreddits, daysBack (default 90), or minScore (default 30 — raise to 60 for only the loudest pains).
4. Click Save & Start. The run takes roughly 1-3 minutes depending on rate-limit waits.
5. Open the Output tab when the run finishes — rows are pre-sorted by painScore descending. Use the Export button to grab JSON / CSV / Excel.
6. (Optional) Open the Key-value store tab → pain-digest.md for a human-readable Markdown digest you can paste into Notion or a weekly review doc.

Input

The Actor accepts six optional fields (all have sensible defaults — you can hit Start with no edits):

{
  "queries": ["claude code", "cursor ai", "github copilot", "ai coding agent", "llm coding"],
  "sources": ["hn", "reddit"],
  "subreddits": ["programming", "MachineLearning", "LocalLLaMA", "ClaudeAI", "ChatGPTCoding", "aipromptprogramming"],
  "maxResultsPerSource": 100,
  "daysBack": 90,
  "minScore": 30
}

Open the Input tab in Apify Console for an interactive form with descriptions for each field.

Output

Each row in the dataset represents one pain-point post. You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

{
  "source": "reddit",
  "title": "Claude Code keeps deleting my tests — anyone else?",
  "url": "https://www.reddit.com/r/ClaudeAI/comments/xyz/claude_code_keeps_deleting_my_tests/",
  "author": "frustrated_dev_42",
  "createdAt": "2026-05-12T18:34:00.000Z",
  "score": 184,
  "commentsCount": 76,
  "subredditOrTag": "ClaudeAI",
  "painKeywordsMatched": ["doesn't work", "annoying"],
  "productsMentioned": ["claude code", "claude"],
  "painScore": 90,
  "excerpt": "Every time I run claude code on my Next.js repo it silently strips my Vitest files and replaces them with `// TODO: add tests`. This doesn't work for any serious codebase..."
}

A Markdown digest (pain-digest.md) is also written to the run's key-value store so you can paste a ranked table straight into a doc.

Data table

Field	Type	Description
painScore	number	0-100 composite signal. Higher = louder, more recent, more agreed-on complaint.
source	text	"hn" (Hacker News) or "reddit".
title	text	Original post title.
url	link	Canonical permalink. Click to read the full thread.
author	text	Original poster's username.
createdAt	ISO date	Post creation timestamp.
score	number	Upvotes (HN points or Reddit score).
commentsCount	number	Number of comments — strong proxy for "me too" agreement.
subredditOrTag	text	Subreddit name (Reddit) or "ask_hn" (HN).
painKeywordsMatched	array	Which pain words triggered (e.g. ["broken", "frustrating"]).
productsMentioned	array	AI tools detected via regex (e.g. ["cursor", "claude code"]).
excerpt	text	First 300 chars of the post body or title.

Pricing / Cost estimation — how much does it cost to scrape HN + Reddit pain points?

On a typical run with the defaults (5 queries × 6 subreddits + 5 HN queries, ~90-day window), the Actor finishes in 1-3 minutes of compute time. That fits comfortably inside Apify's $5/month free tier, so most users will run it for free. Cost scales linearly with query count and maxResultsPerSource; respect the built-in rate limits (1 req/sec to Reddit, 50-call cap on HN Algolia) to keep runs cheap and friendly to the upstream APIs.

Tips and advanced options

• Tighten daysBack to 14-30 when you want only fresh complaints (e.g. tracking how the community reacts to a recent Claude Code release).
• Raise minScore to 60 for executive-summary mode — fewer rows, only the unambiguous pains.
• Add niche subreddits like LangChain, OpenAI, Anthropic, selfhosted to widen the funnel.
• Schedule it weekly. Open the Actor's Schedules tab → cron 0 9 * * 1. Pipe the dataset into Slack/Notion via Apify integrations to get a Monday-morning pain dashboard.
• Combine with an LLM step. Feed the top 20 rows into an Apify LLM Dataset Processor Actor to auto-cluster complaints into MVP themes.

FAQ, disclaimers, and support

Is this legal? This Actor only reads publicly accessible posts via official endpoints (Algolia HN's open API and Reddit's public JSON endpoints). No login, no scraping past auth walls, no personal data beyond the public username field. Respect Reddit's content policy and HN's terms when reusing the data.

Why are some Reddit results missing? Reddit's anti-bot occasionally serves HTML instead of JSON for anonymous requests. The Actor detects this and skips gracefully with a warning — re-run a few minutes later and the missing subreddit usually comes back. For 100% coverage, layer Apify's Reddit Scraper or an authenticated proxy on top.

Why are my queries returning nothing for HN? HN's Algolia index drops very low-engagement story posts. Broaden the query (e.g. "claude" instead of "claude code") or extend daysBack.

Found a bug or want a feature? Open an issue at the Actor's Issues tab in Apify Console, or ping @ianymu on GitHub. Custom builds (e.g. mining a different category — design tools, devops, finance) are available — see github.com/ianymu for contact.

Built by Ian Mu. Part of the 100-Actor portfolio shipping useful Apify Actors for solopreneurs and product teams. MIT-licensed. Cross-link: claude-verify-before-stop — the verify-before-stop hook for Claude Code.