Hacker News Scraper Pro - Enterprise Developer Intelligence

🚀 NO PROXY REQUIRED | Extract stories, users, jobs from Hacker News | Enterprise developer intelligence & tech trend analysis

Description

🚀 NO PROXY REQUIRED | Extract stories, users, jobs from Hacker News | Enterprise developer intelligence & tech trend analysis

Features

📌 Status 🧠 Context [2026-03-12] — outputSchema description added (spec-005)

How It Works

This actor uses intelligent extraction with Apify's proxy infrastructure to gather public data from Hacker News Scraper Pro - Enterprise Developer Intelligence. It handles authentication, pagination, and error recovery automatically.

Quick Start

# Run via Apify API
apify run -a red.cars/hackernews-scraper-pro

# Or configure input and click Run on:
# https://apify.com/red.cars/hackernews-scraper-pro

Using as an MCP Tool

Running on Apify Platform

Set APIFY_API_KEY in your Apify Console secrets, configure input via the Actor input schema UI, and click Run.

Input Schema

Parameter	Type	Description	Default
startUrls	array	List of Hacker News URLs to scrape (stories, lists, or specific pages). Leave empty to use mode setting.	-
mode	string	Which Hacker News section to scrape (used when startUrls is empty) Options: frontpage, newest, ask, show, jobs	frontpage
users	array	List of usernames to extract profile data for (karma, join date, about)	-
maxItems	integer	Maximum number of items to extract (stories, jobs, or users)	100
debugMode	boolean	Enable minimal extraction for health checks and testing. Guarantees success within 300s.	false
includeComments	boolean	Extract comments from story pages (may significantly increase runtime)	false
checkOnly	boolean	When true, returns immediately with capability metadata without scraping. Used by AI agents to verify availability before committing to a paid run. Free, <5 seconds.	false

Infrastructure & Reliability

| proxyType | string | Choose your preferred balance of cost vs reliability. Standard (Datacenter) is faster; Premium (Residential) is most reliable for high-security targets.
Options: DATACENTER, RESIDENTIAL | DATACENTER |

Output Schema

Field	Type	Description
results	string	Hacker News stories, Ask HN threads, job posts, and Show HN launches. Each result contains: type, id, title, url, points, author, commentCount, publishedAt, rank, cleanMarkdown. Use for tech news monitoring, startup discovery, developer sentiment analysis, VC deal flow intelligence. AI agents: filter by type (story/job/ask) and points threshold.

Example Output

{
  "results": [
    {
      "url": "https://example.com/profile",
      "username": "example_user",
      "fullName": "Example User",
      "followersCount": 10000,
      "followingCount": 500,
      "postsCount": 250,
      "isVerified": false,
      "biography": "Example bio text",
      "latestPosts": [],
      "scrapedAt": "2026-03-23T13:24:15.342Z"
    }
  ]
}

Pricing

Use Cases

• Influencer Research: Identify and evaluate potential influencers by engagement metrics and audience quality
• Competitor Monitoring: Track competitor presence and activity on Target Platform
• Lead Generation: Build targeted lead lists from public profiles and contact information
• Market Intelligence: Gather market intelligence and trend data for business decisions
• Content Aggregation: Collect content for analysis, archiving, or republication

FAQ

Do I need an API key?

No, this actor does not require an API key. It uses Apify proxy infrastructure for reliable extraction.

How does pricing work?

This actor uses Apify's Pay-Per-Event model. You are charged per successful result returned. No charge for queries that return zero results.

What are the rate limits?

Rate limits depend on your Apify plan. Higher-tier plans provide more compute units and faster extraction.

How do I increase success rate?

• Enable Premium (Residential) proxy for high-security targets
• Reduce concurrency for rate-limited profiles
• Use debug mode to test before full extraction

Can I run this on a schedule?

Yes, use Apify Scheduler to run this actor on a cron schedule for continuous monitoring.

How do I export to CRM?

Set exportFormat to salesforce or hubspot for direct CRM import format.

Troubleshooting

Error	Cause	Fix
EXTRACTION_ERROR	Profile is private or not found	Check if profile exists and is public
RATE_LIMIT_ERROR	Too many requests	Wait and retry or use residential proxy
NETWORK_ERROR	Network connectivity issue	Check internet connection, retry

Known Limitations

• Private accounts: Cannot extract data from private accounts without following
• Rate limiting: Platform may temporarily block repeated requests from same IP
• Data freshness: Extracted data reflects moment of extraction; historical data may be limited
• Proxy requirements: High-security targets may require residential proxy for reliable extraction
• Content limits: Platform-imposed limits on historical post retrieval

Legal

Data Source: the target platform (the target platform.com)

Terms of Service: This actor is provided for legitimate data collection purposes only. Users must comply with the target platform's Terms of Service and applicable laws. Red.cars is not responsible for misuse of this tool.

Privacy: All extracted data is processed in accordance with applicable privacy laws. Do not use this tool for unauthorized data collection or privacy-violating activities.

Attribution: When using the target platform data, comply with their attribution requirements and data policies.

---

red.cars Intelligence Fleet • apify.com/red.cars