Hacker News Jobs Scraper

Scrape structured job listings from Hacker News "Who is Hiring?" monthly threads. Turns unstructured comment text into clean, filterable job data.

What it does

Every month, Hacker News hosts an "Ask HN: Who is Hiring?" thread where hundreds of companies post job openings. This actor scrapes those threads and parses each comment into structured data: company name, role, location, tech stack, salary, and remote status.

Use cases

• Job seekers: Search HN jobs by technology, location, or remote availability. Filter down to exactly what you want.
• Recruiters: Track which companies are hiring and what tech stacks they use.
• Market research: Analyze hiring trends, salary ranges, and popular technologies across the tech industry.
• Competitive intelligence: Monitor which competitors are scaling up and what roles they need.
• Newsletter curation: Build a curated weekly jobs email from HN data.

Input options

Parameter	Description	Default
Month	Month number (1-12). 0 = latest thread.	0 (latest)
Year	Year (e.g., 2024). 0 = latest thread.	0 (latest)
Keyword Filter	Comma-separated keywords to filter by (e.g., "python, remote")	All jobs
Remote Only	Only return jobs that mention remote work	No
Max Results	Maximum job listings to return	500

Output format

{
    "company": "Stripe",
    "location": "San Francisco, Remote",
    "remote": true,
    "title": "Senior Backend Engineer",
    "description": "Stripe | Senior Backend Engineer | SF or Remote | $180k-$250k...",
    "technologies": ["Go", "Ruby", "AWS", "PostgreSQL", "Kubernetes"],
    "url": "https://stripe.com/jobs/listing/senior-backend-123",
    "salary_info": "$180k-$250k",
    "posted_date": "2024-01-02T16:00:00.000Z",
    "hn_comment_id": "38765432",
    "hn_comment_url": "https://news.ycombinator.com/item?id=38765432"
}

Technology detection

The actor recognizes 50+ technologies across these categories:

• Languages: Python, JavaScript, TypeScript, Rust, Go, Java, C++, C#, Ruby, PHP, Scala, Kotlin, Swift, Elixir, and more
• Frameworks: React, Next.js, Vue, Angular, Django, Flask, FastAPI, Rails, Spring, Node.js
• Infrastructure: AWS, GCP, Azure, Docker, Kubernetes, Terraform
• Databases: PostgreSQL, MongoDB, Redis, Elasticsearch, Kafka
• AI/ML: TensorFlow, PyTorch, LLM, Machine Learning, NLP, Computer Vision

How it works

1. Searches for the "Who is Hiring?" thread using the Algolia HN Search API
2. Fetches all top-level comments from the thread (each comment = one company posting)
3. Parses each comment to extract structured fields using pattern matching
4. Applies your filters (keywords, remote status)
5. Pushes clean results to the Apify dataset

Parsing accuracy

HN job comments follow a loose convention (Company | Role | Location | ...) but there is no strict format. The parser handles most common formats well, including:

• Pipe-separated headers
• Inline URLs
• Salary ranges in various formats ($120k, $120,000, 120k-180k USD)
• Location mentions (cities, countries, "Remote")

Some edge cases with unusual formatting may have incomplete extractions.

Tips

• Run monthly after the 1st to catch the new thread.
• Use keyword filter "python, django" to find Python jobs specifically.
• Enable "Remote Only" to skip office-only positions.
• Export to Google Sheets and sort by technology to spot trends.
• The thread typically gets 500-800 top-level comments (one per company).