Hacker News Algolia Search Scraper

What it does

This actor scrapes Hacker News stories using the Algolia search API. It provides full-text search across all Hacker News stories and returns detailed results including titles, URLs, authors, points, comment counts, and timestamps. The Algolia API offers fast, relevance-ranked search results with page-based pagination. The actor converts Unix timestamps to ISO format and handles all pagination automatically.

Why use this actor

This actor provides a simple, reliable, and scalable way to extract data from public APIs without needing to write any code or manage infrastructure. It handles pagination, rate limiting, error recovery, and data normalization automatically. Whether you are a researcher, analyst, or developer, this actor saves you hours of manual data collection work. The structured JSON output integrates seamlessly with spreadsheets, databases, dashboards, and downstream data pipelines. Run it on Apify platform for automatic scheduling, monitoring, and proxy management.

Input parameters

• searchQuery (string, required): Search term to find stories on Hacker News. Default: "artificial intelligence".
• maxResults (integer, optional): Maximum number of stories to return. Range: 1-1000. Default: 100.

Output data

The actor outputs a dataset with the following fields:

• objectID: Algolia object ID (Hacker News story ID)
• title: Story title
• url: External URL linked in the story
• author: Hacker News username of the poster
• points: Number of upvote points
• numComments: Number of comments on the story
• createdAt: Story creation time (ISO 8601 format)
• storyText: Story text content (for Ask HN, Show HN posts)

Each record is validated and null-checked before being pushed to the dataset. Missing or unavailable fields are set to null rather than being omitted, ensuring consistent schema across all records.

Example output

[
    {
        "objectID": "39876543",
        "title": "GPT-5 Released: What We Know So Far",
        "url": "https://example.com/gpt5-release",
        "author": "techwriter",
        "points": 523,
        "numComments": 287,
        "createdAt": "2024-08-15T09:30:00.000Z",
        "storyText": null
    }
]

Pricing

This actor is priced based on usage:

• $0.01 per result returned in the dataset
• $0.005 per actor start (flat fee per run)

These costs cover Apify platform compute and proxy resources. For large-scale scraping jobs, consider using the maxResults parameter to control costs and stay within your budget. Typical runs of 100 results cost approximately $1.01.

Related scrapers

Explore these related actors from brave_paradise:

• 99designs Contest Scraper
• Abuseipdb Check Scraper
• Accuweather Forecast Scraper
• Adoptapet Listing Scraper
• Advice Slip Scraper

More scrapers from brave_paradise

Check out other actors published by brave_paradise on the Apify Store for more data extraction tools covering scientific databases, developer communities, news aggregators, and government open-data APIs. All actors follow the same high-quality patterns with robust error handling, automatic pagination, and clean structured output.