Actor – Hacker News Design Scraper

Scrape Y Combinator's Hacker News front page and extract only design-oriented posts. The Actor collects:

• title
• upvotes (points)
• rank
• source URL

This tool is ideal for tracking UX/UI and design-related discussions on Hacker News.

Features

• Scrape the Hacker News homepage (news) quickly via HTTP (Cheerio)
• Filter posts by configurable design-related keywords
• Store results in the default Apify dataset

Input Parameters

The input is a JSON object with the following fields:

• startUrl (Optional, String)
  • Hacker News page to scrape. Defaults to https://news.ycombinator.com/.
• keywords (Optional, Array of String)
  • Keywords used to match design-related posts in titles. Defaults include: design, ux, ui, usability, accessibility, typography, interaction, product design, interface, figma, sketch, prototype.

Input Example

{
  "startUrl": "https://news.ycombinator.com/",
  "keywords": [
    "design",
    "ux",
    "ui",
    "usability",
    "accessibility"
  ]
}

During the Run

• The Actor loads the provided startUrl (default: HN front page)
• It parses the list of posts (tr.athing rows)
• For each post, it extracts title, rank, and upvotes (from the adjacent subtext row)
• It filters items whose titles include any of the provided keywords
• Matching items are stored to the default dataset

If incorrect input is provided, the Actor will fail with a descriptive message.

Output

Each dataset item has the following structure:

{
  "title": "Your vs. My in user interfaces",
  "upvotes": 166,
  "rank": 2,
  "url": "https://adamsilver.io/blog/your-vs-my-in-user-interfaces/",
  "source": "https://news.ycombinator.com/"
}

Run Locally

Use Node 18+ (recommended with nvm):

nvm install 18
nvm use 18
npm install
node src/main.js

Local storage paths used by the Actor:

• Input: storage/key_value_stores/default/INPUT.json
• Output dataset: storage/datasets/default

Docker

docker build -f .actor/Dockerfile -t hn-design-scraper .
mkdir -p storage
docker run --rm -v "$PWD/storage:/usr/src/app/storage" hn-design-scraper

Apify CLI (optional)

npm i -g apify-cli
apify login
apify run

Deploy to Apify Platform

apify push --no-prompt
apify call

Notes

• This Actor focuses on the front page and performs keyword-based filtering for design topics.
• If you need historical pages, Show HN, Ask HN, or comments hierarchy, consider adapting the code or using a more feature-complete HN Actor.
• Keep nodejsVersion at 18 in .actor/actor.json for compatibility on the Apify platform.

Reference

• Inspired by the structure of: https://apify.com/epctex/hackernews-scraper