Lobste.rs Scraper

Scrape Lobste.rs tech news aggregator - stories, comments, and metadata. Extract titles, URLs, scores, tags, comment counts, submitter info, and timestamps from this niche but active tech community. Perfect complement to Hacker News scraping for comprehensive tech news coverage.

This actor uses CheerioCrawler for fast HTTP-based scraping, supports multiple modes (hot, newest, comments), tag filtering, score filtering, and automatic pagination. No API key or authentication required.

Features

• Scrape hot stories, newest submissions, or recent comments from Lobste.rs
• Filter stories by tags (e.g., 'programming', 'rust', 'linux')
• Filter by minimum score threshold
• Sort results by hot (default algorithm), newest (chronological), or top (by score)
• Extracts story ID, title, source URL, Lobste.rs URL, score, comment count, submitter, tags, timestamp, and description
• Auto-detects Hacker News links for cross-platform tracking
• Automatic pagination to collect more items
• No API key, authentication, or login required

Input

Configure the actor through the input schema. All fields are optional with sensible defaults.

Field	Type	Default	Description
mode	String	main	Scraping mode: main (hot stories), newest (newest submissions), or comments (recent comments)
maxItems	Integer	30	Maximum number of stories/comments to scrape
sortBy	String	hot	Sort order: hot (algorithmic), newest (chronological), or top (by score)
tags	Array	[]	Filter by tags (e.g., ['programming', 'rust']). Leave empty for all tags
minScore	Integer	0	Only return stories with at least this many upvotes
includeTags	Boolean	true	Include tag information in the output

Example input

{
    "mode": "main",
    "maxItems": 50,
    "sortBy": "hot",
    "tags": ["programming", "rust"],
    "minScore": 5,
    "includeTags": true
}

Output

Results are stored in the default dataset and can be exported as JSON, CSV, Excel, or via API.

Story output format

{
    "story_id": "abc123",
    "title": "Rust for Embedded Systems",
    "url": "https://example.com/rust-embedded",
    "lobste_rs_url": "https://lobste.rs/s/abc123",
    "score": 42,
    "comment_count": 15,
    "submitter": "rustacean",
    "tags": ["programming", "rust", "embedded"],
    "submitted_at": "2026-07-04T10:23:00Z",
    "is_hn_link": false,
    "description": "An introduction to using Rust..."
}

Output fields

Field	Description
story_id	Unique story identifier from Lobste.rs
title	Story title
url	Original source URL
lobste_rs_url	Lobste.rs discussion URL
score	Number of upvotes
comment_count	Number of comments
submitter	Username of the submitter
tags	Array of tags associated with the story
submitted_at	ISO 8601 timestamp when submitted
is_hn_link	Boolean indicating if story links to Hacker News
description	Story description or excerpt (if available)

Use cases

• Tech news aggregation — Combine with Hacker News scraper for comprehensive tech news coverage
• Community monitoring — Track discussions in the Rust, Linux, and broader developer communities
• Content discovery — Find high-quality technical articles and blog posts
• Trend analysis — Identify emerging trends in programming languages and tools
• Competitive intelligence — Monitor competitor launches and community reactions
• Lead generation — Identify active developers and companies in niche tech communities
• Tag-based filtering — Focus on specific topics like 'programming', 'rust', 'linux', 'databases'

How to use

1. Open the actor on the Apify Store.
2. Click Try for free or Start with Free Trial.
3. Adjust the input fields (mode, max items, tags, sort order, min score) — or leave defaults.
4. Click Start and wait for the run to complete.
5. Export the results to JSON, CSV, or connect via the Apify API.

For scheduled runs, use Apify's scheduler to scrape Lobste.rs daily or weekly and auto-export to your webhook, database, or Google Sheets.

Tech News Bundle

Pair this actor with the Hacker News Scraper for a complete tech news bundle:

• Hacker News Scraper — Mainstream tech news, startups, and Silicon Valley discussions
• Lobste.rs Scraper — Niche technical discussions, programming language deep-dives, and hands-on engineering content

Run both in parallel via Apify's task scheduler or webhook integrations for comprehensive coverage.

About Lobste.rs

Lobste.rs is a tech-focused link aggregator similar to Hacker News but with a more technical and hands-on focus. It's particularly strong in discussions about programming languages, systems programming, and practical engineering topics. The community is smaller (3.9M+ runs on existing scrapers) but highly engaged and technically sophisticated.