🦞 Lobste.rs Tech News Scraper

🚀 Export curated tech stories from Lobste.rs in seconds. Get titles, scores, tags, authors, and discussion links. no account, no auth, no rate limits.

Lobste.rs is an invite-only, curated tech community where experts share and discuss links across programming, security, systems, web development, and more. Unlike mainstream aggregators, every submitter is accountable. real usernames, real reputations. This scraper taps Lobste.rs's fully public JSON API to export stories in real time, with zero authentication or proxies required.

Whether you're tracking technology trends, monitoring specific tags like security or python, or building a developer-focused news feed, this actor delivers clean, structured data in seconds.

Coverage: Thousands of curated tech stories across 50+ community-maintained tags, covering programming languages, operating systems, web, security, mathematics, hardware, and more. Every record includes the story's score, flag count, comment count, author username, tags, and optionally the full description for self-posts.

Target Audience / Use Cases

Audience	Use Case
Researchers & Analysts	Track technology trends and community sentiment over time
Content Teams	Monitor developer community discussions for content ideas
Data Scientists	Build datasets of high-quality, expert-curated tech links
News Aggregators	Power developer-focused newsletters or Slack digests
Security Teams	Watch the security tag for emerging vulnerability discussions
Educators	Find the most-discussed topics in specific programming domains

📋 What the Lobste.rs Scraper does

• Fetches stories from the hottest feed (ranked by community score) or newest feed (chronological)
• Filters by any Lobste.rs tag: programming, javascript, python, security, linux, web, ask, show, and 50+ more
• Paginates automatically to collect as many stories as you need (up to 1,000,000)
• Returns structured records with title, URL, score, flags, comment count, tags, author, and timestamps
• Captures full description text for self-posts (ask/show posts written directly on Lobste.rs)
• Uses a polite 500ms delay between pages to respect the community server

💡 Why it matters: Lobste.rs is one of the most signal-rich tech communities on the internet. Unlike Hacker News, every member is vouched by a real person. The stories that trend here represent genuine expert consensus. not viral noise.

📊 Data fields

Each record includes: commentCount, createdAt, flags, lobstersUrl, results, score, scrapedAt, submittedBy, tags, title, url. These field names come straight from the actor's dataset schema, so what you see here is what lands in your dataset.

⚠️ Good to Know: The tag field accepts any tag slug that Lobste.rs uses in their URLs (e.g. https://lobste.rs/t/security). Common tags include: programming, javascript, python, security, linux, web, ask, show, go, rust, c, devops, math, hardware, browsers, practices. Self-posts (ask/show) include description text; link posts have description: null.

🚀 How to use

1. Create a free Apify account (includes $5 free credit)
2. Go to the Lobste.rs Tech News Scraper page
3. Click Try for free
4. Set your feedType (hottest or newest), optional tag, and maxItems
5. Click Start and wait a few seconds
6. Download your dataset as JSON, CSV, Excel, or XML

🔗 Recommended Actors

Actor	Description
Hacker News Scraper	Export stories and comments from Hacker News
GitHub Trending Scraper	Track trending repositories by language and time period
Dev.to Article Scraper	Export developer articles from Dev.to by tag or author

💡 Pro Tip: browse the complete ParseForge collection for 50+ specialized data extractors covering developer tools, tech communities, job boards, and more.

This actor is not affiliated with or endorsed by Lobste.rs. It uses only publicly available data from the official Lobste.rs JSON API. Use responsibly and in accordance with Lobste.rs's terms of service.

🆘 Need Help?

If you hit a bug, have questions about setup, or need a scraper we haven't built yet, open our contact form or write to parseforge@protonmail.com. We also take on paid custom data projects.

For faster answers, join our Discord. It's the best place to get support and suggest new actors.