Lobsters Tech News & Discussion Scraper 🦞

Extract trending technical discussions, high-signal engineering blog posts, security advisories, and community votes from Lobsters. Built for software developers, technology researchers, tech lead intelligence, and AI Model Context Protocol (MCP) workflows.

---

Key Features

• High-Signal Tech Curation: Filter out marketing noise; Lobsters focuses purely on software engineering, distributed systems, architecture, security, and low-level computing.
• Multiple Feeds: Instant access to Hottest (front page trending), Newest (fresh submissions), and Tagged (topic-focused archives).
• Topic / Tag Filtering: Search across hundreds of tags including rust, python, security, ai, linux, devops, databases, web, and hardware.
• Score & Engagement Thresholds: Filter out noise by specifying a minimum upvote score (minScore).
• Complete Structured Fields: Direct article links, discussion threads, submitter usernames, tags, comment counts, and ISO-8601 timestamps.
• MCP & AI Agent Ready: Clean JSON output with standardized properties ready for LLM consumption, daily digest bots, and market intelligence.

---

Input Parameters

Parameter	Type	Default	Description
feed	String (select)	"hottest"	Choose feed: hottest, newest, or tagged.
tag	String	"security"	Tag to filter by when feed is set to tagged (e.g. rust, ai, security).
maxResults	Integer	25	Maximum number of stories to extract (1 to 200).
minScore	Integer	0	Minimum upvote score to filter low-engagement items (0 disables filter).

---

Output Dataset Schema

Each item in the dataset includes:

{
  "shortId": "ttp7aq",
  "title": "Backdoors in Claude Code for VS, Cursor official extensions",
  "score": 145,
  "commentCount": 38,
  "submitter": "dsr",
  "tags": [
    "security"
  ],
  "tagsString": "security",
  "url": "https://cyberplace.social/@GossiTheDog/117528831681803563",
  "commentsUrl": "https://lobste.rs/s/ttp7aq/backdoors_claude_code_for_vs_cursor",
  "createdAt": "2026-10-30T06:04:55.168-05:00",
  "description": ""
}

---

Use Cases

1. Tech News Aggregation: Automate daily developer newsletters or Slack / Discord engineering digest channels.
2. AI & MCP Agent Memory: Feed fresh technology trends and vulnerability reports into AI agents for real-time developer assistance.
3. Security Vulnerability Monitoring: Track zero-day reports, vulnerability disclosures, and critical infrastructure discussions.
4. Talent & Community Outreach: Discover active engineering practitioners and niche tech contributors.

---

Pricing

Simple Pay-Per-Event pricing with zero hidden computation fees:

• Actor Start: $0.00008
• Per Result Item: $0.0034 (equivalent to $3.40 per 1,000 items)
• Platform Usage: $0.00 (Platform usage fee OFF)