Pricing

from $3.00 / 1,000 results

Lobsters Scraper

Scrape Lobsters (lobste.rs) â€" a curated tech link aggregator. Fetch hottest, newest, or tag-filtered stories with scores, comments, tags, and author info. No authentication required.

Pricing

from $3.00 / 1,000 results

Rating

0.0

(0)

Developer

Crawler Bros

Actor stats

Bookmarked

Total users

Monthly active users

3 days ago

Last modified

Features

Multiple feed modes: hottest stories, newest stories, or tag-filtered browsing
25+ technology tags: programming, python, javascript, security, linux, networking, databases, AI, devops, and more
Full metadata: story title, URL, score, flag count, comment count, tags, submitter username, and timestamps
Pagination support: automatically fetches multiple pages up to your maxItems limit
No authentication required: uses the public lobste.rs JSON API

Use Cases

Monitor trending tech articles and discussions
Track community sentiment on specific technologies
Build developer-focused news aggregators
Research tech topics by community voting patterns
Competitive analysis of developer interest areas

Input

Field	Type	Description	Default
`mode`	Select	Feed type: `hottest`, `newest`, or `byTag`	`hottest`
`tag`	Select	Technology tag to browse (mode=byTag). Options: programming, python, javascript, security, linux, networking, databases, ai, devops, etc.	`programming`
`maxItems`	Integer	Maximum number of stories to return (1–500)	`25`

Output

Each record contains:

Field	Type	Description
`shortId`	String	Unique story ID on Lobsters
`title`	String	Story headline
`url`	String	Link to the original article
`score`	Integer	Upvote count from community
`flags`	Integer	Flag count (moderation signal)
`commentCount`	Integer	Number of discussion comments
`description`	String	Author-provided description (if any)
`tags`	Array	Technology tags applied to the story
`submittedBy`	String	Username of the submitter
`createdAt`	String	ISO 8601 timestamp when posted
`shortIdUrl`	String	Permanent story URL on lobste.rs
`commentsUrl`	String	Direct link to comments thread
`userIsAuthor`	Boolean	Whether submitter is the content author
`scrapedAt`	String	ISO 8601 timestamp of when this record was scraped

Example Output

{
  "shortId": "abc123",
  "title": "Understanding Rust's Ownership Model",
  "url": "https://example.com/rust-ownership",
  "score": 89,
  "flags": 0,
  "commentCount": 14,
  "tags": ["programming", "rust"],
  "submittedBy": "johndoe",
  "createdAt": "2026-06-04T10:00:00.000-05:00",
  "shortIdUrl": "https://lobste.rs/s/abc123",
  "commentsUrl": "https://lobste.rs/s/abc123/understanding_rust_ownership",
  "userIsAuthor": false,
  "scrapedAt": "2026-06-04T15:30:00.000000+00:00"
}

FAQs

Does this require an API key? No. Lobsters provides a public JSON API at /hottest.json, /newest.json, and /t/<tag>.json without any authentication.

How many stories are returned per page? Lobsters always returns exactly 25 stories per page. Set maxItems to a multiple of 25 for best results.

Which tags are available? The dropdown includes 25 curated tags. For the full list, visit https://lobste.rs/tags.

Can I get comments text? This scraper returns story metadata only. Use the commentsUrl field to open the discussion thread.

How fresh is the data? Stories are fetched in real time directly from the Lobsters API. No caching.

Lobsters Scraper

epctex/lobsters-scraper

Scrape Lobste.rs posts and users based on any search criteria. Retrieve all the comments, domains, tags, titles, number of upvotes, and published dates. Use this extremely fast actor to retrieve all the information right away. Easy use and no limits!

epctex

5.0

(7)

Lobsters Scraper | Tech News and Discussions

parseforge/lobsters-scraper

Scrape tech stories from Lobsters including titles, links, tags, scores, comment counts, submitters and timestamps. Build datasets of developer-focused news and discussions for trend analysis, content curation and technology research projects at scale across the full catalog

ParseForge

Lobsters Scraper — Tech Stories & Tags

openclawmara/lobsters-scraper

Scrape Lobsters (lobste.rs) tech news stories and tags. Extract story titles, URLs, scores, comments, authors, and tag categories. Filter by tag, date, or popularity. Perfect for niche tech trend analysis and developer news aggregation.

OpenClaw Mara

Lobsters Scraper - Developer News & Stories Extractor

klondikeking/lobsters-scraper

Extract stories, votes, comments, and metadata from the Lobsters developer news community. Clean JSON API — no proxy needed.

Pierrick McD0nald

Hacker News Comments Scraper — Threads & Discussions

openclawmara/hackernews-comments-scraper

Scrape full comment trees from Hacker News stories. Extracts threaded discussions with author, text, points, depth, timestamp. Fetch by story ID, URL, or top/new stories. Ideal for sentiment analysis, tech discourse research, and developer community insights. Uses official Firebase API.

OpenClaw Mara

Stackoverflow Scraper

openclawmara/stackoverflow-scraper

Scrape Stack Overflow questions, answers, tags, and user profiles. Search by keyword, tag, or date range. Extract vote counts, accepted answers, code snippets, and discussion threads. Ideal for developer knowledge mining and technical research.

OpenClaw Mara

Hacker News Scraper: Stories, Comments, Users & Search

perconey/hackernews-scraper

Scrape Hacker News via the official Firebase API + Algolia search. Top/new/best/ask/show/jobs stories, full comment trees, user profiles with karma, free-text search. No browser, no proxies, no auth. Pay only per result item.

Perconey

TripAdvisor Scraper

epctex/tripadvisor-scraper

Explore with our Trip Advisor Scraper: an easy search for hotels, restaurants, attractions, and more by keywords or start URL. Enter check-in/out dates, and select currency and language. Promote locations, capture details, and retrieve emails and phone numbers if shared.

epctex

836

4.8

(10)

Google News Scraper

automation-lab/google-news-scraper

Extract Google News articles by keyword or topic: headlines, sources, publication dates, snippets, and URLs. Pure HTTP, no browser or API key needed. Export to JSON, CSV, or Excel.