# Lobsters Tech News & Discussion Scraper (`authoritative_glen/lobsters-tech-news-trending-radar`) Actor

Scrape trending technical stories, engineering discussions, and community tags from Lobsters. Fast, clean structured data, zero login required.

- **URL**: https://apify.com/authoritative_glen/lobsters-tech-news-trending-radar.md
- **Developed by:** [Rahmad Rizki](https://apify.com/authoritative_glen) (community)
- **Categories:** Developer tools, Business
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Lobsters Tech News & Discussion Scraper 🦞

Extract trending technical discussions, high-signal engineering blog posts, security advisories, and community votes from [Lobsters](https://lobste.rs). Built for software developers, technology researchers, tech lead intelligence, and AI Model Context Protocol (MCP) workflows.

***

### Key Features

- **High-Signal Tech Curation:** Filter out marketing noise; Lobsters focuses purely on software engineering, distributed systems, architecture, security, and low-level computing.
- **Multiple Feeds:** Instant access to `Hottest` (front page trending), `Newest` (fresh submissions), and `Tagged` (topic-focused archives).
- **Topic / Tag Filtering:** Search across hundreds of tags including `rust`, `python`, `security`, `ai`, `linux`, `devops`, `databases`, `web`, and `hardware`.
- **Score & Engagement Thresholds:** Filter out noise by specifying a minimum upvote score (`minScore`).
- **Complete Structured Fields:** Direct article links, discussion threads, submitter usernames, tags, comment counts, and ISO-8601 timestamps.
- **MCP & AI Agent Ready:** Clean JSON output with standardized properties ready for LLM consumption, daily digest bots, and market intelligence.

***

### Input Parameters

| Parameter | Type | Default | Description |
|---|---|---|---|
| `feed` | String (select) | `"hottest"` | Choose feed: `hottest`, `newest`, or `tagged`. |
| `tag` | String | `"security"` | Tag to filter by when `feed` is set to `tagged` (e.g. `rust`, `ai`, `security`). |
| `maxResults` | Integer | `25` | Maximum number of stories to extract (1 to 200). |
| `minScore` | Integer | `0` | Minimum upvote score to filter low-engagement items (0 disables filter). |

***

### Output Dataset Schema

Each item in the dataset includes:

```json
{
  "shortId": "ttp7aq",
  "title": "Backdoors in Claude Code for VS, Cursor official extensions",
  "score": 145,
  "commentCount": 38,
  "submitter": "dsr",
  "tags": [
    "security"
  ],
  "tagsString": "security",
  "url": "https://cyberplace.social/@GossiTheDog/117528831681803563",
  "commentsUrl": "https://lobste.rs/s/ttp7aq/backdoors_claude_code_for_vs_cursor",
  "createdAt": "2026-10-30T06:04:55.168-05:00",
  "description": ""
}
```

***

### Use Cases

1. **Tech News Aggregation:** Automate daily developer newsletters or Slack / Discord engineering digest channels.
2. **AI & MCP Agent Memory:** Feed fresh technology trends and vulnerability reports into AI agents for real-time developer assistance.
3. **Security Vulnerability Monitoring:** Track zero-day reports, vulnerability disclosures, and critical infrastructure discussions.
4. **Talent & Community Outreach:** Discover active engineering practitioners and niche tech contributors.

***

### Pricing

Simple Pay-Per-Event pricing with zero hidden computation fees:

- **Actor Start:** $0.00008
- **Per Result Item:** $0.0034 (equivalent to $3.40 per 1,000 items)
- **Platform Usage:** $0.00 (Platform usage fee OFF)

# Actor input Schema

## `feed` (type: `string`):

Select the Lobsters feed or category to scrape.

## `tag` (type: `string`):

Tag to filter by when Feed is set to 'Tagged' (e.g., 'security', 'rust', 'ai', 'python', 'linux', 'devops', 'web'). Ignored if feed is 'hottest' or 'newest'.

## `maxResults` (type: `integer`):

Maximum number of stories to extract (1 to 200).

## `minScore` (type: `integer`):

Filter out stories with score below this threshold (0 disables filter).

## Actor input object example

```json
{
  "feed": "hottest",
  "tag": "security",
  "maxResults": 25,
  "minScore": 0
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("authoritative_glen/lobsters-tech-news-trending-radar").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("authoritative_glen/lobsters-tech-news-trending-radar").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call authoritative_glen/lobsters-tech-news-trending-radar --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,authoritative_glen/lobsters-tech-news-trending-radar"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/cPHLi0wkVjNieImp9/builds/wegJn61bSbrfysa7t/openapi.json