Anthropic News & Research Scraper

Scrape news articles and research papers from Anthropic's website. Get titles, publish dates, categories, descriptions, image URLs, and optionally full article text — no API key required.

What does Anthropic News & Research Scraper do?

Anthropic News & Research Scraper extracts all articles from the Anthropic News and Anthropic Research sections. For each article it collects the title, URL, slug, publish date, categories, description excerpt, hero image URL, and optionally the full article body text.

The scraper uses standard HTTP requests and Cheerio — no browser or JavaScript execution required. It works reliably because Anthropic's website is server-side rendered.

Why scrape Anthropic?

Anthropic is one of the world's leading AI safety and research companies. Their news and research pages publish significant announcements about AI models (Claude), safety research, interpretability findings, and policy work.

Key reasons to track Anthropic content:

• AI model tracking — Monitor new Claude model releases and capability updates
• Research monitoring — Follow Anthropic's safety, interpretability, and societal impact research
• Competitive intelligence — Track AI product launches and strategic partnerships
• News aggregation — Build feeds or newsletters covering the AI industry
• AI training data — Create structured datasets of AI research for RAG pipelines and knowledge bases
• Due diligence — Stay current on AI policy positions and company announcements

Who is it for?

This actor is for anyone who needs structured, machine-readable access to Anthropic's public content:

• AI researchers tracking the latest Claude model capabilities and benchmarks
• Investors monitoring Anthropic's product announcements and funding activity
• Journalists and analysts covering the AI industry
• Newsletter curators building AI-focused digest content
• Data scientists building AI company monitoring dashboards
• Policy researchers studying AI safety and governance publications
• Engineers staying current with AI API and platform changes
• AI/ML teams building training datasets or knowledge bases

How to use Anthropic News & Research Scraper

1. Go to Anthropic News & Research Scraper on Apify Store
2. Choose which content to scrape: news, research, or both
3. Set max results and whether to include full article text
4. Click Start and wait for results
5. Download data as JSON, CSV, or Excel

Input parameters

Parameter	Type	Default	Description
scrapeType	string	"all"	Which section to scrape: all, news, or research
maxResults	integer	100	Maximum number of articles to return
scrapeContent	boolean	false	Fetch the full article body text for each article
maxRequestRetries	integer	3	Retry attempts for failed HTTP requests

Input example

{
  "scrapeType": "all",
  "maxResults": 20,
  "scrapeContent": false
}

To also get the full article text:

{
  "scrapeType": "news",
  "maxResults": 10,
  "scrapeContent": true
}

Output

Each item in the dataset represents one article:

Field	Type	Description
url	string	Full article URL
slug	string	Article slug (URL path segment)
title	string	Article title
categories	array	Content categories (e.g., ["Product", "Announcements"])
publishedDate	string	Publish date as shown on the page (e.g., "Apr 16, 2026")
description	string	Short excerpt or meta description
imageUrl	string	Hero image URL (if available)
type	string	Content type: news or research
content	string	Full article body text (only when scrapeContent: true)

Output example

{
  "url": "https://www.anthropic.com/news/claude-opus-4-7",
  "slug": "claude-opus-4-7",
  "title": "Introducing Claude Opus 4.7",
  "categories": ["Product", "Announcements"],
  "publishedDate": "Apr 16, 2026",
  "description": "Our latest model, Claude Opus 4.7, is now generally available.",
  "imageUrl": "https://cdn.sanity.io/images/4zrzovbb/website/96ea2509a90e527642c822303e56296a07bcfce4-1920x1080.png",
  "type": "news",
  "content": null
}

API usage

You can run this actor programmatically using the Apify API or the official client libraries.

Node.js (ApifyClient)

const { ApifyClient } = require('apify-client');

const client = new ApifyClient({ token: 'YOUR_API_TOKEN' });

const run = await client.actor('automation-lab/anthropic-scraper').call({
  scrapeType: 'all',
  maxResults: 20,
  scrapeContent: false,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python (ApifyClient)

from apify_client import ApifyClient

client = ApifyClient("YOUR_API_TOKEN")

run = client.actor("automation-lab/anthropic-scraper").call(run_input={
    "scrapeType": "all",
    "maxResults": 20,
    "scrapeContent": False,
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

cURL

# Start the actor
curl -X POST "https://api.apify.com/v2/acts/automation-lab~anthropic-scraper/runs?token=YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"scrapeType":"all","maxResults":20,"scrapeContent":false}'

# Get results (replace RUN_ID and DATASET_ID with actual values)
curl "https://api.apify.com/v2/datasets/DATASET_ID/items?token=YOUR_API_TOKEN&format=json"

Use with MCP (Claude Desktop / AI Agents)

This actor is compatible with the Apify MCP Server, enabling direct use from Claude Desktop, Claude Code CLI, Cursor, or any MCP-compatible AI agent.

Claude Code CLI (recommended)

Add only this actor as an MCP tool — no extra actors loaded:

claude mcp add --transport http apify-anthropic-scraper \
  "https://mcp.apify.com?tools=automation-lab/anthropic-scraper" \
  --header "Authorization: Bearer YOUR_API_TOKEN"

Claude Desktop / Cursor (full Apify MCP server)

{
  "mcpServers": {
    "apify": {
      "command": "npx",
      "args": ["-y", "@apify/mcp-server"],
      "env": { "APIFY_TOKEN": "YOUR_API_TOKEN" }
    }
  }
}

Example prompts

Once connected, try these prompts in Claude:

• "Scrape the latest 20 news articles from Anthropic and summarize the top 3 product announcements."
• "Get all Anthropic research papers from the last month and list the ones related to interpretability."
• "Fetch Anthropic news with full article text and identify any mentions of new Claude model releases."

Integrations

Connect Anthropic News & Research Scraper to your existing workflows with Apify's built-in integrations:

Anthropic News → Google Sheets monitoring dashboard

Use the Google Sheets integration to append every new article to a spreadsheet automatically. Set up a scheduled run (daily or weekly) and each new article lands as a new row with title, date, category, and URL — a living dashboard of Anthropic's publications.

Anthropic News → Slack alerts

Pipe new articles to a Slack channel using the Apify → Slack integration. Configure a webhook trigger: whenever the actor finds articles published after your last run, a message is posted to your #ai-news channel with the title and URL. Never miss a Claude model announcement again.

Anthropic Research → RAG knowledge base

Run the actor weekly with scrapeContent: true and feed the full article text into a vector database (Pinecone, Weaviate, or Qdrant) via Apify's dataset webhooks. Build a retrieval-augmented chatbot that answers questions about Anthropic's research using up-to-date content.

Anthropic News → Make (Integromat) / Zapier

Use the Apify → Make integration or Apify → Zapier integration to fan out new articles to any downstream service: CRM notes, Notion databases, newsletter drafts, or custom webhooks.

Tips

• Start small for testing — set maxResults to 10 and scrapeContent: false on your first run. Confirm the data looks correct before scraping everything.
• Use scrapeType: "research" for technical content — the research section contains peer-reviewed and technical papers (interpretability, alignment, safety). Use scrapeType: "news" for product announcements and press releases.
• Enable scrapeContent selectively — fetching full article text roughly doubles the run time and cost. Only enable it when you actually need the body text (e.g., for RAG ingestion or summarization).
• Schedule incremental runs — Anthropic posts a handful of articles per week. A daily or weekly scheduled run with maxResults: 20 is more than enough to stay current without over-fetching.
• Filter by category in post-processing — the categories field lets you narrow results downstream. For example, filter for ["Product"] to watch only model releases, or ["Policy"] for regulatory content.

Pricing

This actor uses pay-per-event pricing:

Tier	Price per article
Free	$0.001
Bronze	$0.00087
Silver	$0.00067
Gold	$0.00052
Platinum	$0.00035
Diamond	$0.00024

Plus a flat $0.005 fee per run (startup cost).

Scraping 20 news articles costs approximately $0.025 on the Free tier.

Legality and terms of use

This actor scrapes publicly available content from Anthropic's website. All scraped content is publicly accessible without login. The actor respects standard HTTP conventions and does not circumvent any access controls.

Users are responsible for ensuring their use of scraped data complies with Anthropic's Terms of Service and applicable laws. This actor is intended for research, monitoring, and informational purposes.

FAQ

Does this require an Anthropic API key? No. The actor scrapes Anthropic's public website — no API key or authentication is needed.

Does it scrape Claude.ai or the Anthropic API? No. It only scrapes the public marketing website (anthropic.com/news and anthropic.com/research). It does not access Claude.ai, the Anthropic API, or any authenticated endpoints.

How often does Anthropic publish new content? Anthropic typically publishes a few news articles and research papers per week. The news listing shows the most recent ~10–15 articles.

Can I get the full article text? Yes — set scrapeContent: true in the input. This makes additional HTTP requests per article and increases cost proportionally.

What categories exist in the research section? Common research categories include: Interpretability, Alignment, Policy, Economic Research, Societal Impacts, and Safety.

Does it handle pagination? The current version scrapes the articles shown on the main listing pages. Anthropic typically shows the most recent 10–15 articles per section.

Why are some articles missing description or imageUrl? Anthropic doesn't always include a description excerpt or hero image for every article — particularly older research papers. When these fields aren't present in the page HTML, the actor returns null for those fields. This is expected behavior, not a bug.

The run succeeded but returned fewer articles than my maxResults setting — why? maxResults is an upper bound. If Anthropic's listing page contains fewer articles than your limit, the actor returns however many are available. This is normal for fresh installations or when the listing page hasn't been updated recently.

The scraper returned zero results or failed with HTTP errors — what should I do? This usually means Anthropic temporarily changed their page structure or the listing page is returning a non-200 status. Try these steps: (1) check the run log for specific error messages, (2) run again — transient errors often resolve in a retry, (3) if failures persist for more than a day, report an issue so the actor can be updated.

Can I scrape specific articles by URL instead of the full listing? The actor currently scrapes the listing pages (/news, /research) and optionally fetches individual article content. Direct URL scraping of arbitrary articles is not supported in the current version.

Related actors

• Hugging Face Papers Scraper — Scrape AI research papers from HuggingFace
• arXiv Scraper — Scrape papers from arXiv
• TechCrunch Scraper — Scrape tech news from TechCrunch
• Hacker News Scraper — Scrape stories from Hacker News