⛓️ Web3 & Crypto News Intelligence Pipeline

Turn raw RSS feeds into structured AI intelligence. Scrape, summarize, and analyze Web3 news without configuring external API keys.

---

What is the Web3 News Intelligence Pipeline?

This Actor is a specialized Crypto Market API and Web3 Scraper designed for traders, analysts, and developers. Unlike standard RSS readers that only give you a headline and a link, this tool uses a sophisticated AI Pipeline to read and understand the market for you.

It automatically fetches news from top sources (CoinDesk, Cointelegraph, Ethereum Foundation), attempts to scrape the full content, and uses a Large Language Model (LLM) to generate structured intelligence: sentiment analysis, risk ratings, categorical tagging, and executive summaries.

Why use this Actor?

• 🔋 Batteries Included: You do not need your own OpenAI or Brave Search API keys. We handle the infrastructure; you just pay for the results.
• 🧠 AI-Powered Analysis: Every article is processed by an LLM to extract "High Risk" events, identify "DeFi Hacks" vs. "Protocol Upgrades," and summarize the core facts.
• 🛡️ Resilient "Scrape-First" Engine: The actor first attempts to scrape the article directly (free). If blocked, it automatically falls back to a paid Brave Search snippet extraction to ensure you almost always get context for the AI.
• 📊 Structured JSON: Get clean data ready for trading bots, dashboards, or automated newsletters.

What data can this Actor extract?

This tool transforms unstructured web content into a structured dataset. Here is the data model you can expect:

Field	Description
Title	The original headline of the news article.
AI Summary	A dense, 2-sentence executive summary of the event.
Risk Level	The "Impact" level (e.g., High Risk, Medium Risk, Low/Informational).
Category	The thematic tag (e.g., DeFi Hack, Regulation, Protocol Upgrade, Market Analysis).
Key Entities	A list of specific tokens (e.g., ETH), protocols (e.g., Uniswap), or influencers involved.
Source Feed	The origin of the news (e.g., CoinDesk, The Defiant).
Extraction Method	Indicates if the data came from direct scraping or search fallback.

How to use the Web3 News Pipeline

This Actor is designed to be "plug and play."

1. Select a Source: Choose a preset (e.g., coindesk, ethereum-foundation) or select all to aggregate from major crypto outlets. You can also provide a custom RSS feed URL.
2. Set Volume: Define the maxArticles you want to analyze (e.g., 10 recent articles) to control costs.
3. Run: Click Start. The Actor will handle the scraping, fallback searching, and AI analysis automatically.

Tip: Enable Run Test Mode in the input settings to generate dummy data. This allows you to test your integration pipelines without incurring any costs.

Pricing

How much does it cost to analyze crypto news?

This Actor uses the Pay-per-event pricing model. This ensures you only pay when the AI successfully generates intelligence.

• Usage Cost: You pay a small fee per successful AI Article Analysis (article-summary event).
• Platform Cost: Standard Apify platform usage fees apply for the compute time (duration of the run).

Why this model? We pay for the underlying LLM (OpenRouter) and the Search API (Brave) so you don't have to manage subscriptions. The Pay-per-event fee covers these API costs. You are not charged the event fee for articles that fail to be analyzed.

Input Configuration

The input is simple and requires no API keys.

• News Source: Select from a dropdown of top Web3 sites or provide a custom URL.
• Max Articles: Limit the number of items to process per run.
• Region/Time: (Optional) Filter results if the actor falls back to search (e.g., ensure results are from the "Past Week").
• Test Mode: Toggle this to True for zero-cost integration testing.

Output Example

The Actor stores results in the default Apify Dataset. You can export this as JSON, CSV, or Excel.

{
  "title": "SEC Approves Bitcoin ETF Options",
  "source_feed": "CoinDesk",
  "published": "2025-10-15T14:30:00Z",
  "risk_level": "Medium Risk",
  "category": "Regulation",
  "key_entities": [
    "SEC",
    "Bitcoin ETF",
    "BlackRock"
  ],
  "ai_summary": "The SEC has officially approved options trading for spot Bitcoin ETFs, a move expected to increase institutional liquidity. Market analysts predict this will reduce volatility over the long term.",
  "method": "scraped",
  "url": "https://coindesk.com/policy/sec-approves..."
}

FAQ & Troubleshooting

Is scraping these news sites legal? Our scrapers collect data from publicly available RSS feeds and news articles. We do not extract private user data. However, you should ensure your use of the data complies with copyright laws and the Terms of Service of the target websites.

Why did some articles return "Method: search_fallback"? Some crypto websites have strict anti-scraping protections (Cloudflare, etc.). When our direct scraper is blocked, the Actor automatically queries the Brave Search API to find snippets and summaries of the article from across the web. This ensures you still get an AI analysis even if the direct link was inaccessible.

I am getting a "Maintenance" warning or empty results? Check if you have runTestMode enabled. If you are running this locally or via API, ensure you are respecting the schema. If the issue persists, please open an Issue in the Console tab.

Advanced: The Intelligence Pipeline

For developers integrating this into a larger system (e.g., a trading bot or discord alert system), it helps to understand the flow:

1. Ingest: Fetch N items from the RSS feed.
2. Deduplicate: Check against previous runs to avoid analyzing the same story twice (saving you money).
3. Scrape (Tier 1): Attempt fast, direct HTML extraction.
4. Fallback (Tier 2): If Tier 1 fails (403/429), perform a semantic search for the headline using Brave Search.
5. Synthesize: Feed the best available text context to the LLM for structured JSON extraction.