⛓️ Web3 & Decentralized Tech Intelligence Pipeline

A professional-grade Crypto News Scraper and AI Market Analyst. Automatically fetch, verify, and classify Web3 news for trading signals, technical analysis, and market impact.

---

📢 What is the Web3 Intelligence Pipeline?

The Web3 & Decentralized Tech Intelligence Pipeline is an advanced AI-powered tool designed to automate the consumption of cryptocurrency news. Unlike simple RSS readers, this Actor acts as a smart financial & technical analyst.

It aggregates raw news from over 20+ top-tier sources (CoinDesk, The Defiant, AWS Web3, etc.), scrapes the full article content, and uses OpenAI (GPT-4o-mini) to classify and analyze the data based on its type.

🚀 Key Features

• 🧠 Intelligent Classification: Automatically distinguishes between Market News (Price/Regulation) and Developer Content (Tutorials/Tech Stacks).
• 🌍 Multi-Source Aggregation: Ingests data from curated high-quality feeds or custom URLs.
• 🕷️ Full-Text Scraping: Visits the actual article URLs to extract the full body text (Markdown), ensuring the AI analyzes the complete story.
• 🦁 Brave Search Grounding: Uses the Brave Search API to find corroborating sources and extra context, reducing hallucinations.
• 📊 Conditional Analysis:
  • For Investors: Generates Sentiment, Trading Signals (Long/Short), and Key Entities.
  • For Developers: Extracts Tech Stack (e.g., Rust, Solidity) and Technical Complexity (Low/High).
• 🔔 Discord Integration: Automatically sends color-coded alerts (Green for Long, Purple for Tech Deep Dives) to your Discord server.

---

🤖 Why use this Web3 News Scraper?

Traders need signals; Developers need documentation. This Actor separates the signal from the noise.

Common Use Cases

• Algorithmic Trading: Feed trading_signal directly into your bot (only for "Market News" items).
• CTO/Dev Recruiting: Track "High Complexity" technical articles to find emerging tech stacks (e.g., "Zero-Knowledge Proofs").
• Market Research: Aggregate trends across DeFi, NFTs, and Regulation without visiting 20 different websites.

---

💸 Pricing & Costs

This Actor utilizes the Pay-Per-Event (PPE) monetization model to ensure you only pay for the value you receive.

How it works

You are charged a small fee per analyzed article.

• Event: analyzed-article
• Trigger: The charge occurs only when the Actor successfully fetches, scrapes, analyzes, and pushes a finished record to the dataset.
• Skipped items: You are not charged for duplicate articles or failed scrapes.

External API Costs (BYOK)

This Actor follows a Bring Your Own Key (BYOK) model for privacy and control. You will need:

1. OpenAI API Key: To power the LLM analysis (GPT-4o-mini is highly affordable).
2. Brave Search API Key: (Optional) For context grounding. The Free Tier (2,000 queries/month) is sufficient for most users.

---

📋 Input Configuration

The Actor is highly configurable. Here are the key settings:

Setting	Description
News Source	Choose from presets (CoinDesk, Cointelegraph, AWS, etc.) or "All".
OpenAI API Key	Required. Your key for generating intelligence.
Brave API Key	Optional. Improves accuracy by cross-referencing facts.
Discord Webhook	Optional. Paste a webhook URL to receive real-time alerts.
Country Context	Bias the search grounding (e.g., set to 'ZA' for South Africa).

---

📦 Output Data Schema

The Actor stores results in the default Apify Dataset. The output fields change dynamically based on the Content Type.

Example 1: Market News (Trading Focus)

{
  "content_type": "Market News",
  "title": "SEC Approves Bitcoin ETF Options",
  "sentiment": "Bullish Trend",
  "trading_signal": "LONG",
  "category": "Regulation",
  "subcategory": "ETF Products",
  "technical_complexity": null,
  "tech_stack": null
}

Example 2: Developer Tutorial (Tech Focus)

{
  "content_type": "Developer/Tech",
  "title": "Building a ZK-Rollup with Circom and SnarkJS",
  "sentiment": null,
  "trading_signal": "N/A",
  "category": "Blockchain Infrastructure",
  "subcategory": "Zero-Knowledge Proofs",
  "technical_complexity": "High",
  "tech_stack": ["Circom", "SnarkJS", "Solidity", "Rust"]
}

---

🎓 How to Scrape Crypto News (Tutorial)

1. Get your Keys: Obtain an API Key from OpenAI and Brave Search.
2. Configure the Actor:
   • Paste your keys into the input fields.
   • Select Source: All for broad coverage.
3. Set Alerts (Optional): Create a Webhook in your Discord channel settings and paste it into the Actor configuration.
4. Run: Click Start. The Actor will scrape content, classify it (News vs. Tech), and produce your intelligence report.

---

⚖️ Legal & Disclaimer

Not Financial Advice: The trading_signal and sentiment fields are generated by an AI model based on news text. They should not be used as the sole basis for financial decisions.

Scraping Ethics: This Actor respects robots.txt where possible and is designed to scrape news for personal and analytical use. It does not extract private user data.

---

💬 Support & Feedback

If you encounter issues or have feature requests, please use the Issues tab on the Actor page.

• Bugs: Report any crashes or failed URL parsings.
• Features: Request new output fields or integrations.