🤖 AI News Intelligence Pipeline

This actor provides a powerful, automated pipeline to transform raw cybersecurity news from top RSS feeds into structured, actionable threat intelligence. It uses the Google Programmable Search API to gather real-time context and an LLM (OpenAI) to perform advanced analysis, ensuring your data is both current and insightful.

---

Features

• Comprehensive Source Aggregation: Gathers news from a curated list of top-tier cybersecurity RSS feeds.
• Real-Time Grounding: Uses the Google Programmable Search API to enrich each article with up-to-the-minute context before analysis, preventing outdated results.
• Advanced AI Analysis: Leverages a Large Language Model to analyze each article for sentiment, categorize the threat type (e.g., Malware, Data Breach), and extract key entities (e.g., threat groups, CVEs, companies).
• Duplicate Prevention: Intelligently tracks processed articles across runs to ensure you only process and pay for new information.
• Cost-Saving Test Mode: Includes a test mode to run the full workflow with dummy data, allowing for development and testing without incurring API costs.

---

Setup and Configuration

Before running the actor, you need to provide API keys for the services it relies on.

1. Google Programmable Search API:
   • You will need a Google API Key and a Search Engine ID.
   • Follow the instructions here to get your credentials.
2. OpenAI API Key:
   • You will need an API key from your OpenAI account.

Add Keys to Apify Secrets

For security, add these keys as secret environment variables in your Apify Actor settings:

• GOOGLE_API_KEY: Your Google API Key.
• GOOGLE_CSE_ID: Your Programmable Search Engine ID.
• OPENAI_API_KEY: Your OpenAI API Key.

---

Cost of Usage 💸

This actor incurs costs from three sources:

1. Apify Platform Usage: Standard platform costs for running the actor, which depends on the duration of the run.
2. Google Programmable Search API: The actor makes one search query for every article it processes. Google provides a free tier of 100 queries/day, after which it costs approximately $5 per 1,000 queries.
3. OpenAI API: This is the primary cost. The actor makes two LLM calls for every article: one for analysis and one for summarization. The cost depends on the model you use and the number of tokens processed.

---

Input

Field	Type	Default	Description
source	String	all	The cybersecurity news source category to use.
customFeedUrl	String	null	A custom RSS feed URL to use if source is set to custom.
maxArticles	Integer	20	The maximum number of new articles to fetch and process in a single run.
useSummarization	Boolean	true	If enabled, the actor will generate an AI summary for each article, incurring an additional LLM cost.
runTestMode	Boolean	false	Bypasses all external API calls for zero-cost testing. Do not enable in production.

---

Output

The actor saves its results in the dataset. Each item is a structured JSON object with the following fields:

Field	Type	Description
source	String	The name of the news source (e.g., 'The Hacker News').
title	String	The original title of the news article.
url	String	The URL of the original article.
published	String	The publication date string from the RSS feed.
summary	String	The AI-generated summary of the article.
sentiment	String	The AI-analyzed sentiment (e.g., High Risk, Medium Risk).
category	String	The AI-assigned category (e.g., 'Malware/Ransomware', 'Data Breach/Hack').
key_entities	Array of Strings	A list of key entities like companies, threat actors, or CVEs mentioned.