⚡ AI Agent Product Intelligence & Token-Optimized Web Cleaner

A dual-purpose, high-performance toolkit designed specifically for AI Agents, LLM pipelines, and automated market research.

Empower your Autonomous Agents (LangChain, AutoGPT, CrewAI, Claude Desktop, Cursor) with structured product feedback and ultra-compact web data.

---

🌟 Core Capabilities

1. Product Mentions & Customer Complaints Intelligence

• Autonomous Market & Feedback Mining: Scrapes and aggregates live discussions from multiple verified sources (Reddit, Hacker News, Tech Publications, GitHub Discussions, Google News).
• AI-Ready Sentiment & Urgency Breakdown: Delivers an overall sentiment score (-1.0 to +1.0) along with positive/negative ratios.
• Categorized Complaints & Real Quotes: Extracts concrete pain points with verified user quotes across categories (Performance, Pricing, UX/UI, Features, Reliability).
• Anti-Hallucination & Zero False Positives: Built-in negation handling and context disambiguation (e.g. distinguishing praises like "zero bugs" from negative complaints).

2. Token-Optimized Web Cleaner

• 70%–90% Token Reduction: Strips out cookie banners, navigation menus, ads, headers, footers, tracking scripts, and HTML clutter.
• Table & Structure Preservation: Retains tables formatted cleanly in Markdown, preserves relevant hyperlinked context [text](url), and tags code blocks with syntax highlighting.
• SSRF & Security Protected: Built-in IP filtering preventing SSRF against private subnets, streaming size limits, and safe parsing.

---

🚀 How to Run

Example 1: Extract Product Complaints & Mentions (Default)

{
  "action": "mentions",
  "product": "Cursor IDE",
  "timeframe": "90d",
  "max_results": 25
}

Example 2: Clean a Web Page for AI Agent Context

{
  "action": "clean_url",
  "url": "https://en.wikipedia.org/wiki/Artificial_intelligence",
  "preserve_links": true,
  "max_tokens": 4000
}

---

📊 Output Schema

The actor produces structured JSON saved directly to Apify Dataset:

{
  "product": "Linear",
  "status": "ok",
  "timeframe": "90d",
  "total_mentions_scanned": 21,
  "sources_breakdown": {
    "Hacker News": 5,
    "Reddit": 9,
    "Tech Press": 7
  },
  "sentiment": {
    "score": 0.03,
    "label": "Mixed / Neutral",
    "positive_ratio": "14%",
    "negative_ratio": "5%",
    "neutral_ratio": "81%"
  },
  "top_complaints": [
    {
      "category": "Performance & Stability",
      "frequency": 2,
      "urgency": "medium",
      "quotes": [
        "linear feels slightly heavier after recent updates"
      ]
    }
  ]
}

---

💳 Pricing (Pay-Per-Event)

• Pay-Per-Event: $0.005 per successful result ($5.00 per 1,000 queries).
• Only charged when data is successfully extracted.

---

🤖 Integrate with Python

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run_input = {
    "action": "mentions",
    "product": "Notion",
    "timeframe": "30d"
}

run = client.actor("maybad/ai-agent-product-intelligence").call(run_input=run_input)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)