🚀 Market Gap Analyzer: AI Strategic Intelligence Agent

The Market Gap Analyzer is a high-performance AI Agent built on the Apify platform. It doesn't just scrape data; it investigates your competition. By combining real-time web crawling with "Negative Sentiment Mining" on Reddit and Trustpilot, it identifies exactly where your competitors are failing and tells you what to build next.

---

🧠 The Agentic Philosophy

Unlike standard scrapers that return raw JSON, this agent uses a ToolLoopAgent (Vercel AI SDK) to reason through the data. It autonomously:

1. Audits your product to establish a baseline.
2. Generates aggressive search queries (e.g., "why I stopped using [Competitor]") to find high-churn discussions.
3. Outsources heavy lifting to specialized Apify Actors for Reddit and Trustpilot to bypass anti-bot walls.
4. Prunes and Ranks results to ensure only the most impactful community feedback is processed by the LLM.

---

✨ Key Features

• Negative Sentiment Mining: Specifically targets "Real Talk" on Reddit to find complaints that don't appear in polished marketing materials.
• Hybrid Scraping Logic: Uses a stealth PlaywrightCrawler for landing pages and specialized Sub-Actors for high-security sites (G2/Trustpilot).
• Context-Aware Pruning: Automatically filters and sorts hundreds of reviews by upvotes/impact to prevent context_length_exceeded errors while maintaining high-quality output.
• Actionable Roadmaps: Outputs a prioritized P0/P1/P2 feature list and a "Sales Kill-Shot" to help you win more deals.

---

📥 Input Configuration

Field	Type	Description
myProductUrl	String	Your SaaS/Product landing page or review profile.
competitorUrls	Array	List of competitor domains or review pages.
modelProvider	String	openai or google.

Here are the added sections for your README.md. These provide the technical transparency users need for the output and the "secret sauce" tips to get the highest ROI from the agent.

---

📤 Output Configuration

The agent stores its findings in the Apify Dataset. Each run generates a structured JSON object containing the strategic report. You can export this as a JSON, CSV, or Excel file, or view the formatted Markdown directly in the Apify UI.

Field	Type	Description
marketIntelligenceReport	String (Markdown)	The primary output. Includes Internal Audit, Competitor Comparison, Gaps, P0-P2 Roadmap, and Sales Kill-Shot.
timestamp	String	ISO 8601 timestamp of when the analysis was performed.
sources	Array	List of platforms successfully mined (e.g., ["Reddit", "Trustpilot", "Web"]).

---

💡 Pro-Tips for Maximum Intelligence

To get the most out of the Market Gap Analyzer, follow these strategic guidelines:

1. Target the "Churn Clusters"

When providing competitorUrls, prioritize competitors that have a large, vocal user base. The more "Reddit noise" there is, the better the agent can identify high-impact feature gaps.

2. Use "Vibe-Check" URLS

Instead of just providing a homepage (e.g., https://competitor.com), try providing their Trustpilot profile URL or a specific pricing page. This helps the internal scrapeProduct tool focus immediately on where users feel the most "pain" (the wallet).

3. Lean on gpt-4o-mini for Speed, gpt-4o for Depth

• For quick daily monitoring: Use gpt-4o-mini. It is incredibly fast and handles the "pruned" data perfectly.
• For board-level strategy: Switch to gpt-4o (or claude-3-5-sonnet). The reasoning capabilities are superior when connecting dots between complex technical complaints on Reddit.

4. Mind the "Product Name" Input

If your product is new or has a generic name (e.g., "Flow"), make sure to include the productName in the input if available. This prevents the Reddit scraper from bringing in irrelevant data about "water flow" or "yoga flow" and keeps the focus on your SaaS.

5. Review the "Pruned" Dataset

If you notice the agent missed a specific complaint, check the logs. The agent sorts by Upvotes/Impact. If a complaint is valid but has zero engagement on Reddit, the agent may prune it to save context window space.