IdeaBrowser Idea of the Day Scraper

Extract comprehensive startup idea data from IdeaBrowser.com including market analysis, opportunity scores, keyword metrics, and community signals. Perfect for entrepreneurs, investors, and product managers tracking startup opportunities.

Zero configuration required - Just click Start and get structured data in seconds.

Why IdeaBrowser Scraper?

• Complete Market Analysis - Get opportunity scores, problem severity, feasibility, and timing metrics all in one place
• Keyword Intelligence - Search volume and growth rates for related keywords to validate market demand
• Business Fit Metrics - Revenue potential, execution difficulty, and go-to-market scores
• Community Signals - Engagement data from Reddit, Facebook, YouTube to gauge audience interest
• Offer Strategy - Pre-built pricing tier suggestions (lead magnet, frontend, core product)
• Stealth Technology - Uses Patchright browser automation to bypass bot detection
• Structured Output - Clean JSON format ready for databases, spreadsheets, or AI analysis

Quick Start

Via Apify Console

No configuration required. Just click Start with default settings:

{
    "maxRetries": 3,
    "navigationTimeout": 60,
    "extractKeywords": true,
    "debugMode": false
}

Via Apify API (cURL)

curl -X POST "https://api.apify.com/v2/acts/YOUR_USERNAME~ideabrowser-scraper/runs?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"extractKeywords": true}'

Via Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });

const run = await client.actor('YOUR_USERNAME/ideabrowser-scraper').call({
    extractKeywords: true,
    maxRetries: 3,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items[0].title);           // Startup idea name
console.log(items[0].scores);          // Opportunity scores
console.log(items[0].keywords);        // Keywords with volume/growth
console.log(items[0].communitySignals); // Reddit, Facebook, YouTube metrics

Via Python

from apify_client import ApifyClient

client = ApifyClient('YOUR_TOKEN')

run = client.actor('YOUR_USERNAME/ideabrowser-scraper').call(run_input={
    'extractKeywords': True,
    'maxRetries': 3,
})

items = client.dataset(run['defaultDatasetId']).list_items().items
idea = items[0]

print(idea['title'])                    # Startup idea name
print(idea['scores']['opportunity'])    # Opportunity score
print(idea['businessFit'])              # Business fit metrics
print(idea['communitySignals'])         # Community engagement data

How It Works

1. Navigate - The actor opens IdeaBrowser.com using Patchright stealth browser
2. Extract - Parses the "Idea of the Day" page for all structured data
3. Enrich - Optionally interacts with keyword dropdowns to get volume/growth metrics
4. Return - Outputs clean JSON with all market analysis data

Input Parameters

Parameter	Type	Required	Default	Description
maxRetries	integer	No	3	Number of retry attempts if page fails to load (1-10)
navigationTimeout	integer	No	60	Page load timeout in seconds (30-120)
extractKeywords	boolean	No	true	Extract keyword volume and growth data. Disable for faster runs
proxyConfiguration	object	No	-	Optional proxy settings for geographic restrictions
debugMode	boolean	No	false	Save screenshots and HTML on failure for troubleshooting

Proxy Configuration

Most users do not need proxy configuration. The actor uses Patchright's built-in stealth mode. If you experience blocking:

{
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": ["RESIDENTIAL"]
    }
}

Output Format

Complete Output Example

{
    "title": "Parental Control App for Roblox",
    "badges": ["Perfect Timing", "Unfair Advantage"],
    "description": "Your kid's Roblox account just got hacked. You need a dedicated parental control solution that understands gaming platforms.",
    "keywords": [
        { "keyword": "parental controls for roblox", "volume": 27100, "growth": 653 },
        { "keyword": "roblox safety for kids", "volume": 8500, "growth": 125 },
        { "keyword": "roblox parental monitoring", "volume": 3200, "growth": 89 }
    ],
    "scores": {
        "opportunity": { "score": 9, "label": "Exceptional" },
        "problem": { "score": 9, "label": "Severe Pain" },
        "feasibility": { "score": 6, "label": "Challenging" },
        "timing": { "score": 8, "label": "Great Timing" }
    },
    "businessFit": {
        "revenuePotential": { "value": "$$$", "description": "$1M-$10M ARR potential" },
        "executionDifficulty": { "value": 8, "description": "Complex build, 6-month MVP" },
        "goToMarket": { "value": 9, "description": "Strong demand across platforms" },
        "rightForYou": "Best for founders with security/gaming background"
    },
    "offer": {
        "leadMagnet": {
            "name": "Free Roblox Safety Guide",
            "price": "Free",
            "description": "10-page PDF on protecting kids online"
        },
        "frontend": {
            "name": "Basic Monitoring",
            "price": "$9/mo",
            "description": "Activity alerts and play time limits"
        },
        "core": {
            "name": "Full Protection Suite",
            "price": "$29/mo",
            "description": "Complete monitoring, chat filtering, spending controls"
        }
    },
    "whyNow": "Roblox has 70M+ daily active users, mostly children. Recent high-profile hacking incidents have parents searching for solutions.",
    "proofAndSignals": "Parent complaints trending on Reddit r/parents and r/roblox. Multiple news articles about Roblox safety concerns.",
    "marketGap": "Existing parental controls are generic. No solution specifically addresses gaming platform risks like in-game purchases and chat.",
    "executionPlan": "Start with browser extension for monitoring. Expand to mobile app with real-time alerts. Partner with parent bloggers for distribution.",
    "categorization": {
        "type": "SaaS",
        "market": "B2C",
        "target": "Parents of children ages 6-14",
        "mainCompetitor": "Bark",
        "trendAnalysis": "Rising concern over child online safety, especially in gaming"
    },
    "communitySignals": {
        "reddit": { "count": "5", "members": "2.5M+", "score": 8 },
        "facebook": { "count": "5", "members": "150K+", "score": 7 },
        "youtube": { "count": "1000+", "score": 7 },
        "other": { "count": "Multiple forums", "score": 6 }
    },
    "url": "https://www.ideabrowser.com/idea-of-the-day",
    "extractedAt": "2026-01-23T12:00:00.000Z"
}

Output Fields Reference

Field	Type	Description
title	string	Name of the startup idea
badges	array	Special badges (e.g., "Perfect Timing", "Unfair Advantage")
description	string	Detailed description of the problem and solution
keywords	array	Related search keywords with metrics
keywords[].keyword	string	Search term
keywords[].volume	number	Monthly search volume
keywords[].growth	number	Growth rate percentage
scores	object	Opportunity assessment scores (1-10)
scores.opportunity	object	Overall opportunity rating
scores.problem	object	Problem severity rating
scores.feasibility	object	Implementation feasibility
scores.timing	object	Market timing assessment
businessFit	object	Business viability metrics
businessFit.revenuePotential	object	Revenue potential ($, $$, $$$)
businessFit.executionDifficulty	object	Build complexity (1-10)
businessFit.goToMarket	object	GTM score (1-10)
businessFit.rightForYou	string	Founder fit assessment
offer	object	Suggested pricing tiers
offer.leadMagnet	object	Free offer to attract leads
offer.frontend	object	Low-cost entry product
offer.core	object	Main product offering
whyNow	string	Market timing analysis
proofAndSignals	string	Evidence of market demand
marketGap	string	Gap in current solutions
executionPlan	string	Suggested launch strategy
categorization	object	Business classification
categorization.type	string	Business type (SaaS, App, etc.)
categorization.market	string	Target market (B2B, B2C)
categorization.target	string	Target audience
categorization.mainCompetitor	string	Primary competitor
categorization.trendAnalysis	string	Market trend insights
communitySignals	object	Social platform metrics
communitySignals.reddit	object	Reddit engagement data
communitySignals.facebook	object	Facebook group data
communitySignals.youtube	object	YouTube content metrics
communitySignals.other	object	Other platform signals
url	string	Source URL
extractedAt	string	ISO timestamp of extraction

Use Cases

Startup Ideation

Get daily inspiration for your next venture with validated market data. Use the opportunity scores and keyword volume to prioritize ideas worth pursuing.

Market Research

Track trending problems and emerging opportunities. Build a database of startup ideas by scheduling daily runs and analyzing patterns over time.

Investment Analysis

Identify promising market segments before they become crowded. Use community signals and growth metrics to spot early-stage opportunities.

Content Creation

Source material for startup-focused newsletters, podcasts, and YouTube channels. Each idea comes with ready-to-use narrative content (why now, market gap, execution plan).

Competitive Intelligence

Monitor emerging competitors and market trends. Use the categorization data to understand where new solutions are appearing.

Product Validation

Validate your own startup ideas against IdeaBrowser's scoring methodology. Compare your concept to daily featured ideas.

Integrations

Google Sheets

Export data directly to Google Sheets using Apify's native integration. Create a live dashboard of startup opportunities.

Airtable / Notion

Push results to your preferred database for team collaboration and idea tracking.

Slack / Discord

Set up webhook notifications to receive daily ideas in your team channel.

Zapier / Make

Connect to 1000+ apps for custom workflows. Trigger actions based on opportunity scores or keyword growth.

AI/MCP Integration

This actor is fully compatible with Apify's MCP server for AI agent integrations. Use with Claude, GPT, or other AI assistants to analyze and discuss startup opportunities.

Scheduling for Daily Updates

Build a comprehensive database of startup ideas over time:

1. Go to the Actor's Schedules tab in Apify Console
2. Click Create new schedule
3. Set frequency to Daily at your preferred time (ideas update each morning)
4. Each run captures that day's featured idea with all metrics

After 30 days, you'll have a valuable dataset of startup opportunities with trends and patterns.

Performance and Cost

Metric	Value
Typical run time	30-60 seconds
Memory usage	~1 GB (browser automation)
Compute cost	~$0.02 per run
Success rate	95%+ with default settings

Limitations

• Single idea per run - Extracts only the current "Idea of the Day"
• Daily updates - IdeaBrowser changes their featured idea once per day
• Content availability - Some fields may be empty if not present on the source page
• Bot detection - Occasional blocks possible; increase maxRetries if needed
• Keyword extraction - Requires JavaScript interaction; can be disabled for faster runs

FAQ

Why did my run fail with "bot detection"?

The actor uses Patchright for stealth browser automation, but IdeaBrowser may occasionally detect automated access. Solutions:

1. Increase maxRetries to 5-10
2. Enable proxy configuration with residential proxies
3. Enable debugMode to see screenshots of what went wrong

Can I scrape historical ideas?

No, IdeaBrowser only displays the current day's featured idea. To build an archive, schedule daily runs. Each run captures that day's idea.

Why are some fields empty or null?

IdeaBrowser's content varies by idea. Fields like communitySignals or specific keywords may not be available for every idea depending on the source data.

How do I speed up extraction?

Set extractKeywords: false to skip keyword volume/growth extraction. This reduces run time but you'll only get keyword names without metrics.

Can I use this with AI assistants?

Yes, this actor is compatible with Apify's MCP (Model Context Protocol) server. Connect it to Claude, GPT, or other AI assistants for automated startup analysis.

What timezone does IdeaBrowser use?

IdeaBrowser updates their featured idea based on US timezone. For best results, schedule runs in the morning US time.

How reliable is the data?

The actor extracts exactly what IdeaBrowser displays. Data quality depends on IdeaBrowser's analysis methodology. Scores, keywords, and metrics come directly from their platform.

Technical Details

Component	Technology
Browser Automation	Patchright (stealth Playwright fork)
Bot Evasion	Built-in fingerprint randomization
Runtime	Node.js 20 with TypeScript
Output Format	JSON to Apify Dataset
Error Handling	Automatic retries with exponential backoff

Changelog

v1.0.2

• Fixed dataset schema for Apify Store display
• Improved output schema documentation
• Added multiple dataset views (Overview, Scores, Keywords, etc.)

v1.0.1

• Added input configuration options
• Improved reliability with retry logic
• Better error handling and debug mode

v1.0.0

• Initial release

---

Built by Ekipnico | Data source: IdeaBrowser.com