Redbubble Market Intelligence Crawler

Automatically research any Redbubble niche with AI. Enter keywords, pick product types, and get back structured JSON analysis — saturation levels, bestseller signals, top design themes, and market gaps — powered by a stealth Firefox browser and LLM extraction.

---

What It Does

For each keyword + product type combination you provide, the actor:

1. Opens Redbubble in a stealth Firefox browser (Camoufox — undetectable by bot filters)
2. Navigates via residential proxies for IP rotation and Cloudflare bypass
3. Runs global block detection — retries automatically if a challenge page is served
4. Extracts page content and sends it to an LLM
5. Returns a flat JSON record per job with your custom analysis

---

Input Fields

Keywords (required unless using Raw URLs)

The niches or topics you want to research. Each keyword is paired with every product type you select, creating one crawl job per combination.

electrician
dark academia
nurse life
cottagecore

Product Types

Which Redbubble categories to search. Defaults to All Products for a broad view. Select multiple to compare saturation across categories for the same keyword.

Available options: All Products, T-Shirts, Stickers, Hoodies & Sweatshirts, Phone Cases, Art Prints, Posters, Mugs, Tote Bags, Throw Pillows, Laptop Skins, Notebooks & Journals, Tapestries, Masks.

Sort Order

Controls how Redbubble ranks results before the LLM sees them.

Option	Best for
Top Selling (default)	Market research — shows proven bestsellers
Most Relevant	General search intent
Newest	Spotting emerging trends early
Price: Low / High	Pricing research

LLM Extraction Instruction

The prompt sent to the AI for each page. Write it however you need — the actor adapts to any output shape. Examples:

• "List every product title visible on this page as a JSON array"
• "Extract the price range and number of reviews for each item"
• "Identify any recurring color palettes or art styles in the top 20 results"

The default instruction extracts: total results, saturation level, sales signals in top 10, recurring design themes, niche assessment, and demand/oversupply indicators.

Include Mature Content

Toggle adult content inclusion. Off by default.

Use Apify Proxy

Routes requests through Apify residential proxies. Strongly recommended — Redbubble uses Cloudflare and will block datacenter IPs. Enabled by default.

Proxy Groups

Which Apify proxy pool to use. Default: RESIDENTIAL.

Raw URLs (advanced)

Skip the keyword builder entirely and crawl specific URLs directly. Useful when you've already built a Redbubble URL with custom filters, or when crawling a non-Redbubble site.

---

Example Inputs

Simple — one keyword, one product type:

{
  "keywords": ["electrician"],
  "product_types": ["t-shirts"],
  "sort_order": "top selling"
}

Batch research — multiple niches across multiple product types:

{
  "keywords": ["electrician", "nurse life", "dark academia"],
  "product_types": ["t-shirts", "stickers", "mugs"],
  "sort_order": "top selling"
}

This creates 9 crawl jobs (3 keywords × 3 product types).

Custom extraction instruction:

{
  "keywords": ["cottagecore"],
  "product_types": ["all-departments"],
  "sort_order": "top selling",
  "instruction": "List the top 10 product titles visible, their approximate price, and any visible review counts. Return as a JSON object."
}

Advanced — raw URLs:

{
  "urls": [
    "https://www.redbubble.com/shop?query=electrician&sortOrder=top+selling&iaCode=t-shirts",
    "https://www.redbubble.com/shop?query=nurse+life&sortOrder=recent&iaCode=stickers"
  ]
}

---

Dyanmic Output Format (Example)

Each completed job produces one flat JSON record in the dataset. The fields returned by the LLM vary based on your instruction — the four metadata fields are always present:

{
  "keyword": "electrician",
  "product_type": "t-shirts",
  "sort_order": "top selling",
  "url": "https://www.redbubble.com/shop?query=electrician&...",

  "total_results": "1000+",
  "saturation_level": "high",
  "sales_signals_in_top_10": 7,
  "recurring_design_themes": [
    "Funny electrician puns",
    "Vintage lightning bolt graphics",
    "Minimalist wiring diagrams"
  ],
  "niche_assessment": "opportunity",
  "demand_oversupply_indicators": {
    "strong_demand_signals": ["Multiple bestseller badges", "Recurring popular themes"],
    "oversupply_indicators": ["High listing volume"]
  }
}

The Overview tab in Apify Console shows the stable metadata columns (keyword, product_type, sort_order, saturation_level, niche_assessment, url). The All Fields tab shows the complete LLM output.

Failed jobs are non-fatal — the run continues and records the error:

{
  "keyword": "...",
  "product_type": "...",
  "sort_order": "...",
  "url": "https://...",
  "error": "reason for failure"
}

---

Performance & Limits

Setting	Default	Env var to override
Rate limit	35 req/min	RATE_LIMIT_RPM
Concurrency	3 parallel browsers	MAX_CONCURRENCY
Retries	2 (backoff: 2s, 4s)	MAX_RETRIES
Timeout	120s per page	CRAWL_TIMEOUT_SECONDS

Typical cost: ~$0.06 per keyword/product combination. A batch of 50 keywords × 3 product types (150 jobs) runs for roughly $9.

---

Calling via API

Start a run:

curl -X POST \
  "https://api.apify.com/v2/acts/YOUR_USERNAME~camoufox-crawler/runs?token=YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "keywords": ["electrician", "nurse life"],
    "product_types": ["t-shirts", "stickers"],
    "sort_order": "top selling"
  }'

Fetch results when done:

$curl "https://api.apify.com/v2/acts/YOUR_USERNAME~camoufox-crawler/runs/last/dataset/items?token=YOUR_API_TOKEN"

---

How Block Detection Works

After every page load the actor checks:

• Page title — scanned for Cloudflare signals ("Just a moment", "Attention Required", etc.)
• Body HTML — scanned for known challenge fingerprints (cf_chl_opt, recaptcha, px-captcha, etc.)

If a block is detected the request is retried with a fresh browser session and proxy rotation. After all retries are exhausted the job is recorded as failed and the run continues.