SaaS Review Competitor Battlecard Miner

Production-ready Apify Actor for mining public SaaS review pages or supplied HTML into review rows and battlecard/theme rows.

Public-data-only posture

• Uses public URLs and supplied HTML only.
• No login flows, no private APIs, no anti-bot bypass, no gated scraping.
• If a page returns 403 or is otherwise blocked, the actor emits warning records instead of retrying aggressively.

What it does

• Accepts startUrls, reviewUrls, and htmlInputs.
• Extracts review-like rows from Capterra/GetApp-style public pages with best-effort support.
• Provides best-effort support for public G2 pages when the HTML is accessible.
• Mines battlecard rows from review text using deterministic heuristics.

Theme mining includes signals for:

• pricing and cost
• implementation and setup
• support quality
• integrations
• usability
• performance
• missing reporting, API, mobile, and customization capabilities
• competitor mentions from competitorKeywords
• quotable language

Inputs

See .actor/input_schema.json for the Apify input definition.

Main fields:

• startUrls: Apify request-list style objects with url
• reviewUrls: plain string URLs
• htmlInputs: raw HTML strings or local file paths
• maxReviews: cap the number of kept review rows
• minRating / maxRating: inclusive filters
• dateFrom / dateTo: ISO date filters
• competitorKeywords: names to flag as competitor mentions

Outputs

The actor pushes dataset rows with a type field:

• review: extracted review rows
• theme: derived battlecard/theme rows
• warning: blocked pages, fetch failures, or pages with no review-like rows

Useful key-value outputs:

• OUTPUT_SUMMARY: counts for reviews, themes, warnings, and sources
• OUTPUT_RECORDS_PREVIEW: first 25 emitted records

Example theme fields:

• battlecardType: objection, praised_feature, missing_feature, competitor_mention, or quote
• themeKey
• quote
• rating
• date
• sourceUrl
• evidenceSnippet

Local run

Install and test:

npm install
npm test

Run the actor locally against the deterministic fixture:

mkdir -p storage/key_value_stores/default
cp examples/sample-input.json storage/key_value_stores/default/INPUT.json
CRAWLEE_STORAGE_DIR=./storage npm start

If you prefer a direct Node run with raw HTML or fixture paths, the smoke test already exercises the real pipeline without network access.

Example input

{
  "htmlInputs": ["tests/fixtures/capterra-sample.html"],
  "competitorKeywords": ["HubSpot", "Pipedrive"],
  "maxReviews": 10
}

Warning records

Examples:

• code: "blocked" for 403 public-page responses
• code: "fetch_failed" for non-403 HTTP failures
• code: "fetch_error" for transport/runtime failures
• code: "no_reviews_found" when a page is accessible but no review-like rows match the heuristics

Buyer use cases

• Build competitor battlecards from public review evidence.
• Summarize recurring objections before sales enablement calls.
• Identify praised features to sharpen positioning.
• Flag missing capabilities buyers repeatedly ask for.
• Collect quotable customer language for messaging research.