getapp-scraper

Scrapes software listings from GetApp.com (Gartner Digital Markets' software-review and lead-gen site, sister to Capterra) — ratings, reviews, pricing plans, features, categories, and more — via the site's own sitemap and each listing's server-rendered Next.js data. No login required.

Contents

• Key features
• Output
• Input
• Input examples
• Incremental (delta) mode
• FAQ

🔑 Key features

• Rich, structured fields from the page's own Next.js data, not just the page's basic schema.org markup — rating, review count, full pricing-plan breakdown, features, integrations, supported platforms, typical customers, alternatives, pros/cons, support/training options, FAQ.
• No internal API used. GetApp's own /api/* exists but is robots-disallowed — every field here comes from the same server-rendered HTML a browser gets, never that API.
• Delta mode built in. Every run classifies each item as new, changed, unchanged, or delisted against a persisted baseline — pay for what changed, not a full re-scrape every time.
• Confirmed-dead listings are never re-fetched. A removed listing returns a plain HTTP 404 — tracked in a persisted registry so a future run skips it without a wasted request.
• maxItems defaults to 50 — a fast, cheap preview, and what keeps an unconfigured run within Apify's automated 5-minute QA check. Clear it (null) for a full crawl (~45,000 listings as of this build).

📋 Output

One dataset item per software listing — see .actor/dataset_schema.json for the full field list.

Example record (trimmed):

{
  "source": "getapp",
  "external_id": "8c7f8bf5-dbf6-47ee-9df7-a6d200b64092",
  "url": "https://www.getapp.com/customer-management-software/a/salesforce/",
  "name": "Salesforce Sales Cloud",
  "company_name": "Salesforce",
  "tagline": "Cloud-based CRM and sales automation software",
  "rating_avg": 4.4,
  "num_reviews": 18810,
  "categories": [{ "id": "...", "name": "Business Management", "slug": "business-management" }],
  "pricing_amount": 25.0,
  "pricing_currency": "USD",
  "pricing_periodicity": "month",
  "pricing_model": "Per User",
  "pricing_raw": { "plans": [{ "name": "Starter Suite", "startingPrice": 25 }] },
  "change_type": "new"
}

Several fields (features_raw, integrations_raw, alternatives_raw, pros_cons_raw, support_options_raw, training_options_raw, faq_raw, vendor_raw) are kept as the site's own raw nested JSON rather than flattened into fixed columns — real but not fully mapped shapes, same call as tappan-collective-scraper's tags/body_html fields.

🔧 Input

Field	Type	Default	Description
startUrls	array	—	Specific listing URLs to scrape directly instead of the full sitemap.
maxItems	integer	50	Stop after pushing this many items. Set to null for a full crawl.
mode	string	auto	auto / full / incremental — see Incremental mode.
concurrency	integer	15	How many listing pages to fetch in parallel.
impersonate	string	chrome	curl_cffi TLS-impersonation target.
proxyConfiguration	object	RESIDENTIAL	Apify Proxy config. Required — see FAQ.

📥 Input examples

Default preview (50 listings):

{}

Full catalog:

{ "maxItems": null }

🔁 Incremental (delta) mode

auto mode does a full scan the first time it runs, then only pushes new/changed items on later runs. Only an uncapped, unscoped run (no maxItems limit reached, no startUrls, no item errors) can detect delistings or update the baseline.

❓ FAQ

Does this need a proxy? Yes, a real requirement — not the usual "leave it off" answer most actors in this fleet give. GetApp.com returns HTTP 403 to a real share of requests from Apify's datacenter/cloud IP ranges regardless of TLS fingerprint (confirmed via a live cloud probe: chrome/firefox/safari impersonation all landed only ~55-65% clean with no proxy) — a genuine IP-reputation block, not a fingerprint issue. Apify's Residential proxy group clears it (100% clean in that same probe) and is the schema default; leaving it off or switching to a datacenter group will likely cause sustained failures.

Why are some fields raw/unflattened JSON instead of proper columns? Those fields (features, integrations, alternatives, pros/cons, support/training options, FAQ, vendor) have real but not-yet-fully-mapped shapes across GetApp's ~45,000 listings — kept as opaque passthrough JSON rather than guessed at, to avoid a schema crash on a shape this build didn't happen to sample.