Kickstarter Projects Scraper — Funding, Backers & Rewards

Reads Kickstarter's own discovery API (/discover/advanced?format=json) — the endpoint kickstarter.com itself calls. No key, no auth, no cookie.

What you get

recordType	One per	Carries
PROJECT	project	goal, pledged, USD-converted pledged, percent funded, backers, currency + FX rate, state, created/launched/deadline dates, category tree, location, creator, photo — plus reward tiers and live counters if you switch them on
SEARCH_SUMMARY	query	the exact query sent, Kickstarter's own hit count, how much of it is actually reachable, and which filters upstream honoured
ERROR	failed input	a named reason — every input maps to at least one row

Optional second pass:

• Reward tiers (includeProjectDetails) — title, minimum pledge, backers per tier, shipping, estimated delivery, add-ons, tags, update/comment counts.
• Live counters (includeLiveStats) — a 168-byte call per project for up-to-the-minute backers, pledged and comments. Discovery results are cached and can lag; both numbers are kept side by side rather than merged.

Three things this tells you that the API does not

1. One query can only ever reach 2,400 rows. Page 200 works; page 201 is a genuine HTTP 404. A query claiming 296,780 hits will hand you 2,400 of them and say nothing about the rest. Every summary carries totalHits, estimatedReachable, estimatedUnreachable and hitPageCeiling, so the gap is visible instead of inferred. The way past it is to split the query — that is what the queries list is for (by category, state, location or funding band).

2. A wrong categoryId or woeId is not an error — it returns everything. category_id=999999 comes back HTTP 200 with the full unfiltered baseline (685,886 projects) and no field saying the filter was dropped. This actor proves what was applied from the rows themselves, so it costs no extra request: categoryFilterHonoured / locationFilterHonoured in the summary, and categoryMatchesRequested / locationMatchesRequested on every row.

3. A subcategory is a relevance filter, not a strict one. Asking for category 35 (Video Games) returns Comic Books, Product Design, Print and Public Art alongside it — about 58% of a page actually matches. The hit count still moves (24,585 against a 685,886 baseline), so the usual "did the count change?" check passes on exactly the case that matters. Parent categories (Games, Technology, Design…) do filter strictly. Rows carry the per-row verdict and the summary carries categoryMatchRateFirstPage; filter on it for a strict set, ignore it for Kickstarter's own notion of the category.

Two inputs this actor refuses on purpose

• state=suspended. It looks like a sixth state and is not a filter value. Kickstarter answers it with 681,135 projects — the sum of all five real states — and returns the same rows as a nonsense string. Valid: live, successful, failed, canceled, upcoming.
• An unrecognised sort. Upstream does not error; it silently substitutes the default (magic), so the run would look sorted and not be. Verified by comparing returned id order, not counts — the count is identical for every sort.

term is honest, for contrast: a term with no matches returns zero, not the baseline.

Notes

• 12 projects per request.
• Cloudflare here is a TLS-fingerprint gate rather than a challenge for everyone; the actor uses a measured profile allowlist and rotates on failure.
• The project-detail page is intermittently unavailable over HTTP/2 even after retries. When that happens the row keeps all of its discovery data and carries detailFetched: false plus detailError — a detail failure never costs you the project.
• A proxy is optional and off by default. Worth switching on for long runs.