Kickstarter Scraper

Since the official Kickstarter API does not provide structured output of search results, you can get list of Kickstarter news and projects with this Kickstarter Scraper.

This Fork

This fork has the following fixes/changes:

• Fixes an issue where the image/photo of a project is not included correctly. See https://console.apify.com/actors/h9kEJvo62ufE8PkSC#/issues/qNFoNx3PwZLS7bBGo/comments
• Changed 'photo' to 'image'
• Adds the image to the 'description' so that it can be parsed by RSS feed readers. I have only tested this with feedly but hopefully others should work too.

Features

• scrape Kickstarter data up to 2400 items/per run
• get data from the most recent projects of any category and from any city
• search and scrape based on the amount of money aimed for, pledged and collected
• sort by status, recency or popularity
• name your Kickstarter datasets to keep track of all the various runs

Why scrape Kickstarter?

Kickstarter website is full of crowdfunding projects that are changing the world in many little ways. Here's what you can do with the data about them:

• Monitor both existing and upcoming Kickstarter projects in your country or city
• Analyze which projects are the most likely to get supported and estimate the potential for success
• Strengthen your own crowdfunding campaign with real-time data
• Follow the projects competing in the same category all over the country
• Oversee your own project and get notifications every time it gets funding
• Back up your Kickstarter investment analysis with the most recent data
• Keep track and archive the past successful projects in your city

Tutorial

For a more detailed explanation of how to scrape Kickstarter read a step-by-step tutorial on our blog.

And for more ideas on how to use the extracted data, check out our industries pages for concrete ways web scraping results are already being used across businesses of various scale and direction - in NGO and Government work or Journalism, for instance.

Cost of usage

On average, scraping 1000 items from Kickstarter via Apify platform will cost you as little as 0.03 USD credits off your subscription plan. For more details about the plans we offer, platform credits and usage, see the platform pricing page.

If you're not sure how much credits you've got left on your plan and whether you might need to upgrade, you can always check your limits in the Settings -> Usage and Billing tab in your Console. The easiest way to know how many credits your actor will need is to perform a test run.

Input

Input of this actor should be JSON containing filter specification. Allowed filters are:

Field	Description	Allowed values
query	Search term	Any string value, e.g. Nimbus 2000 project
category	Category to search in	Category slug from this list - anything from books to community gardens
location	Location to search around	Name of the location, e.g. Prague
status	Status of the project	All, Live or Successful
pledged	Amount pledged	One of: All, <$1,000 pledged, $1,000 to $10,000 pledged, $10,000 to $100,000 pledged, $100,000 to $1,000,000 pledged and >$1,000,000 pledged
goal	Goal amount	One of: All, <$1,000 goal, $1,000 to $10,000 goal, $10,000 to $100,000 goal, $100,000 to $1,000,000 goal and >$1,000,000 goal
raised	Amount % raised	One of: All, < 5% raised, 75% to 100% raised, >100% raised
Sort	Sort by	Popularity, newest, end_date, most_funded or most_backed
maxResults	Maximum number of projects in output	Has to be a positive number, from 0 to 2400 results
datasetName	Name of dataset that will be overwritten with data on each run	Alphabet characters, numbers and dash (e.q. my-dataset)

Input example:

1{
2    "query":  "Board games",  "
3    maxResults":  100,  
4    "category":  "games",  
5    "location":  "United States",  
6    "sort":  "newest",  "proxyConfig":  
7    {  "useApifyProxy":  true  },  
8    "status":  "Successful",  
9    "pledged":  "$1,000 to $10,000 pledged",  
10    "goal":  "$1,000 to $10,000 goal",  
11    "raised":  "All",  
12    "datasetName":  ""
13}

Important considerations

• maxResults - Kickstarter website can return a maximum of 200 pages, so at most, you will get 2400 results for any query. To get over 2400 results, run multiple instances of this actor with more specific search terms.

• datasetName - If you provide name to a named dataset, every time you run this Kickstarter search actor, it will clear the existing dataset and rewrite it with new data. You can use this option if you want to use named dataset as RSS feed or if you are creating an API.

• Location - You can also run another actor Kickstarter Location To Ids to find the specific ID associated with the location.

During the run

During the run, the actor will output messages letting the you know what is going on. If you provide incorrect input the Kickstarter search actor, it will immediately stop with Failure state and output an explanation of what is wrong.

Output

Once input is properly parsed the actor will load first page of results and output information about the number of found results and how many results are remaining. Every 10th page load will output prediction of how long it will take for the run to finish.

Output example

1{  
2"id":  1662550845,  
3"photo":  null,  
4"name":  "Escape from HelL",  
5"blurb":  "A board game about getting to Manhattan from Williamsburg once the L train shuts down.",  
6"goal":  2500,  
7"pledged":  7580,  
8"state":  "successful",  
9"slug":  "escape-from-hell",  
10"disable_communication":  false,  
11"country":  "US",  
12"country_displayable_name":  "the United States",  
13"currency":  "USD",  
14"currency_symbol":  "$",  
15"currency_trailing_code":  true,  
16"deadline":  1547854200,  
17"state_changed_at":  1547854200,  
18"created_at":  1543533003,  
19"launched_at":  1545138257,  
20"staff_pick":  false,  
21"is_starrable":  false,  
22"backers_count":  195,  
23"static_usd_rate":  1,  
24"usd_pledged":  "7580.0",  
25"converted_pledged_amount":  7580,  
26"fx_rate":  1,  
27"usd_exchange_rate":  1,  
28"current_currency":  "USD",  
29"usd_type":  "international",  
30"spotlight":  true,  
31"creatorId":  696946511,  
32"creatorName": "Escape from Hell",  
33"creatorAvatar": "https://ksr-ugc.imgix.net/assets/023/572/288/d31afa951de00ed08ad0c8876cfbb0f2_original.png?ixlib=rb-4.0.2&w=160&h=160&fit=crop&v=1544981648&auto=format&frame=1&q=92&s=c7c176c48efa1a78213c52d58f7b5b98",  
34"creatorUrl":  "https://www.kickstarter.com/profile/escapefromhell",  
35},

Kickstarter open source

You can manage search results in any language (Python, PHP, Node JS/NPM). See the FAQ or our API reference to learn more about getting results from this Kickstarter Search Actor. The code of this Kickstarter search actor is also open source, so you can create your own solution if you need.