Prediction Markets Scraper (Kalshi + Polymarket)
Pricing
from $1.00 / 1,000 results
Kalshi and Polymarket odds in one run: prices, implied probability, volume, open interest, order book and daily history, plus matched cross-venue pairs with the price spread. Public market data, no login. Bounded retries; failed venues are never charged.
Which prediction markets to read. Cross-venue pairs need both.
One keyword or phrase per line, such as nfl, super bowl, fed rate or bitcoin. Each line is searched separately on each venue and fails or succeeds on its own. Kalshi matches every word against series and event titles, tickers and tags; Polymarket uses its own search, or lists the tag when the keyword is a tag (nfl, nba). Leave empty for the most-traded markets.
Optional. One of sports, politics, elections, economics, finance, weather, climate, crypto, entertainment, tech, science, business, world, health, commodities, ai. Any other value is tried as a Polymarket tag (for example nfl or nba) and is unknown to Kalshi, so the Kalshi part fails (and is not charged). Optional with a keyword too: without a category, Kalshi's whole series catalog is searched.
Open markets (trading now), closed markets (trading ended, including resolved ones with their outcome) or all.
Skip markets with lower lifetime volume. Units differ by venue: contracts on Kalshi (each pays $1), US dollars on Polymarket. 0 keeps everything.
Adds a pair row (type: pair) for each question found on both venues, with both Yes prices and the spread. Game winners are matched by league, teams and date; other markets by fuzzy title and resolution date. Matching is approximate: check both markets' rules before acting on a spread.
Adds the top 5 bid and ask levels, total depth and the bid-ask spread for the Yes side. One extra request per market.
Adds one Yes price per day for the last N days (see History days). One extra request per market.
How many days of daily price history to keep when price history is on.
Stop after this many markets from each venue for each keyword. Kalshi markets come from its most-traded series first; Polymarket markets by 24-hour volume.
Hard cap on dataset rows for the whole run, markets and pairs together. Pairs are added last, so a low cap can leave no room for them. Leave empty for no cap.
Requests in flight to each venue at once. Requests are also paced to stay under the venues' public rate limits and slow down further on HTTP 429.
Each attempt uses a fresh proxy session and replays what the previous one already fetched. A venue/keyword that still fails is listed in the FAILED_SOURCES record and is not charged.
How long to wait for one HTTP response before the attempt counts as failed and is retried on a fresh session.
