# Polymarket Scraper (`shahidirfan/polymarket-scraper`) Actor Extract comprehensive predictions, market trends, and betting volume data from Polymarket with ease. Scrape individual markets, category pages, search results, or the entire active predictions catalog. - **URL**: https://apify.com/shahidirfan/polymarket-scraper.md - **Developed by:** [Shahid Irfan](https://apify.com/shahidirfan) (community) - **Categories:** Automation, Developer tools, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Polymarket Scraper Extract comprehensive predictions, market trends, and betting volume data from Polymarket with ease. Scrape individual markets, category pages, search results, or the entire active predictions catalog. --- ### Features - **Dynamic URL Extraction** — Automatically detects and processes category pages, event pages, individual markets, or general catalog listings. - **Keyword Search Support** — Search for markets and events dynamically using keywords or search terms. - **Detailed Data Fields** — Extracts prediction questions, descriptions, outcomes, current prices, total volume, liquidity, resolution dates, and more. - **Data Sanitization** — Automatically removes empty or null values from the dataset to deliver clean, import-ready results. - **Pagination Control** — Efficiently paginates through hundreds of prediction markets using configurable limits. --- ### Use Cases #### Sentiment Analysis Track public sentiment on major geopolitical events, elections, and cultural milestones in real-time. Use prediction market prices as a leading indicator of public opinion. #### Crypto & Asset Research Monitor crypto price levels, protocols, and regulatory predictions. Combine market sentiment data with token price action to build comprehensive trading strategies. #### Market Intelligence Track trading volumes, total liquidity, and outcome histories for various categories to analyze betting behavior, liquidity pools, and market efficiency. --- ### Input Parameters | Parameter | Type | Required | Default | Description | | -------------------- | ------- | -------- | ---------------------------------------- | -------------------------------------------------------------------------------- | | `url` | String | No | `https://polymarket.com/predictions/all` | Polymarket URL to scrape (e.g. predictions page, event page, or market page). | | `keyword` | String | No | `"bitcoin"` | Search keyword to look up matching events/markets. Ignored if a URL is provided. | | `location` | String | No | — | Optional location filter to maintain schema compatibility. | | `results_wanted` | Integer | No | `20` | Maximum number of market results to collect. | | `max_pages` | Integer | No | `10` | Safety cap on the number of pages/offsets to fetch. | | `proxyConfiguration` | Object | No | `{"useApifyProxy": false}` | Optional proxy settings to bypass IP limits. | --- ### Output Data Each market in the dataset contains the following fields (all null/undefined values are filtered out): | Field | Type | Description | | ---------------- | ------- | ------------------------------------------------------- | | `id` | String | Unique identifier of the market. | | `question` | String | The prediction question or title. | | `slug` | String | URL-friendly slug of the market. | | `description` | String | Specific rules and conditions for resolving the market. | | `outcomes` | Array | List of possible outcomes (e.g., `["Yes", "No"]`). | | `outcomePrices` | Array | Current trading prices for each outcome. | | `volume` | Number | Total volume traded on the market. | | `liquidity` | Number | Total liquidity available in the market. | | `startDate` | String | ISO timestamp representing when the market opened. | | `endDate` | String | ISO timestamp representing when the market resolves. | | `active` | Boolean | Whether the market is currently active. | | `closed` | Boolean | Whether the market has closed. | | `resolvedBy` | String | On-chain resolution contract or address. | | `clobTokenIds` | Array | Unique asset IDs for the outcomes. | | `volume24hr` | Number | Trading volume in the last 24 hours. | | `lastTradePrice` | Number | The price of the most recent trade. | | `bestBid` | Number | Current best bid price. | | `bestAsk` | Number | Current best ask price. | | `image` | String | Cover image URL associated with the market. | | `url` | String | The direct URL to view the market event on Polymarket. | --- ### Usage Examples #### Basic Extraction Extract all active markets from the main predictions page: ```json { "url": "https://polymarket.com/predictions/all", "results_wanted": 50 } ```` #### Keyword Search Search for predictions related to "Bitcoin" or other crypto assets: ```json { "keyword": "bitcoin", "results_wanted": 20 } ``` #### Event Specific Extract all sub-markets associated with a specific event: ```json { "url": "https://polymarket.com/event/what-will-happen-before-gta-vi", "results_wanted": 10 } ``` *** ### Sample Output ```json { "id": "540817", "question": "New Rihanna Album before GTA VI?", "slug": "new-rhianna-album-before-gta-vi-926", "description": "This market will resolve to \"Yes\" if Rihanna officially releases a new album before Grand Theft Auto VI is officially released in the US. Otherwise, this market will resolve to \"No\".", "outcomes": ["Yes", "No"], "outcomePrices": ["0.575", "0.425"], "volume": 742741.7449710022, "liquidity": 23875.3299, "startDate": "2025-05-02T15:48:10.582Z", "endDate": "2026-07-31T12:00:00Z", "active": true, "closed": false, "clobTokenIds": [ "98022490269692409998126496127597032490334070080325855126491859374983463996227", "53831553061883006530739877284105938919721408776239639687877978808906551086026" ], "volume24hr": 4963.8506320000015, "lastTradePrice": 0.59, "bestBid": 0.57, "bestAsk": 0.58, "image": "https://polymarket-upload.s3.us-east-2.amazonaws.com/what-will-happen-before-gta-vi-7hpNkEzQEqUE.jpg", "url": "https://polymarket.com/event/what-will-happen-before-gta-vi" } ``` *** ### Tips for Best Results #### URL Formats Ensure URLs are well-formed. The following patterns are supported: - `https://polymarket.com/predictions/all` (All predictions) - `https://polymarket.com/predictions/{category}` (e.g. `politics`, `crypto`, `sports`, `science`, `pop-culture`, `business`) - `https://polymarket.com/event/{event-slug}` (e.g. `what-will-happen-before-gta-vi`) #### Adjust Results Count - Set `results_wanted` to a smaller value (e.g., 20) for rapid validation testing. - Increase the count for complete historical or market snapshots. *** ### Integrations Connect your Polymarket data with modern applications: - **Google Sheets** — Export for spreadsheets and analytics. - **Airtable** — Build dynamic, searchable prediction trackers. - **Slack** — Get alerts for new predictions or significant price movements. - **Webhooks** — Automate triggers on custom endpoints. #### Export Formats - **JSON** — Import-ready for databases and custom apps. - **CSV** — Standard format for spreadsheets and Excel. *** ### Frequently Asked Questions #### Can I scrape closed prediction markets? This scraper focuses primarily on active and open predictions to provide real-time sentiment signals. Closed markets are generally resolved. #### What happens to null values? Any data fields that are not defined by the source API are removed from the outputs to ensure the exported dataset is neat and ready for data models. #### Does the scraper require a proxy? While not strictly required, a proxy configuration (such as Apify Proxy) is recommended for high-volume runs to prevent temporary rate limiting. *** ### Support For issues or custom requests, contact support through the Apify Console. #### Resources - [Apify Documentation](https://docs.apify.com/) - [API Reference](https://docs.apify.com/api/v2) *** ### Legal Notice This scraper is designed for research and public data gathering purposes. Users are responsible for complying with the target platform's terms of service and applicable data retrieval laws. # Actor input Schema ## `url` (type: `string`): Provide a category URL (e.g. predictions/politics), event URL (e.g. event/will-a-new-country-be-created-in-2026), or market URL. ## `keyword` (type: `string`): Search keyword for markets or events (e.g., 'bitcoin', 'election'). Ignored if URL is provided. ## `location` (type: `string`): Optional location filter to maintain schema compatibility. ## `results_wanted` (type: `integer`): Maximum number of market results to collect. ## `max_pages` (type: `integer`): Maximum number of pages/offsets to fetch (pagination safety cap). ## `proxyConfiguration` (type: `object`): Use Apify Proxy for reliable requests. ## Actor input object example ```json { "url": "https://polymarket.com/predictions/all", "keyword": "bitcoin", "results_wanted": 20, "max_pages": 10, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "url": "https://polymarket.com/predictions/all", "keyword": "bitcoin", "results_wanted": 20, "max_pages": 10 }; // Run the Actor and wait for it to finish const run = await client.actor("shahidirfan/polymarket-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "url": "https://polymarket.com/predictions/all", "keyword": "bitcoin", "results_wanted": 20, "max_pages": 10, } # Run the Actor and wait for it to finish run = client.actor("shahidirfan/polymarket-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "url": "https://polymarket.com/predictions/all", "keyword": "bitcoin", "results_wanted": 20, "max_pages": 10 }' | apify call shahidirfan/polymarket-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=shahidirfan/polymarket-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Polymarket Scraper", "description": "Extract comprehensive predictions, market trends, and betting volume data from Polymarket with ease. Scrape individual markets, category pages, search results, or the entire active predictions catalog.", "version": "0.0", "x-build-id": "xpFZ0l2K1HgwvnknB" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/shahidirfan~polymarket-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-shahidirfan-polymarket-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/shahidirfan~polymarket-scraper/runs": { "post": { "operationId": "runs-sync-shahidirfan-polymarket-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/shahidirfan~polymarket-scraper/run-sync": { "post": { "operationId": "run-sync-shahidirfan-polymarket-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "url": { "title": "Polymarket URL", "type": "string", "description": "Provide a category URL (e.g. predictions/politics), event URL (e.g. event/will-a-new-country-be-created-in-2026), or market URL." }, "keyword": { "title": "Keyword Search", "type": "string", "description": "Search keyword for markets or events (e.g., 'bitcoin', 'election'). Ignored if URL is provided." }, "location": { "title": "Location Filter (Placeholder)", "type": "string", "description": "Optional location filter to maintain schema compatibility." }, "results_wanted": { "title": "Results Wanted", "minimum": 1, "type": "integer", "description": "Maximum number of market results to collect.", "default": 20 }, "max_pages": { "title": "Max Pages / Offsets", "minimum": 1, "type": "integer", "description": "Maximum number of pages/offsets to fetch (pagination safety cap).", "default": 10 }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Use Apify Proxy for reliable requests.", "default": { "useApifyProxy": false } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```