# CFTC COT Report Scraper - Futures Positioning Data (`jungle_synthesizer/cftc-cot-positions-scraper`) Actor Pull the CFTC Commitments of Traders (COT) report as structured rows. Covers legacy, disaggregated, and financial-futures variants. Filter by commodity, market code, and date range. Long, short, spread positions per trader category plus open interest and trader counts. - **URL**: https://apify.com/jungle\_synthesizer/cftc-cot-positions-scraper.md - **Developed by:** [BowTiedRaccoon](https://apify.com/jungle_synthesizer) (community) - **Categories:** Business, Developer tools, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## CFTC COT Report Scraper — Commitments of Traders Weekly Positions Extracts weekly Commitments of Traders (COT) reports from the [U.S. Commodity Futures Trading Commission](https://publicreporting.cftc.gov). Returns structured rows — long, short, and spread positions by trader category — for over 680,000 historical records across energy, agriculture, metals, FX, and equity-index markets. Bloomberg charges for this. The CFTC publishes it for free on a Socrata API that nobody made an Apify actor for, until now. ### CFTC COT Report Scraper Features - **Extracts all four report variants** — Legacy Combined, Legacy Futures Only, Disaggregated Combined, and Traders in Financial Futures (TIFF) - **Returns 30+ fields per row** — long, short, spread positions for commercial, non-commercial, and non-reportable traders, plus open interest, trader counts, and top-4 concentration metrics - **Filters by commodity** — case-insensitive contains match on commodity_name (e.g. `WHEAT`, `CRUDE OIL`, `GOLD`) - **Filters by date range** — pull just last week's data or the full history back to 2006 - **Filters by market code** — target a specific CFTC contract market code if you know exactly what you're after - **No proxies needed** — pure API, no browser, no rate-limit gymnastics required. The CFTC wants you to have this data. - **Coerces numeric types** — all position and open interest fields arrive as strings from the Socrata API; this actor returns them as numbers ### Who Uses COT Data? - **Commodity traders** — track non-commercial positioning to gauge speculative sentiment in crude oil, wheat, gold, and natural gas - **Macro newsletter writers** — automate weekly COT updates instead of manually downloading CFTC CSV files every Friday - **Quant researchers** — build positioning datasets for backtesting momentum and contrarian strategies across futures markets - **Prop shops** — monitor commercial vs. non-commercial positioning divergence as a leading indicator - **FX traders** — pull currency futures positioning (EUR, GBP, JPY, AUD) from the TIFF report to track institutional flows - **Data engineers** — feed structured COT rows into a warehouse, replacing the unreliable CFTC CSV download workflow ### How It Works 1. Select a report type — Legacy Combined covers most commodity and FX futures. Disaggregated splits producers and swap dealers. TIFF covers financial futures and currencies. 2. Set optional filters — commodity name, date range, and market code narrow the results. Leave them blank for everything. 3. The scraper hits the CFTC's public Socrata API, paginates through the results at 1,000 rows per request, and coerces all numeric fields from strings to numbers before saving. 4. Results arrive as structured JSON rows in the Apify dataset. Each row is one market-and-week observation. ### CFTC COT Input Configuration ```json { "reportType": "legacy_combined", "commodityName": "CRUDE OIL", "reportDateFrom": "2024-01-01", "reportDateTo": "2025-12-31", "maxItems": 100, "sp_intended_usage": "Commodity positioning analysis", "sp_improvement_suggestions": "" } ```` | Field | Type | Default | Description | |-------|------|---------|-------------| | `reportType` | string | `legacy_combined` | Report variant: `legacy_combined`, `legacy_futures_only`, `disaggregated_combined`, or `financial` | | `commodityName` | string | *(empty)* | Case-insensitive commodity name filter (e.g. `WHEAT`, `GOLD`, `CRUDE OIL`) | | `marketCode` | string | *(empty)* | CFTC contract market code filter (e.g. `001612`) | | `reportDateFrom` | string | *(empty)* | Start date in YYYY-MM-DD format | | `reportDateTo` | string | *(empty)* | End date in YYYY-MM-DD format | | `socrataAppToken` | string | *(empty)* | Optional free Socrata app token for higher rate limits | | `maxItems` | integer | `10` | Maximum rows to return. `0` = unlimited | #### Report Type Reference | Value | Coverage | |-------|----------| | `legacy_combined` | Legacy Futures + Options Combined — the most-cited weekly report | | `legacy_futures_only` | Legacy Futures Only — excludes options positions | | `disaggregated_combined` | Disaggregated: splits commercial into producers/merchants/swap dealers/managed money | | `financial` | Traders in Financial Futures (TIFF) — financial instruments and currencies | ### CFTC COT Output Fields ```json { "market_and_exchange_names": "CRUDE OIL, LIGHT SWEET-WTI - NEW YORK MERCANTILE EXCHANGE", "report_date": "2025-05-06", "report_week": "2025 Report Week 18", "contract_market_name": "CRUDE OIL, LIGHT SWEET", "cftc_contract_market_code": "067651", "cftc_market_code": "NYM", "cftc_region_code": "NYC", "cftc_commodity_code": "067", "commodity_name": "CRUDE OIL", "report_type": "legacy_combined", "open_interest_all": 1802413, "noncomm_positions_long_all": 399214, "noncomm_positions_short_all": 241380, "noncomm_positions_spread_all": 72881, "comm_positions_long_all": 1207844, "comm_positions_short_all": 1364788, "tot_rept_positions_long_all": 1679939, "tot_rept_positions_short_all": 1679048, "nonrept_positions_long_all": 122474, "nonrept_positions_short_all": 123365, "change_in_open_interest_all": -34892, "pct_of_oi_noncomm_long_all": 22.1, "pct_of_oi_noncomm_short_all": 13.4, "pct_of_oi_comm_long_all": 67.0, "pct_of_oi_comm_short_all": 75.7, "traders_tot_all": 482, "traders_noncomm_long_all": 152, "traders_noncomm_short_all": 106, "traders_comm_long_all": 67, "traders_comm_short_all": 78, "conc_gross_le_4_tdr_long_all": 10.2, "conc_gross_le_4_tdr_short_all": 11.4 } ``` | Field | Type | Description | |-------|------|-------------| | `market_and_exchange_names` | string | Full CFTC market and exchange name | | `report_date` | string | Report date (YYYY-MM-DD) | | `report_week` | string | YYYY Report Week WW label | | `contract_market_name` | string | Contract market name | | `cftc_contract_market_code` | string | CFTC contract market code | | `cftc_market_code` | string | CFTC market code | | `cftc_region_code` | string | CFTC region code | | `cftc_commodity_code` | string | CFTC commodity code | | `commodity_name` | string | Commodity name | | `report_type` | string | Report variant identifier | | `open_interest_all` | number | Total open interest | | `noncomm_positions_long_all` | number | Non-commercial long positions | | `noncomm_positions_short_all` | number | Non-commercial short positions | | `noncomm_positions_spread_all` | number | Non-commercial spread positions | | `comm_positions_long_all` | number | Commercial long positions | | `comm_positions_short_all` | number | Commercial short positions | | `tot_rept_positions_long_all` | number | Total reportable long positions | | `tot_rept_positions_short_all` | number | Total reportable short positions | | `nonrept_positions_long_all` | number | Non-reportable long positions | | `nonrept_positions_short_all` | number | Non-reportable short positions | | `change_in_open_interest_all` | number | Week-over-week change in open interest | | `pct_of_oi_noncomm_long_all` | number | Non-commercial long as % of open interest | | `pct_of_oi_noncomm_short_all` | number | Non-commercial short as % of open interest | | `pct_of_oi_comm_long_all` | number | Commercial long as % of open interest | | `pct_of_oi_comm_short_all` | number | Commercial short as % of open interest | | `traders_tot_all` | number | Total reportable traders | | `traders_noncomm_long_all` | number | Non-commercial long traders | | `traders_noncomm_short_all` | number | Non-commercial short traders | | `traders_comm_long_all` | number | Commercial long traders | | `traders_comm_short_all` | number | Commercial short traders | | `conc_gross_le_4_tdr_long_all` | number | Top-4 traders long concentration (%) | | `conc_gross_le_4_tdr_short_all` | number | Top-4 traders short concentration (%) | ### 🔍 FAQ **How do I scrape CFTC COT data with Apify?** CFTC COT Report Scraper pulls directly from the CFTC's public Socrata API. Configure a report type, set any filters, and run — results are in the dataset within seconds. **What data can I get from the CFTC Commitments of Traders report?** CFTC COT Report Scraper returns long, short, and spread positions broken out by trader category (commercial, non-commercial, non-reportable) plus open interest, week-over-week changes, trader counts, and top-4 concentration percentages. That's essentially everything the CFTC publishes for each market and week. **Does CFTC COT Report Scraper need proxies?** No. The CFTC public reporting API is open to anyone with an internet connection. No proxies, no browser, no authentication required. **How far back does the COT data go?** The Socrata API covers legacy reports back to 2006, with over 270,000 rows in the Legacy Combined dataset alone. **Can I get currency futures positioning with this actor?** Yes. Use `reportType: "financial"` to pull the Traders in Financial Futures (TIFF) report, which covers EUR, GBP, JPY, AUD, CAD, and other currency futures alongside equity index and Treasury markets. **How much does it cost to run?** CFTC COT Report Scraper charges $0.10 per run plus $0.001 per row. Pulling the most recent week of legacy data (roughly 300 markets) costs about $0.40 total. ### Need More Features? Need custom fields, additional date granularity, or a combined multi-report run? [File an issue](https://console.apify.com/actors/issues) or get in touch. ### Why Use CFTC COT Report Scraper? - **Free data source** — the CFTC publishes this weekly; the scraper just makes it programmable without writing your own API client - **Clean output** — all numeric fields are coerced from strings to numbers, so your downstream pipeline doesn't have to parse "1802413" into a real number - **No proxies, no browser** — pure API call with a 100ms courtesy delay, which is more polite than most things that hit government servers # Actor input Schema ## `sp_intended_usage` (type: `string`): Please describe how you plan to use the data extracted by this crawler. ## `sp_improvement_suggestions` (type: `string`): Provide any feedback or suggestions for improvements. ## `sp_contact` (type: `string`): Provide your email address so we can get in touch with you. ## `reportType` (type: `string`): Which COT report variant to pull. ## `commodityName` (type: `string`): Filter by commodity name (e.g. WHEAT, CRUDE OIL, GOLD). Case-insensitive. Leave blank for all commodities. ## `marketCode` (type: `string`): Filter by CFTC contract market code (e.g. 001612 for WHEAT-HRW). Leave blank for all. ## `reportDateFrom` (type: `string`): Start date for report\_date filter. Leave blank for earliest available (2006+). ## `reportDateTo` (type: `string`): End date for report\_date filter. Leave blank for the most recent report. ## `socrataAppToken` (type: `string`): Free Socrata app token to raise rate limits. Register at https://data.cftc.gov/login. ## `maxItems` (type: `integer`): Maximum number of COT rows to return. 0 = unlimited (full history). ## Actor input object example ```json { "sp_intended_usage": "Describe your intended use...", "sp_improvement_suggestions": "Share your suggestions here...", "sp_contact": "Share your email here...", "reportType": "legacy_combined", "maxItems": 10 } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "sp_intended_usage": "Describe your intended use...", "sp_improvement_suggestions": "Share your suggestions here...", "sp_contact": "Share your email here...", "reportType": "legacy_combined", "commodityName": "", "marketCode": "", "reportDateFrom": "", "reportDateTo": "", "socrataAppToken": "", "maxItems": 10 }; // Run the Actor and wait for it to finish const run = await client.actor("jungle_synthesizer/cftc-cot-positions-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "sp_intended_usage": "Describe your intended use...", "sp_improvement_suggestions": "Share your suggestions here...", "sp_contact": "Share your email here...", "reportType": "legacy_combined", "commodityName": "", "marketCode": "", "reportDateFrom": "", "reportDateTo": "", "socrataAppToken": "", "maxItems": 10, } # Run the Actor and wait for it to finish run = client.actor("jungle_synthesizer/cftc-cot-positions-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "sp_intended_usage": "Describe your intended use...", "sp_improvement_suggestions": "Share your suggestions here...", "sp_contact": "Share your email here...", "reportType": "legacy_combined", "commodityName": "", "marketCode": "", "reportDateFrom": "", "reportDateTo": "", "socrataAppToken": "", "maxItems": 10 }' | apify call jungle_synthesizer/cftc-cot-positions-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=jungle_synthesizer/cftc-cot-positions-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "CFTC COT Report Scraper - Futures Positioning Data", "description": "Pull the CFTC Commitments of Traders (COT) report as structured rows. Covers legacy, disaggregated, and financial-futures variants. Filter by commodity, market code, and date range. Long, short, spread positions per trader category plus open interest and trader counts.", "version": "0.1", "x-build-id": "OOqUgqKkWTuU7hjTR" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/jungle_synthesizer~cftc-cot-positions-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-jungle_synthesizer-cftc-cot-positions-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/jungle_synthesizer~cftc-cot-positions-scraper/runs": { "post": { "operationId": "runs-sync-jungle_synthesizer-cftc-cot-positions-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/jungle_synthesizer~cftc-cot-positions-scraper/run-sync": { "post": { "operationId": "run-sync-jungle_synthesizer-cftc-cot-positions-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "sp_intended_usage", "sp_improvement_suggestions" ], "properties": { "sp_intended_usage": { "title": "What is the intended usage of this data?", "minLength": 1, "type": "string", "description": "Please describe how you plan to use the data extracted by this crawler." }, "sp_improvement_suggestions": { "title": "How can we improve this crawler for you?", "minLength": 1, "type": "string", "description": "Provide any feedback or suggestions for improvements." }, "sp_contact": { "title": "Contact Email", "minLength": 1, "type": "string", "description": "Provide your email address so we can get in touch with you." }, "reportType": { "title": "Report Type", "enum": [ "legacy_combined", "legacy_futures_only", "disaggregated_combined", "financial" ], "type": "string", "description": "Which COT report variant to pull.", "default": "legacy_combined" }, "commodityName": { "title": "Commodity Name Filter", "type": "string", "description": "Filter by commodity name (e.g. WHEAT, CRUDE OIL, GOLD). Case-insensitive. Leave blank for all commodities." }, "marketCode": { "title": "Market Code Filter", "type": "string", "description": "Filter by CFTC contract market code (e.g. 001612 for WHEAT-HRW). Leave blank for all." }, "reportDateFrom": { "title": "Report Date From (YYYY-MM-DD)", "type": "string", "description": "Start date for report_date filter. Leave blank for earliest available (2006+)." }, "reportDateTo": { "title": "Report Date To (YYYY-MM-DD)", "type": "string", "description": "End date for report_date filter. Leave blank for the most recent report." }, "socrataAppToken": { "title": "Socrata App Token (Optional)", "type": "string", "description": "Free Socrata app token to raise rate limits. Register at https://data.cftc.gov/login." }, "maxItems": { "title": "Max Items", "minimum": 0, "type": "integer", "description": "Maximum number of COT rows to return. 0 = unlimited (full history).", "default": 10 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```