# United States Department of Agriculture QuickStats Scraper (`fortuitous_pirate/usda-quickstats-scraper`) Actor - **URL**: https://apify.com/fortuitous\_pirate/usda-quickstats-scraper.md - **Developed by:** [Fortuitous Pirate](https://apify.com/fortuitous_pirate) (community) - **Categories:** Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $3.50 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## USDA QuickStats Scraper Scrapes agricultural production statistics from the USDA QuickStats API including crop yields, livestock inventory, and farm economics data. Access comprehensive agricultural data from the National Agricultural Statistics Service (NASS) database covering decades of US agricultural history. ### Features - Query any agricultural commodity (corn, soybeans, wheat, cattle, etc.) - Filter by year, state, and statistic category - Access national, state, and county-level data - Configurable result limits up to 50,000 records - Automatic data parsing and normalization - Works with DEMO_KEY (no API key required for basic usage) ### Input Parameters | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | `commodity` | string | No | `CORN` | Agricultural commodity to search for (e.g., CORN, SOYBEANS, WHEAT, CATTLE, HOGS, CHICKENS) | | `year` | integer | No | - | Year for data (1900-2030). Leave empty for all years | | `state` | string | No | - | US state abbreviation (e.g., IA, IL, NE). Leave empty for national data | | `statisticCategory` | string | No | - | Type of statistic (e.g., AREA PLANTED, PRODUCTION, YIELD, INVENTORY) | | `limit` | integer | No | `100` | Maximum number of records to return (1-50,000) | ### Output Schema Each record contains the following fields: ```json { "commodity": "CORN", "statisticCategory": "YIELD", "unit": "BU / ACRE", "value": 176.7, "year": 2023, "period": "ANNUAL", "state": "IOWA", "stateCode": "IA", "county": null, "countyCode": "000", "aggLevel": "STATE", "domainCategory": "NOT SPECIFIED", "sourceDesc": "SURVEY", "cvPercent": "2.1", "shortDesc": "CORN, GRAIN - YIELD, MEASURED IN BU / ACRE", "scrapedAt": "2024-01-15T12:00:00.000Z" } ```` ### API Information - **Source**: [USDA QuickStats API](https://quickstats.nass.usda.gov/api) - **Documentation**: [API Guide](https://quickstats.nass.usda.gov/api#param_define) - **Authentication**: Optional. Uses `DEMO_KEY` by default, which has lower rate limits - **Custom API Key**: Set the `USDA_API_KEY` environment variable for higher limits - **Rate Limits**: - DEMO\_KEY: 30 requests/hour, 50 requests/day - Registered Key: 1,000 requests/hour - **Request Timeout**: 120 seconds (the API can be slow for large queries) ### Example Usage #### Apify Console Run the actor with default settings to get CORN data: ```json { "commodity": "CORN", "year": 2023, "state": "IA", "statisticCategory": "YIELD", "limit": 100 } ``` Get soybean production data for Illinois: ```json { "commodity": "SOYBEANS", "year": 2023, "state": "IL", "statisticCategory": "PRODUCTION", "limit": 500 } ``` Get national cattle inventory: ```json { "commodity": "CATTLE", "statisticCategory": "INVENTORY", "limit": 1000 } ``` #### Apify API (JavaScript) ```javascript import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN', }); const input = { commodity: 'WHEAT', year: 2023, state: 'KS', statisticCategory: 'AREA PLANTED', limit: 200 }; const run = await client.actor('YOUR_USERNAME/usda-quickstats-scraper').call(input); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items); ``` #### Apify API (cURL) ```bash curl -X POST "https://api.apify.com/v2/acts/YOUR_USERNAME~usda-quickstats-scraper/runs?token=YOUR_APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "commodity": "CORN", "year": 2023, "limit": 100 }' ``` ### Troubleshooting | Issue | Cause | Solution | |-------|-------|----------| | No data returned | Invalid commodity or filter combination | Check commodity spelling (use uppercase). Try removing filters to broaden search | | Rate limit exceeded | Too many requests with DEMO\_KEY | Register for a free API key at [USDA QuickStats](https://quickstats.nass.usda.gov/api) and set `USDA_API_KEY` environment variable | | Request timeout | Query returns too many results | Add more filters (year, state) or reduce limit | | Empty value fields | Data not available for that combination | Some statistics are only available for certain years/states | | Invalid state code | Incorrect state abbreviation | Use standard 2-letter US state codes (e.g., IA, TX, CA) | ### Common Commodities - **Crops**: CORN, SOYBEANS, WHEAT, COTTON, HAY, RICE, BARLEY, OATS - **Livestock**: CATTLE, HOGS, CHICKENS, TURKEYS, SHEEP - **Dairy**: MILK - **Specialty**: HONEY, MAPLE SYRUP, CHRISTMAS TREES ### Common Statistic Categories - AREA PLANTED - AREA HARVESTED - PRODUCTION - YIELD - INVENTORY - SALES - PRICE RECEIVED ### Legal & Attribution This actor accesses publicly available data from the USDA National Agricultural Statistics Service (NASS) QuickStats database. Data is provided by the United States Department of Agriculture and is in the public domain. - **Data Source**: USDA NASS QuickStats - **Terms of Use**: [USDA Data Policies](https://www.usda.gov/policies-and-links) - **API Terms**: Users should comply with USDA API usage guidelines and rate limits When using this data, please cite: "Source: USDA National Agricultural Statistics Service" # Actor input Schema ## `commodity` (type: `string`): Agricultural commodity to search for (e.g., CORN, SOYBEANS, WHEAT, CATTLE) ## `year` (type: `integer`): Year for data (leave empty for all years) ## `state` (type: `string`): US state abbreviation (e.g., IA, IL, NE) - leave empty for national data ## `statisticCategory` (type: `string`): Type of statistic (e.g., AREA PLANTED, PRODUCTION, YIELD) ## `limit` (type: `integer`): Maximum number of records to return ## Actor input object example ```json { "commodity": "CORN", "limit": 100 } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = {}; // Run the Actor and wait for it to finish const run = await client.actor("fortuitous_pirate/usda-quickstats-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = {} # Run the Actor and wait for it to finish run = client.actor("fortuitous_pirate/usda-quickstats-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{}' | apify call fortuitous_pirate/usda-quickstats-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=fortuitous_pirate/usda-quickstats-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "United States Department of Agriculture QuickStats Scraper", "version": "1.0", "x-build-id": "abDhFD79ausK3JgVa" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/fortuitous_pirate~usda-quickstats-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-fortuitous_pirate-usda-quickstats-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/fortuitous_pirate~usda-quickstats-scraper/runs": { "post": { "operationId": "runs-sync-fortuitous_pirate-usda-quickstats-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/fortuitous_pirate~usda-quickstats-scraper/run-sync": { "post": { "operationId": "run-sync-fortuitous_pirate-usda-quickstats-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "commodity": { "title": "Commodity", "type": "string", "description": "Agricultural commodity to search for (e.g., CORN, SOYBEANS, WHEAT, CATTLE)", "default": "CORN" }, "year": { "title": "Year", "minimum": 1900, "maximum": 2030, "type": "integer", "description": "Year for data (leave empty for all years)" }, "state": { "title": "State", "type": "string", "description": "US state abbreviation (e.g., IA, IL, NE) - leave empty for national data" }, "statisticCategory": { "title": "Statistic Category", "type": "string", "description": "Type of statistic (e.g., AREA PLANTED, PRODUCTION, YIELD)" }, "limit": { "title": "Maximum Results", "minimum": 1, "maximum": 50000, "type": "integer", "description": "Maximum number of records to return", "default": 100 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```