# US Agricultural Market Trend Monitor (`zeekr011/us-agricultural-market-trend-monitor`) Actor

Query and monitor public USDA AMS agricultural market bids with a bounded daily/weekly fallback chain and normalized, traceable Dataset rows.

- **URL**: https://apify.com/zeekr011/us-agricultural-market-trend-monitor.md
- **Developed by:** [hugo liu](https://apify.com/zeekr011) (community)
- **Categories:**
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $4.00 / 1,000 result items

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## US Agricultural Market Trend Monitor

一个面向农产品贸易商、食品加工企业和农业 SaaS 开发者的 Apify Actor MVP。它读取公开/免费的 USDA AMS 市场报告，输出可追溯、字段统一的美国农产品现货报价，并支持一次性查询和增量监控。

当前版本的重点是“稳定、可解释、可回溯”，不是实时盘口、交易建议或完整历史数据库。

### Before you run

- 本版本不需要 USDA MyMarketNews API Key、USDA NASS Quick Stats API Key 或 NOAA Key。
- Actor 默认通过 Apify Proxy、US 地理位置访问 USDA；如需直接诊断上游，可在 `proxyConfiguration` 中关闭代理。代理流量可能产生额外 Apify Proxy 费用。
- Apify 的计算、Dataset 和存储费用仍按账户套餐规则计算；大范围报告和较大的 `maxResults` 会增加运行时间与资源消耗。
- 从一个报告 ID、一个品类和较小的 `maxResults` 开始。`maxResults` 限制标准化后的输出，不等于限制上游下载字节数。
- 数据是上游发布时点的报告数据，不承诺实时刷新、交易级可成交性或 SLA。
- Actor 不维护永久历史库。需要长期历史时，应将 Dataset 复制到自己的数据库或数据仓库。

### 首版数据源与 fallback

每日和每周严格分开，不能在一次运行中混用两个频率的报告。

#### DAILY

| 权重 | 来源 | 传输 | 语义 |
| ---: | --- | --- | --- |
| 100 | USDA AMS MyMarketNews Public Data JSON | `PUBLIC_JSON` | 首选结构化报价 |
| 90 | USDA AMS 官方报告 PDF | `PDF` | JSON 无有效报价时的官方回退 |

执行顺序是：Public Data JSON → 官方 PDF 文本解析 → 仍失败则该报告没有可用记录并给出 warning。

#### WEEKLY

| 权重 | 来源 | 传输 | 语义 |
| ---: | --- | --- | --- |
| 100 | USDA AMS 官方周报 PDF | `PDF` | 周报主来源 |
| 80 | USDA Open Ag Transport Dataset | `DATASET_JSON` | 周度趋势上下文，不伪装成 AMS 周报替代品 |

周度 Open Ag Transport 数据来自 [USDA Open Ag Transport](https://agtransport.usda.gov/resource/g92w-8cn7.json)，字段和覆盖可能与所选 AMS 报告不同，输出的 `parseWarnings` 会标记 `trend_context_only`。

#### PDF 是否需要 OCR

已验证的 USDA AMS 报告是带文字元素和字体资源的 Reporting Services PDF，价格、basis、日期和表头可以直接提取；页眉中的 JPEG 只是标识图，不是整页扫描。因此 P0 使用 `pdfjs-dist` 文本提取，不引入 OCR。

只有 PDF 没有可提取文字，或没有报告标题/日期/报价表头时，才会标记 `needs_ocr` warning。当前免费 MVP 不自动调用 OCR，也不会把图片猜成报价。

### 已登记的报告

Actor 不默认无界扫描 AMS 报告目录，只接受内置登记的、已验证过的报告 ID。可以在 `marketReportIds` 中填写多个同频率 ID。

#### DAILY 报告

| ID | 报告 | 州/范围 |
| ---: | --- | --- |
| 2850 | Iowa Daily Cash Grain Bids | IA |
| 3192 | Illinois Grain Bids | IL |
| 2886 | Kansas Daily Grain Bids | KS |
| 3225 | Nebraska Daily Elevator Grain Bids | NE |
| 2960 | Arkansas Daily Grain Bids | AR |
| 2892 | Kentucky Daily Grain Bids | KY |
| 3049 | Southern Minnesota Daily Grain Bids | MN |
| 2928 | Mississippi Daily Grain Bids | MS |
| 2771 | Montana Daily Elevator Grain Bids | MT |
| 3156 | North Carolina Cash Grain Bids | NC |
| 3878 | North Dakota Daily Grain Bids | ND |
| 2851 | Ohio Daily Grain Bids | OH |
| 3100 | Oklahoma Daily Grain Bids | OK |
| 2787 | South Carolina Daily Grain Bids | SC |
| 3186 | South Dakota Daily Grain Bids | SD |
| 3088 | Tennessee Daily Grain Bids | TN |
| 2711 | Texas Daily Grain Bids | TX |
| 3167 | Virginia Daily Grain Bids | VA |
| 3239 | Wyoming Daily Grain Bids | WY |
| 2932 | Missouri Daily Grain Bids | MO |
| 3043 | Iowa-Southern Minnesota Barge Terminal Grain Bids | Barge |
| 3147 | Louisiana and Texas Export Bids | Export |
| 2887 | National Daily Sunflower, Canola, Millet, and Flaxseed Report | National |

不填写 `marketReportIds` 时，DAILY 默认使用 `2850`。

#### WEEKLY 报告

| ID | 报告 | 州 |
| ---: | --- | --- |
| 3146 | California Grain Bids | CA |
| 3463 | Indiana Grain Bids | IN |
| 2714 | Maryland Grain Bids | MD |
| 3091 | Pennsylvania Grain Bids | PA |

不填写 `marketReportIds` 时，WEEKLY 默认使用 `3146`。

报告页面示例：[AMS report 2850](https://mymarketnews.ams.usda.gov/viewReport/2850)，PDF 示例：[AMS\_2850.pdf](https://www.ams.usda.gov/mnreports/ams_2850.pdf)。

### P0 品类

默认允许以下六类，保留官方原始商品名，同时生成统一的 `commodityCode`：

| 代码 | 常见原始名称 |
| --- | --- |
| `CORN` | Corn、Maize、Yellow Corn |
| `SOYBEANS` | Soybean、Soybeans |
| `WHEAT` | Wheat、Hard Red Winter Wheat |
| `SORGHUM` | Sorghum、Milo |
| `BARLEY` | Barley |
| `OATS` | Oats、White Oats |

国家报告还可能出现 `CANOLA`、`SUNFLOWER`、`MILLET` 和 `FLAXSEED`，标准化器可以保留这些代码；用户可以通过 `commodities` 明确筛选它们。首版不做 bushel、ton、lb、cwt 之间的隐式换算。

### Run modes

| 模式 | 输出 | 状态 |
| --- | --- | --- |
| `lookup` | 当前匹配记录，`changeType=CURRENT` | 不读取、不修改监控状态 |
| `export` | 有界当前数据集，`changeType=CURRENT` | 不读取、不修改监控状态 |
| `monitor` | 新记录 `ADDED`，内容变化 `UPDATED` | 使用命名 KVS 检查点 |

“一次性查询”和“持续监控”的区别在于：一次性查询只回答本次运行上游有什么；持续监控会把本次标准化快照与上一次快照比较，并只投递新增/变化记录。`monitor` 首次运行可以设置 `emitInitialSnapshot=false`，只建立状态而不输出初始全量记录。

监控投递顺序为：读取旧快照 → 计算差异 → 成功写入 Dataset → 写入 KVS checkpoint。若 Dataset 写入成功而 checkpoint 写入失败，下一次可能重复投递；请使用 `idempotencyKey` 去重。进程内锁只覆盖同一个 Node.js 进程，不能宣称跨容器 exactly-once 或账户级限流。

### Input API

```json
{
  "mode": "lookup",
  "frequency": "DAILY",
  "marketReportIds": ["2850"],
  "commodities": ["CORN", "SOYBEANS"],
  "states": ["IA"],
  "includeFuturesSettlements": false,
  "priceChangeThresholdPercent": 5,
  "maxResults": 20
}
```

| 字段 | 类型/默认值 | 说明 |
| --- | --- | --- |
| `mode` | `lookup` | `lookup`、`export` 或 `monitor` |
| `frequency` | `DAILY` | `DAILY` 或 `WEEKLY`，不可混用 |
| `marketReportIds` | `[]` | 已登记报告 ID；空值使用该频率默认报告 |
| `commodities` | 六个 P0 品类 | 本地商品筛选 |
| `states` | `[]` | 可选美国两位州代码；空值表示所选报告覆盖的州 |
| `startDate` / `endDate` | 无 | 包含边界的 `YYYY-MM-DD` 观测日期范围 |
| `includeFuturesSettlements` | `false` | 从 PDF 中额外输出期货结算曲线 |
| `priceChangeThresholdPercent` | `5` | 可填写任意大于 0 且不超过 100 的数值，例如 `2.75` |
| `maxResults` | `500` | 1–5000；作用于标准化、过滤、去重之后 |
| `emitInitialSnapshot` | `true` | 仅 monitor 生效 |
| `monitorId` | 无 | monitor 必填，用于隔离不同监控任务 |
| `stateStoreName` | `us-agricultural-market-trend-state` | 命名 KVS |
| `proxyConfiguration` | Apify Proxy / US | USDA 请求的代理配置；默认启用，可在 Console 中关闭或选择代理组 |

#### 当前查询

```json
{
  "mode": "lookup",
  "frequency": "DAILY",
  "marketReportIds": ["2850"],
  "commodities": ["CORN"],
  "maxResults": 20
}
```

#### 周度查询

```json
{
  "mode": "export",
  "frequency": "WEEKLY",
  "marketReportIds": ["3146"],
  "commodities": ["CORN", "SORGHUM"],
  "startDate": "2026-01-01",
  "endDate": "2026-12-31",
  "maxResults": 100
}
```

#### 增量监控

```json
{
  "mode": "monitor",
  "frequency": "DAILY",
  "marketReportIds": ["2850", "3192"],
  "commodities": ["CORN", "SOYBEANS"],
  "states": ["IA", "IL"],
  "emitInitialSnapshot": false,
  "monitorId": "midwest-grains-daily",
  "stateStoreName": "us-agricultural-market-trend-state"
}
```

### Unified output fields

每个 Dataset item 都是一个扁平标准化报价。不同来源的同义字段先在适配层映射到同一字段，再做过滤、去重和监控。

#### 来源与身份

| 字段 | 说明 |
| --- | --- |
| `source` | `usda-ams-market-news` 或 `usda-open-ag-transport` |
| `sourceRecordId` | 由来源、报告、频率、日期、品类、市场组、地区、等级、报价类型和交付条件组成的稳定 ID |
| `sourceFamily` | `USDA_AMS` 或 `USDA_OPEN_AG_TRANSPORT` |
| `sourceTransport` | `PUBLIC_JSON`、`PDF` 或 `DATASET_JSON` |
| `sourcePriority` | 100、90 或 80；用于 fallback 和去重 |
| `sourceReportId` / `sourceReportName` | 官方报告标识和名称 |
| `sourceUrl` | 实际读取的官方端点或 PDF |

#### 时间、市场和商品

`observationDate`、`periodStart`、`periodEnd`、`weekEnding`、`publishedAt`、`quoteAsOf`、`retrievedAt`；
`marketType`、`marketName`、`marketGroup`、`marketStateCode`、`marketCity`、`facilityName`、`facilityType`；
`commodityCode`、`commodityName`、`commodityRawName`、`grade`、`className`、`protein`、`packageType`。

#### 报价与交付

`quoteSide`（`BUYER_BID`、`SELLER_OFFER`、`SETTLEMENT`）、`quoteType`、`priceValue`、`priceMin`、`priceMax`、`priceAverage`、`priceYearAgo`、`priceCurrency`、`priceUnit`、`basisValue`、`basisMin`、`basisMax`、`basisUnit`、`basisFuturesMonth`、`priceChange`、`priceChangeMin`、`priceChangeMax`、`priceChangePercent`、`priceDirection`、`basisChange`、`basisChangeMin`、`basisChangeMax`、`basisDirection`、`deliveryPoint`、`deliveryStatus`、`deliveryStart`、`deliveryEnd`、`freight`、`transportMode`。

范围值不会被强行压成一个数。例如 `4.8975-5.0875` 同时保留 `priceMin=4.8975`、`priceMax=5.0875`；只有官方提供平均值时才填 `priceAverage`。不同单位不做隐式换算。

#### 状态与解析

`reportStatus`、`isFinal`、`dataStatus`、`extractionMethod`、`parserVersion`、`parseWarnings`、`signal`；监控模式另外包含 `changeType`、`changedFields`、`detectedAt`、`contentHash` 和 `idempotencyKey`。

示例输出（字段已截短）：

```json
{
  "source": "usda-ams-market-news",
  "sourceRecordId": "usda_ams:2850:daily:market_price:2026_09_03:corn:country_elevator:ia:northwest:na:us_2:na:bid:current:na:na",
  "recordType": "market_price",
  "frequency": "DAILY",
  "sourceTransport": "PDF",
  "sourcePriority": 90,
  "sourceReportId": "2850",
  "sourceReportName": "Iowa Daily Cash Grain Bids",
  "sourceUrl": "https://www.ams.usda.gov/mnreports/ams_2850.pdf",
  "observationDate": "2026-09-03",
  "marketName": "Northwest",
  "marketStateCode": "IA",
  "commodityCode": "CORN",
  "commodityRawName": "US #2 Yellow Corn (Bulk)",
  "grade": "US #2",
  "quoteSide": "BUYER_BID",
  "priceValue": 4.9661,
  "priceMin": 4.8975,
  "priceMax": 5.0875,
  "priceUnit": "$/bu",
  "basisMin": -51,
  "basisMax": -32,
  "basisUnit": "cents_per_bushel",
  "priceChange": -0.0275,
  "priceChangePercent": -0.5507,
  "priceDirection": "down",
  "dataStatus": "fallback",
  "extractionMethod": "pdf_text",
  "changeType": "CURRENT"
}
```

### 规则信号

- `price_spike`：有可用前值，价格相对前值上涨达到 `priceChangeThresholdPercent`。
- `price_drop`：有可用前值，价格相对前值下跌达到阈值。
- 没有官方前值时不猜测涨跌，也不输出季节性异常结论。
- `futures_settlement` 是报告中明确的结算价，不等于现货报价。

### Apify Console、CLI 和 REST

Console 中选择 Input，指定 `mode`、`frequency`、报告 ID 和小的 `maxResults` 后运行。Dataset 的 JSON、CSV、Excel 都来自同一个 Dataset：

Output Schema 的模板只使用 `/items` 和 `format` 参数，避免把 `clean`、`attachment` 这类布尔查询参数放进 Console 模板；Apify Console 的模板解析器会把 URL 查询值当作字符串。直接调用 REST API 时，可以按下面示例传递 `clean=true` 和 `attachment=true`。

```bash
apify call <ACTOR_ID> \
  --input '{"mode":"lookup","frequency":"DAILY","marketReportIds":["2850"],"commodities":["CORN"],"maxResults":20}' \
  --output-dataset
```

```bash
curl -X POST \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  --data '{"mode":"lookup","frequency":"DAILY","marketReportIds":["2850"],"commodities":["CORN"],"maxResults":20}' \
  "https://api.apify.com/v2/acts/<ACTOR_ID>/runs?waitForFinish=60"
```

取得 `defaultDatasetId` 后：

```text
GET https://api.apify.com/v2/datasets/<DATASET_ID>/items?format=json&clean=true
GET https://api.apify.com/v2/datasets/<DATASET_ID>/items?format=csv&clean=true&attachment=true
GET https://api.apify.com/v2/datasets/<DATASET_ID>/items?format=xlsx&clean=true&attachment=true
```

持续监控可使用 Apify Schedule 触发 `monitor`，使用 Apify Platform Webhook 接收 run 生命周期事件，再读取该 run 的 Dataset。当前 Actor 不自行发送邮件、短信或 Slack，也不在容器内生成二进制 Excel 文件。

### Reliability and limitations

- HTTP 客户端设置超时、最小请求间隔和有限重试；重试网络异常、超时、429 和 5xx，并尊重 `Retry-After`。
- 其他 4xx、错误 JSON 和无法解析的 PDF 不会被静默当成“没有数据”；运行会产生 warning，必要时返回空 Dataset 或失败。
- Public JSON 与 PDF 的字段可能有缺失；未知值使用 `null`，原始商品名和 `parseWarnings` 尽量保留。
- PDF fallback 会降低结构化字段完整度；`sourceTransport`、`dataStatus` 和 `extractionMethod` 可用于下游质量判断。
- 记录缺席不产生 `REMOVED`，因为滚动报告窗口或上游分页缺席不能证明官方删除。
- 不把地方现货报告、Open Ag 周度上下文、期货结算价和实时盘口混成同一种报价。

### Local development

要求 Node.js 20+：

```bash
npm install
npm test
npm run build
apify validate-schema
python3 /Users/hugo/.codex/skills/apify-actor-deployer/scripts/preflight.py . --profile generic --require-build
```

有界真实源 smoke：

```bash
npm run test:real
```

真实 smoke 只读取一个报告、一个品类和有限行数；不会做全美无界导出。Store 发布、Schedule、代理和付费 API Key 均不属于当前版本。

### Attribution

本 Actor 使用 USDA AMS、USDA Open Ag Transport 的公开数据。请保留 `sourceUrl` 和来源标识，并在大规模运行前阅读上游使用说明。数据不构成投资、采购、保险、农业或法律建议。

# Actor input Schema

## `mode` (type: `string`):

lookup/export return current records without touching monitor state; monitor emits only new or changed records.

## `frequency` (type: `string`):

DAILY and WEEKLY are separate source contracts and cannot be mixed in one run.

## `marketReportIds` (type: `array`):

Optional verified report IDs. If empty, DAILY defaults to 2850 and WEEKLY defaults to 3146. Maximum 25 IDs from the built-in registry.

## `commodities` (type: `array`):

Commodity names or codes. P0 includes corn, soybeans, wheat, sorghum, barley and oats; source rows outside this list are filtered locally.

## `states` (type: `array`):

Optional two-letter state filters. Empty means all states represented by the selected reports; it does not scan unregistered reports.

## `startDate` (type: `string`):

Optional inclusive observation date in YYYY-MM-DD format.

## `endDate` (type: `string`):

Optional inclusive observation date in YYYY-MM-DD format.

## `includeFuturesSettlements` (type: `boolean`):

PDF reports often contain a futures settlement curve. These are output as futures\_settlement rows, separate from cash bids.

## `priceChangeThresholdPercent` (type: `number`):

Continuous numeric threshold used for the optional price\_spike/price\_drop signal. This is a rule signal, not investment advice.

## `maxResults` (type: `integer`):

Maximum normalized rows emitted after source retrieval, local filtering and source-priority deduplication. Keep it small for the first run.

## `emitInitialSnapshot` (type: `boolean`):

Only applies to monitor. When false, the first run seeds the checkpoint and emits an empty Dataset.

## `monitorId` (type: `string`):

Stable namespace for a recurring monitor. Required in monitor mode.

## `stateStoreName` (type: `string`):

Named Apify Key-Value Store for monitor fingerprints. Reuse it with the same monitor scope.

## `proxyConfiguration` (type: `object`):

Routes USDA requests through Apify Proxy. Enabled by default with US geolocation. Proxy traffic may incur Apify Proxy charges.

## Actor input object example

```json
{
  "mode": "lookup",
  "frequency": "DAILY",
  "commodities": [
    "CORN",
    "SOYBEANS",
    "WHEAT",
    "SORGHUM",
    "BARLEY",
    "OATS"
  ],
  "states": [],
  "includeFuturesSettlements": false,
  "priceChangeThresholdPercent": 5,
  "maxResults": 500,
  "emitInitialSnapshot": true,
  "stateStoreName": "us-agricultural-market-trend-state",
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyCountry": "US"
  }
}
```

# Actor output Schema

## `json` (type: `string`):

No description

## `csv` (type: `string`):

No description

## `excel` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("zeekr011/us-agricultural-market-trend-monitor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("zeekr011/us-agricultural-market-trend-monitor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call zeekr011/us-agricultural-market-trend-monitor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,zeekr011/us-agricultural-market-trend-monitor"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/z2TKt9hyXRySDtbSs/builds/uyIJuJN3hzvo88zlV/openapi.json