# USDA Report Watcher: New WASDE & ERS Data File Alerts (`fayoussef/usda-report-watcher`) Actor

Get every new or updated USDA data file the moment it is published: WASDE, ERS, NASS, AMS. Detects files still marked 'not yet available', downloads XLSX, XLS, CSV and PDF, and reads spreadsheets into JSON tables. Schedule it on report days.

- **URL**: https://apify.com/fayoussef/usda-report-watcher.md
- **Developed by:** [Youssef Farhan](https://apify.com/fayoussef) (community)
- **Categories:** Automation, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $10.00 / 1,000 new or updated files

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## USDA Report Watcher: New WASDE & ERS Data File Alerts

Get every **new or updated USDA data file the moment it is published**: the WASDE report, ERS data products, NASS and AMS releases, and anything on a USDA page with download links. Each new XLSX, XLS or CSV is downloaded and **read into JSON tables**, so the numbers are ready for your model or spreadsheet without opening Excel.

### Why this watcher

- **Knows when a file is really out.** USDA publishes report links before the release time, and until then a link like `wasde1126.xls` returns a "This file is not yet available" page instead of the file. Most scrapers save that page as if it were the report. This Actor marks it **PENDING** and reports it as **NEW** only once the real file is live.
- **Catches silent updates.** ERS replaces files in place and bumps a `?v=` version; usda.gov changes the file's ETag and date. Both are reported as **UPDATED**.
- **Data, not just links.** Every sheet of a new spreadsheet is saved as JSON (up to 50,000 rows per file), and the original file is kept as an exact copy of what USDA published that day.
- **Cheap to schedule.** Unchanged files produce no row and no download: a quiet run only checks headers.

### Statuses

| Status | Meaning |
|---|---|
| **NEW** | A file that was not there before, or a PENDING file that is now released |
| **UPDATED** | Same file, new version, date or size |
| **PENDING** | The link is published but the file is not out yet |
| **BASELINE** | First run: the files already on the page are recorded, not downloaded |

### How to use it

1. Add the USDA pages to watch, one per line. Use the page that lists the download links, for example:
   - WASDE: `https://www.usda.gov/about-usda/general-information/staff-offices/office-chief-economist/commodity-markets/wasde-report`
   - ERS Wheat Data: `https://www.ers.usda.gov/data-products/wheat-data`
   - WASDE archive on ESMIS: `https://usda.library.cornell.edu/concern/publications/3t945q76s`
2. Run once to record the baseline.
3. Add a **Schedule**: hourly is plenty, or every 10 minutes around a known release time (WASDE is out at 12:00 ET on its release day). Connect an integration (email, Slack, webhook) to get the NEW rows sent to you.

Pages must be on `usda.gov` or one of its agencies' sites, or on `usda.library.cornell.edu`. Use **Watchlist name** to keep separate memories for separate sets of pages.

### Output example

```json
{
  "status": "NEW",
  "fileName": "wasde1026.xls",
  "fileType": "xls",
  "linkText": "Excel",
  "fileUrl": "https://www.usda.gov/oce/commodity/wasde/wasde1026.xls",
  "pageTitle": "WASDE Report | USDA",
  "detectedAt": "2026-10-09T16:00:04+00:00",
  "firstSeenAt": "2026-10-06T14:20:11+00:00",
  "lastModified": "Fri, 09 Oct 2026 15:44:22 GMT",
  "sizeBytes": 354304,
  "storedFileUrl": "https://api.apify.com/v2/key-value-stores/.../records/506087d3c1-wasde1026.xls",
  "tablesUrl": "https://api.apify.com/v2/key-value-stores/.../records/506087d3c1-wasde1026.xls-tables.json",
  "sheetNames": ["WASDE Text", "Page 8", "Page 9", "Page 10"],
  "rowCounts": {"WASDE Text": 25, "Page 8": 55, "Page 9": 52, "Page 10": 62}
}
```

The tables JSON maps each sheet name to its rows: `{"Page 9": [["Total Grains 4/", "2024/25", null, 2405.84, ...], ...]}`.

### Pricing

Pay per event:

| Event | What triggers it |
|---|---|
| Run start | Each run, so a frequent schedule stays fair for both sides |
| New or updated file | Each NEW or UPDATED file delivered, with its download and tables |

BASELINE and PENDING rows are free, and a run where nothing changed charges only the run start. The current rates are shown on this Actor's Store page.

- **Free plan:** up to 2 pages per run. A [paid Apify plan](https://apify.com/pricing?fpr=youssef) watches every report you follow.

### FAQ

**Is this an official USDA product?** No. It reads public USDA web pages and files.

**Does it read PDFs?** PDFs are detected, downloaded and saved; tables are read from XLSX, XLS and CSV.

**What about a page that loads its links with JavaScript?** The Actor reads the page as served. If a page lists no files, the run log says so; point it at the page with the download links.

💼 Need a custom version or another site? [Contact me through my Apify profile](https://apify.com/fayoussef)

# Actor input Schema

## `pages` (type: `array`):

One per line: a USDA page that lists the download links of a report or data product (usda.gov and its agencies such as ERS, NASS, AMS, FAS, or usda.library.cornell.edu).

## `fileTypes` (type: `array`):

Which download links count. Links to other file types on the page are ignored.

## `extractTables` (type: `boolean`):

For each new or updated XLSX, XLS or CSV file, every sheet is read into rows and saved as JSON, linked from the result as tablesUrl (up to 50,000 rows per file).

## `saveFiles` (type: `boolean`):

Stores each new or updated file in the run's key-value store and links it as storedFileUrl, so you keep the exact version USDA published that day.

## `reportExistingAsNew` (type: `boolean`):

Off: the first run only records what is on each page (free, no downloads). On: the files already on the page are delivered as NEW in the first run, with downloads and tables.

## `watchlistName` (type: `string`):

Optional label for this watchlist's memory. Use a different name for each set of pages you track separately.

## Actor input object example

```json
{
  "pages": [
    "https://www.usda.gov/about-usda/general-information/staff-offices/office-chief-economist/commodity-markets/wasde-report",
    "https://www.ers.usda.gov/data-products/wheat-data",
    "https://usda.library.cornell.edu/concern/publications/3t945q76s"
  ],
  "fileTypes": [
    "xlsx",
    "xls",
    "csv",
    "pdf",
    "xml",
    "txt",
    "zip"
  ],
  "extractTables": true,
  "saveFiles": true,
  "reportExistingAsNew": false
}
```

# Actor output Schema

## `files` (type: `string`):

Status (NEW, UPDATED, PENDING or BASELINE), file link, saved copy, JSON tables and the page each file was found on.

## `storage` (type: `string`):

The downloaded files and their JSON tables.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "pages": [
        "https://www.usda.gov/about-usda/general-information/staff-offices/office-chief-economist/commodity-markets/wasde-report",
        "https://www.ers.usda.gov/data-products/wheat-data",
        "https://usda.library.cornell.edu/concern/publications/3t945q76s"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("fayoussef/usda-report-watcher").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "pages": [
        "https://www.usda.gov/about-usda/general-information/staff-offices/office-chief-economist/commodity-markets/wasde-report",
        "https://www.ers.usda.gov/data-products/wheat-data",
        "https://usda.library.cornell.edu/concern/publications/3t945q76s",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("fayoussef/usda-report-watcher").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "pages": [
    "https://www.usda.gov/about-usda/general-information/staff-offices/office-chief-economist/commodity-markets/wasde-report",
    "https://www.ers.usda.gov/data-products/wheat-data",
    "https://usda.library.cornell.edu/concern/publications/3t945q76s"
  ]
}' |
apify call fayoussef/usda-report-watcher --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,fayoussef/usda-report-watcher"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/BnGTHNtalEnqhs2Am/builds/RzdYDhXmBdiGsDgfI/openapi.json