# Federal Register Data Scraper (Rules & Notices JSON) (`steadfetch/federal-register-monitor`) Actor Pull US Federal Register documents from the official public API: one JSON item per new document with document number, title, type, publication date, agencies, citation, docket IDs and links. Filter by date, type, agency or keyword. Deduplicated. For regulatory and compliance monitoring. - **URL**: https://apify.com/steadfetch/federal-register-monitor.md - **Developed by:** [Steadfetch](https://apify.com/steadfetch) (community) - **Categories:** News, Business - **Stats:** 2 total users, 1 monthly users, 33.3% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $2.00 / 1,000 document delivereds This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Federal Register Data Scraper (Rules & Notices JSON) Monitor US federal rulemaking documents from the official, keyless [Federal Register API](https://www.federalregister.gov/developers/documentation/api/v1). Filter by publication date, document type, agency or keyword, and get one structured dataset item per document. No browser, no login, no API key. ### What it does The Actor calls `https://www.federalregister.gov/api/v1/documents.json`, the Federal Register's official developer API, and returns one dataset item per matching document with its document number, type, publication date, publishing agencies, abstract, citation, effective and comment dates, docket and regulation identifiers, and official HTML/PDF links. - **Monitor mode (default):** keeps a per-filter deduplication state in the Actor's key-value store and emits only documents whose `document_number` was never delivered by an earlier run with the same filters. Re-running the same monitor never bills the same document twice. - **Search mode:** emits every document in the selected window, deduplicated within the run. Use it for one-off research or backfills. - **Filters:** publication date range (`since`/`until`, default last 7 days), document type (`RULE`, `PRORULE`, `NOTICE`, `PRESDOCU`), agency (official slug or human name, e.g. `securities-and-exchange-commission` or `Securities and Exchange Commission`) and full-text keyword. - **Schema validation:** every item is validated against `.actor/dataset_schema.json` before it reaches the dataset. With `strictSchema: true` (the default) an invalid item fails the run instead of pushing malformed data. The Federal Register states that programmatic access to its sites is limited to its developer APIs, so this Actor **only** uses the official API and never scrapes the HTML site. The data is public US government information. The Actor returns documents and agencies only: no personal data and no contact fields. ### Example output ```json { "url": "https://www.federalregister.gov/documents/2026/10/20/C1-2026-20360/interval-fund-modernization-expansion-of-multiple-share-class-to-registered-closed-end-management", "title": "Interval Fund Modernization; Expansion of Multiple Share Class to Registered Closed-End Management Investment Companies and Business Development Companies", "scrapedAt": "2026-10-20T08:30:00.000Z", "document_number": "C1-2026-20360", "type": "Proposed Rule", "publication_date": "2026-10-20", "agencies": ["Securities and Exchange Commission"], "abstract": "", "citation": "91 FR 66293", "effective_on": "2026-10-05", "comments_close_on": "2026-12-04", "docket_ids": ["Release No. 33-11444", "34-106534", "IC-36351", "File No. S7-2026-34"], "regulation_id_numbers": ["3235-AN83"], "html_url": "https://www.federalregister.gov/documents/2026/10/20/C1-2026-20360/interval-fund-modernization-expansion-of-multiple-share-class-to-registered-closed-end-management", "pdf_url": "https://www.govinfo.gov/content/pkg/FR-2026-10-20/pdf/C1-2026-20360.pdf", "source_url": "https://www.federalregister.gov/api/v1/documents/C1-2026-20360.json" } ``` `source_url` is the official per-document API endpoint for the record. Optional dates (`effective_on`, `comments_close_on`) and `pdf_url` are omitted when the source has no value; `abstract` and `citation` are empty strings in that case, and identifier arrays are empty. ### Input Minimal run (monitor mode, last 7 days by default): ```json { "since": "2026-10-01", "maxResults": 10 } ``` A focused monitor: ```json { "mode": "monitor", "since": "2026-10-01", "type": "PRORULE", "agency": "Securities and Exchange Commission", "keyword": "fund", "maxResults": 500 } ``` | Field | Type | Description | | --- | --- | --- | | `mode` | `monitor` or `search` | Default `monitor`: only document numbers not delivered by earlier runs with the same filters. `search`: every matching document in the window (deduplicated within the run). | | `since` | string | Earliest publication date (`YYYY-MM-DD`, inclusive). Defaults to 7 days before the run date (UTC). | | `until` | string | Latest publication date (`YYYY-MM-DD`, inclusive). Defaults to the run date (UTC). | | `type` | `RULE`, `PRORULE`, `NOTICE` or `PRESDOCU` | Optional document type. | | `agency` | string | Optional agency slug or name. Names are resolved against the official `/agencies` endpoint. | | `keyword` | string | Optional full-text search term. | | `maxResults` | integer | Default 500, maximum 5000. Paging stops as soon as that many new documents are collected. | | `strictSchema` | boolean | Default `true`; fail the run on schema violations. | The monitor state is keyed by the filter signature (`type`, `agency`, `keyword`, case-insensitive); the date window and `maxResults` are deliberately excluded, so widening a date range does not re-deliver documents already seen by the same monitor. ### Project structure ``` .actor/actor.json Actor definition (payPerEvent, dataset schema, output schema) .actor/input_schema.json INPUT options (mode, since, until, type, agency, keyword, maxResults) .actor/dataset_schema.json Item JSON Schema (fields) + table view for the Store .actor/pay_per_event.json PPE pricing (two synthetic platform events) src/federal-register-client.ts Federal Register API client: filters, pagination, retries, agency lookup src/extract.ts Pure mapping and deduplication (fixture-tested, no network) src/monitor-state.ts Pure monitor-state key and merge rules src/main.ts Entry point: fetch, validate, push, persist monitor state src/validate-output.ts Ajv validator shared by the run and the tests test/ Offline JSON fixtures and Vitest suites ``` ### Pay-per-event pricing The Actor uses Apify **pay-per-event** pricing with the two synthetic platform events: | Event | Charged when | Price | | --- | --- | --- | | `apify-actor-start` | a run starts (synthetic, automatic) | 0.00005 USD per run (Apify-recommended default; includes 5 seconds of free compute) | | `apify-default-dataset-item` | a new, validated document is saved to the dataset (synthetic, automatic) | 0.002 USD per document = **2.00 USD per 1,000 documents** | There is no custom per-item event and no `Actor.charge()` call: the platform already charges `apify-default-dataset-item` per pushed item, so adding another event would bill users twice. Empty results, API errors, retries and documents skipped by monitor deduplication are never charged. The price sits at the top of the direct-competitor band on the Apify Store (0.001-0.002 USD per document = 1-2 USD per 1,000) and inside Steadfetch's 1-10 USD band, justified by complete API coverage and cross-run deduplication. #### Cost per 1,000 documents A 1,000-document run needs one JSON request (`per_page` 1000), a few seconds of a 256 MB container and no proxy. Measured locally with `apify run`: 1,000 documents fetched, mapped, schema-validated and written in **7.5 seconds** end to end (including Node/CLI startup), using 256 MB. At Apify compute rates that is roughly **0.0001 USD of compute plus the 0.00005 USD run-start event per 1,000 documents**, well below **0.01 USD per 1,000**. At the 2.00 USD price the gross margin is above 99%. ### Development ```bash npm install npm test # lint + typecheck + offline Vitest suites (no credit) npm run publish:check # Steadfetch publish gate npm run schemas:validate # apify-cli schema validation npm run smoke # local Apify run against the live API (no platform credit) ``` The default test suite runs against committed API fixtures and never touches the network or the Apify platform. It covers the real API response shape, pagination, agency-name resolution, monitor deduplication, malformed records and schema validation. ### FAQ **Do I need a Federal Register API key?** No. The documents API is keyless and public. **Does it need a browser?** No. The Federal Register returns structured JSON; plain HTTP is roughly 10x cheaper and faster than a browser-based Actor. **How does monitor deduplication work?** The Actor stores the `document_number` of every delivered document in a named key-value store that persists between runs (the default store is run-scoped in Apify), under a key derived from the `type`/`agency`/`keyword` filters. The next run with the same filters filters those numbers out before counting toward `maxResults`, so only new documents are pushed and billed. The state keeps the 20,000 most recent document numbers. **Does it return personal data?** No. Only documents and agencies are returned. There are no personal names, email addresses or phone numbers in the output schema. **When am I charged?** Once per run start (`apify-actor-start`) and once per new document saved to the dataset (`apify-default-dataset-item`). Failed requests, retries, empty results and documents already delivered by a previous monitor run are not charged. **Is the output schema stable?** Yes. New fields are additive and the required fields in `.actor/dataset_schema.json` do not change. ### Maintenance This Actor is **tested daily**: automated runs check that the official Federal Register API is reachable and that the output still matches the published schema. Failures are reported as issues in the Steadfetch Actors repository. # Actor input Schema ## `mode` (type: `string`): monitor: emit only documents not delivered by previous runs with the same type/agency/keyword filters (deduplicated by document_number). search: emit every matching document in the window, deduplicated within the run. Both modes use the same official API. ## `since` (type: `string`): Earliest publication date (YYYY-MM-DD, inclusive). Defaults to 7 days before the run date (UTC). ## `until` (type: `string`): Latest publication date (YYYY-MM-DD, inclusive). Defaults to the run date (UTC). ## `type` (type: `string`): Optional Federal Register document type: RULE (final rule), PRORULE (proposed rule), NOTICE or PRESDOCU (presidential document). ## `agency` (type: `string`): Optional agency slug or name, for example "securities-and-exchange-commission" or "Securities and Exchange Commission". ## `keyword` (type: `string`): Optional full-text search term (for example "fund" or "emissions"). ## `maxResults` (type: `integer`): Maximum number of documents to save to the dataset. ## `strictSchema` (type: `boolean`): Fail the run when an item does not match the OUTPUT schema. Keep enabled: Steadfetch never publishes Actors without output-schema validation. ## Actor input object example ```json { "mode": "search", "since": "2026-10-01", "until": "2026-10-20", "agency": "securities-and-exchange-commission", "maxResults": 10, "strictSchema": true } ``` # Actor output Schema ## `results` (type: `string`): Dataset items validated against the item schema in dataset_schema.json (fields). # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "search", "since": "2026-10-01", "until": "2026-10-03", "maxResults": 10, "strictSchema": true }; // Run the Actor and wait for it to finish const run = await client.actor("steadfetch/federal-register-monitor").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "search", "since": "2026-10-01", "until": "2026-10-03", "maxResults": 10, "strictSchema": True, } # Run the Actor and wait for it to finish run = client.actor("steadfetch/federal-register-monitor").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "search", "since": "2026-10-01", "until": "2026-10-03", "maxResults": 10, "strictSchema": true }' | apify call steadfetch/federal-register-monitor --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,steadfetch/federal-register-monitor" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/FRm9ryrIPeRLjcZ32/builds/G3GRv2bPGaLJ0roin/openapi.json