Federal Register API Scraper — rules, notices & agency monitor

Query the official U.S. Federal Register public JSON API and get one normalized dataset item per document. Filter by agency slug, search term, document type (RULE / PRORULE / NOTICE / PRESDOCU), significant-only flag, and publication date — built for compliance, policy, and legal-ops monitoring.

Not affiliated with the U.S. Federal Register / NARA / OFR. Public API data only. Verify every document on the official htmlUrl before taking action. Do not use official NARA or OFR logos when republishing.

Features

• Official documents.json API — HTTP GET only; no browser, no HTML scraping
• Filters — term, agency slugs, document types, publication date range, significant-only (EO 12866)
• Normalized output — stable fields (documentNumber, title, type, agencies, publicationDate, effectiveOn, commentsCloseOn, htmlUrl, pdfUrl, abstract, significant, cfrReferences, …)
• Safety by default — empty input auto-applies a ~14-day publication window so Store Try / first runs stay bounded
• Bounded & resilient — maxItems (default 50, hard cap 1000), pagination with ~200 ms pacing, retry/backoff on HTTP 429 / 5xx, clear User-Agent
• Optional raw payload — attach the original Federal Register JSON under raw when includeRaw is true
• AI-agent ready — works as a tool in Apify's MCP server, LangChain, CrewAI and the OpenAI Agents SDK; opt-in outputFormat: "compact" for small, LLM-friendly rows (see Use with AI agents / MCP)
• Opt-in “Only new since last run”: for schedules, onlyNewSinceLastRun: true returns only documents you haven’t received yet. The watermark is kept in your own Apify account.

Who is it for

• Compliance & regulatory-affairs teams watching rules and notices by agency
• Policy / government-affairs shops tracking topics (e.g. cybersecurity, PFAS, tariffs)
• Legal-ops and research workflows that need machine-readable FR metadata + links
• Developers who want a ready-made Apify Actor instead of hand-rolling the Federal Register client

How it works

1. Open this Actor on Apify and set filters (or leave empty for the ~14-day default window).
2. Optionally set maxItems, significantOnly, includeRaw, and abstractMaxLength.
3. Run the Actor. It calls
   GET https://www.federalregister.gov/api/v1/documents.json?…
   with only documented conditions[...] parameters and selected fields[].
4. Read results from the default dataset — one row per document — or download via the dataset API URL in the run output.

Tips: Prefer filtered queries. Upstream pagination covers only the first ~2000 matches; use date bounds for large corpora. Agency slugs look like environmental-protection-agency (see agencies).

Official docs: Federal Register API v1.

Input

Field	Maps to API	Notes
term	conditions[term]	Full-text search
agencies	conditions[agencies][]	Agency ID slugs
documentTypes	conditions[type][]	RULE, PRORULE, NOTICE, PRESDOCU
publicationDateStart / publicationDateEnd	conditions[publication_date][gte] / [lte]	Prefer YYYY-MM-DD
significantOnly	conditions[significant]=1	EO 12866 significant docs only
maxItems	(client)	Default 50, max 1000
includeRaw	(client)	Attach original JSON under raw
abstractMaxLength	(client)	Default 500
onlyNewSinceLastRun	(client)	Default false. For scheduled runs: push only documents newer than the last run’s watermark (see below)
outputFormat	(client)	full (default, all fields) or compact (key fields only, abstract ≤ 200 chars — for AI agents; same price per row)

Input example

{
  "term": "cybersecurity",
  "agencies": [],
  "documentTypes": ["RULE", "PRORULE", "NOTICE"],
  "publicationDateStart": "2026-06-01",
  "publicationDateEnd": "2026-09-24",
  "significantOnly": false,
  "maxItems": 25,
  "includeRaw": false
}

Store Try / first-run uses a similar term + date window + modest maxItems so an unedited run succeeds. If you clear all filters, the Actor auto-applies a ~14-day publication window.

Only new since last run (delta mode, opt-in)

Built for scheduled / recurring runs, so you are not re-charged for documents you already received.

{ "term": "cybersecurity", "documentTypes": ["RULE", "PRORULE"], "maxItems": 50, "onlyNewSinceLastRun": true }

How it works:

1. First run with the flag = baseline. It behaves exactly like a normal run (your full filtered window, newest first, capped by maxItems). Then it saves a watermark: the newest publicationDate it pushed plus the documentNumbers already pushed on that date.
2. Later runs query the API from the watermark date onward (conditions[publication_date][gte]). They push only documents with a newer publication date, or the same date but an unseen documentNumber. Zero new documents still means a SUCCEEDED run, with an empty dataset (OUTPUT.message = "No new documents since last run").
3. The watermark is updated only after items were pushed successfully. A failed run does not move it.

Where the state lives: a named key-value store federal-register-monitor-state in the Apify account that runs the Actor (yours, not the developer’s). It holds one record per filter combination (WATERMARK-<hash of your filters>), so two schedules with different filters never share a watermark. Changing any filter starts a new baseline. To reset, delete that record (Console → Storage → Key-value stores). Turning the flag off does not delete it; the record is just ignored while the flag is off.

The run OUTPUT (default key-value store) includes onlyNewSinceLastRun, watermarkKey, baselineRun, watermarkBefore, watermarkAfter, skippedAlreadySeen, truncated.

Honest limitations:

• “New” means a newer publication date, or an unseen document number on the watermark date. Corrections or metadata edits to documents you already received are not re-emitted.
• If more new documents exist than maxItems, the oldest new ones come first and the rest arrive on the next run. Raise maxItems if you want everything at once. Each delta run's dataset is still sorted newest-first. OUTPUT.truncated = true means more new documents were waiting when the run stopped.
• Caveat for empty-filter schedules: the Federal Register publishes roughly 100 documents per business day, more than the default maxItems of 50. If maxItems is below the daily volume (or a schedule is paused for a long time), the backlog keeps growing, and anything older than the rolling ~14-day window is never backfilled. For empty-filter schedules, raise maxItems above the daily volume or narrow the filters (agency, term, document type).
• Your publicationDateStart / publicationDateEnd still apply. A schedule whose end date is in the past will eventually return nothing new. With empty filters the rolling ~14-day window is used, and the watermark key stays the same from day to day.
• Watermarks are per Apify account and per filter set. Runs from a different account, organization, or a local machine keep their own state. Two runs with identical filters started at the same time can both return the same documents.
• Off by default. Existing inputs and schedules behave exactly as before.

Output example

{
  "documentNumber": "2026-19584",
  "title": "Flight Operations: Pilot requirements; Use of oxygen",
  "type": "Proposed Rule",
  "agencies": ["Transportation Department", "Federal Aviation Administration"],
  "publicationDate": "2026-09-24",
  "effectiveOn": null,
  "commentsCloseOn": "2026-11-23",
  "htmlUrl": "https://www.federalregister.gov/documents/2026/09/24/2026-19584/…",
  "pdfUrl": "https://www.govinfo.gov/content/pkg/FR-2026-09-24/pdf/2026-19584.pdf",
  "publicInspectionPdfUrl": "https://public-inspection.federalregister.gov/2026-19584.pdf?…",
  "abstract": "FAA proposes to raise the altitudes at which a pilot is required to don an oxygen mask…",
  "significant": null,
  "topics": ["Air carriers", "Aircraft", "Aviation safety"],
  "cfrReferences": ["14 CFR part 91", "14 CFR part 135"],
  "source": "federalregister-gov"
}

Use with AI agents / MCP

This Actor works as a ready-made tool for AI agents: an agent passes a keyword, agencies, document types or dates, and gets back one JSON row per Federal Register document.

Apify MCP server. Add the Actor as a tool by listing it in the tools parameter of the hosted server (https://mcp.apify.com):

https://mcp.apify.com?tools=brightish_ascent/federal-register-monitor

MCP client config (Claude Desktop, Cursor, VS Code and other MCP clients). You sign in to Apify with OAuth on first connect:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=brightish_ascent/federal-register-monitor"
    }
  }
}

• The tool is named brightish_ascent--federal-register-monitor. Its arguments are the input fields above.
• A call returns the run status and a datasetId. The agent then reads the rows with the get-dataset-items tool (limit, offset, fields).
• With the default tool set, an agent can also run it via call-actor with actor: "brightish_ascent/federal-register-monitor".
• Local stdio server: npx @apify/actors-mcp-server --tools brightish_ascent/federal-register-monitor, with APIFY_TOKEN set.

Agent frameworks.

• LangChain: ApifyActorsTool("brightish_ascent/federal-register-monitor") from langchain-apify.
• CrewAI: ApifyActorsTool(actor_name="brightish_ascent/federal-register-monitor") from crewai_tools.
• OpenAI Agents SDK: MCPServerStreamableHttp with the MCP URL above and an Authorization: Bearer <APIFY_TOKEN> header.
• LlamaIndex: ApifyActor(token).load_data(actor_id="brightish_ascent/federal-register-monitor", run_input={...}, dataset_mapping_function=...).

The LangChain and CrewAI tools return every dataset row inline, so keep maxItems small.

Example agent prompt:

Which proposed rules has the EPA published in the last 30 days, and when do their comment periods close? Use the Federal Register tool with agencies: ["environmental-protection-agency"], documentTypes: ["PRORULE"], publicationDateStart set to the date 30 days ago (YYYY-MM-DD), maxItems: 15, outputFormat: "compact", then list title, commentsCloseOn and htmlUrl.

Cost per call. You pay $0.00005 per run plus $0.0025 per returned document:

maxItems	Max cost per call
10	≈ $0.025
25	≈ $0.063
50 (default)	≈ $0.13

Zero matches cost only the start fee. With call-actor, you can also cap spend with maxTotalChargeUsd in callOptions.

Tips for agents:

• Always set filters (term, agencies, documentTypes, dates) plus a small maxItems (5–20). Results are newest first. With no filters, the last ~14 days are returned.
• Agency filters take slugs like environmental-protection-agency, not display names or acronyms.
• Use outputFormat: "compact" to keep rows short. Leave includeRaw off.
• For "check again later" tasks, set onlyNewSinceLastRun: true so repeat runs return only new documents.
• The Federal Register is the official source, but these rows are convenience copies. Cite the official htmlUrl for every document.

Pricing

Pay-per-event pricing:

Event	Price (USD)	Notes
Actor start (apify-actor-start)	$0.00005	Charged once per run
Dataset item (apify-default-dataset-item)	$0.0025	Per normalized document row ($2.50 per 1,000 results)

You pay only for the documents the Actor returns. For example, a 50-document run costs about $0.125 in result charges plus the start fee.

Source, rate limits & ToS

• Public API, no API key. Docs explicitly invite HTTP clients: API Documentation, REST API overview.
• FederalRegister.gov restricts aggressive HTML scraping and steers programmatic access to this API. This Actor uses only /api/v1/… endpoints.
• robots.txt disallows some HTML search paths (/documents/search, etc.); it does not disallow /api/. Prefer the API (as this Actor does).
• Usage note from official docs: republishers must not use official NARA or OFR logos/seals.
• Upstream pages only the first ~2000 matches; use date filters for larger pulls. This Actor paces ~200 ms between pages and retries with backoff on 429 / 5xx.
• Always verify critical decisions against the official document page in htmlUrl.

FAQ / disclaimer

Is this official Federal Register / NARA software? No. Not affiliated with, endorsed by, or operated by NARA, OFR, or FederalRegister.gov.

What happens if I set no filters? The Actor auto-applies a ~14-day publicationDateStart / publicationDateEnd window so Store Try / first runs succeed without dumping years of documents. Pass explicit filters for precision.

Is the data complete? Dataset items are convenience transforms of public API metadata and may be incomplete, delayed, or truncated (abstract). Full document body is not fetched by default (use htmlUrl / pdfUrl). Verify every document on the official page before compliance or legal action.

What are valid agency IDs? Use Federal Register agency slugs (e.g. environmental-protection-agency), not display names. See https://www.federalregister.gov/agencies.

What does significant mean? When present, it reflects documents associated with the Unified Agenda and deemed significant under EO 12866. Many records return null.

Why might a hit's title or abstract not contain my term? term is full-text search on the upstream Federal Register API (conditions[term]). Matches can come from body or other indexed fields, so the returned title / abstract may not literally include the keyword.

Is this legal advice? No.

How do I schedule this Actor for ongoing monitoring? In Apify Console open Schedules → Create new, pick a cron expression (e.g. 0 12 * * 1-5 for every weekday — the Federal Register publishes on business days — or 0 12 * * 1 for weekly), then add this Actor with the input you want saved (term, agency slugs, document types, significantOnly). Each scheduled run writes a fresh dataset; pair it with Apify integrations / webhooks (Slack, email, Make, Zapier, your own endpoint) to get a notification when a run finishes. You can also start runs from the Apify API or client libraries on your own cron.

Easiest for recurring runs: turn on onlyNewSinceLastRun. Each scheduled run then returns only documents you haven’t received yet (see Only new since last run above). If you'd rather manage dates yourself, the rolling-window tip below still works.

Tip — use a rolling date window for recurring runs. Fixed publicationDateStart / publicationDateEnd values are fine for a one-off pull, but a schedule that keeps them will re-pull the same old documents every time (and you pay per dataset item). For ongoing monitoring keep the window short and recent — e.g. the last 1–7 days for a daily/weekly schedule — with maxItems sized to that window. This Actor has no relative-date input, so either (a) leave all filters empty and let it auto-apply its ~14-day rolling window, (b) compute the dates when you trigger the run via the API or an integration, or (c) periodically update the dates saved on the schedule. Results come back newest-first; dedupe downstream on documentNumber if windows overlap.

Is this Actor affiliated with the Federal Register, NARA, or OFR? No — reminder: this is an independent tool built on the public FederalRegister.gov API. It is not affiliated with, endorsed by, or operated by NARA, the Office of the Federal Register, or FederalRegister.gov. Do not use official NARA/OFR logos when republishing, and verify documents on the official htmlUrl.

Limitations

• Metadata + links only; no full-text body download in this Actor
• Upstream pagination ceiling (~2000 matches) — partition by date/agency for large jobs
• significant is often null even for important rules
• Agency filter requires correct slugs
• onlyNewSinceLastRun tracks publication date + document number only (see the delta-mode section)
• Public government data may lag or change; not an official gazette substitute for citation

License

MIT (Actor scaffold). Upstream Federal Register content remains U.S. government public information; do not use official NARA/OFR logos when republishing.