US Federal Register Documents

Federal Register scraper returns published document records, source summaries, dates, citations, and document links.

Federal Register scraper uses the public source named below and writes its results to the Actor dataset.

Use the official Federal Register API to find final rules, proposed rules, notices and presidential documents. Each document becomes one row; optional detail enrichment adds citations, dockets, effective dates, comment deadlines and text links. Summaries come from the source's abstract; no AI-generated summary is added.

What it returns

Fields vary by result type and optional enrichment. The examples below are from sample_output.json.

Output field	Type	Example value or excerpt from saved sample
status	string	"ok"
documentNumber	string	"2025-24102"
title	string	"Effluent Limitations Guidelines and Standards for the Steam Electric Power Generating Point Source …
documentType	string	"RULE"
type	string	"Rule"
publicationDate	string	"2025-12-31"
agencies	array	["Environmental Protection Agency"]
agencySlugs	array	["environmental-protection-agency"]
summary	string	"The U.S. Environmental Protection Agency (the EPA or Agency) is finalizing a Clean Water Act (CWA) …
excerpt	string	"Unleashing American Energy, directs Federal agencies to review and remove, as appropriate and to th…
htmlUrl	string	"https://www.federalregister.gov/documents/2025/12/31/2025-24102/effluent-limitations-guidelines-and…
pdfUrl	string	"https://www.govinfo.gov/content/pkg/FR-2025-12-31/pdf/2025-24102.pdf"
publicInspectionPdfUrl	string	"https://public-inspection.federalregister.gov/2025-24102.pdf?1767102318"
apiUrl	string	"https://www.federalregister.gov/api/v1/documents/2025-24102.json"
citation	string	"90 FR 61328"
action	string	"Final rule."
effectiveDate	string	"2026-03-02"
commentsCloseDate	null	null
commentUrl	null	null
docketIds	array	["EPA-HQ-OW-2009-0819"]
regulationIdNumbers	array	["2040-AG54"]
cfrReferences	array	[{"chapter": null, "citation_url": null, "part": 423, "title": 40}]
topics	array	["Environmental protection"]
rawTextUrl	string	"https://www.federalregister.gov/documents/full_text/text/2025/12/31/2025-24102.txt"
bodyHtmlUrl	string	"https://www.federalregister.gov/documents/full_text/html/2025/12/31/2025-24102.html"
fullTextXmlUrl	string	"https://www.federalregister.gov/documents/full_text/xml/2025/12/31/2025-24102.xml"
retrievedAt	string	"2026-10-05T14:05:58.524200+00:00"
errors	array	[]

Field	Meaning
status, errors	ok for a normalized result; error plus messages for a failed document
documentNumber, title	Source identifier and document title
documentType, type	API code (RULE, PRORULE, NOTICE, PRESDOCU) and human-readable type
publicationDate	Publication date in YYYY-MM-DD format
agencies, agencySlugs	Publishing agency names and official identifiers
summary	Original abstract with HTML markup removed; null when absent
excerpt	Keyword-matching source excerpt, when available
htmlUrl, pdfUrl	Source document page and published PDF link
publicInspectionPdfUrl	Source public-inspection PDF link, when available
apiUrl	Official document JSON endpoint
citation, action	Federal Register citation and agency action description (detail mode)
effectiveDate, commentsCloseDate, commentUrl	Source dates and comment link (detail mode; nullable)
docketIds, regulationIdNumbers	Source docket and regulation identifiers (detail mode)
cfrReferences, topics	Source CFR references and topic arrays (detail mode)
rawTextUrl, bodyHtmlUrl, fullTextXmlUrl	Source document-content links (detail mode)
retrievedAt	Time this row was normalized, in UTC

Missing optional text, dates and links remain null; missing optional arrays are empty. If detail enrichment fails, the row retains available search metadata with status: "error" and is not charged. Malformed search records can have only identity fields and errors.

The dataset has Documents, Dates and dockets, and Source links views. The default key-value store also receives SUMMARY with result/error counts, elapsed seconds, applied filters, resolved agencies and budget-stop status. A failed search writes its error in SUMMARY and fails the run.

Input example

{
  "keyword": "climate",
  "agencies": ["EPA"],
  "documentType": "RULE",
  "dateFrom": "2025-01-01",
  "dateTo": "2025-12-31",
  "order": "newest",
  "includeDetails": true,
  "maxItems": 5,
  "concurrency": 3,
  "maxRetries": 3,
  "requestTimeoutSecs": 30
}

Input	Default	Behavior
keyword	Empty	Official full-text search expression; empty searches without a keyword
agencies	[]	Slugs, full names, abbreviations or numeric ID strings; any listed agency can match
documentType	all	all, RULE, PRORULE, NOTICE or PRESDOCU
dateFrom, dateTo	Empty	Inclusive publication bounds, YYYY-MM-DD; either bound can be omitted
order	newest	newest, oldest or relevance
includeDetails	true	Fetch individual metadata; false returns search metadata without detail requests
maxItems	100	1–10,000; limits successful and error rows combined
concurrency	5	1–20 simultaneous detail requests
maxRetries	3	0–5 retries after transient network/source errors
requestTimeoutSecs	30	10–120 seconds per network operation

The official agency directory lists accepted names, abbreviations, IDs and slugs. For example, EPA, Environmental Protection Agency, 145 and environmental-protection-agency resolve to the same agency. Unknown or ambiguous agency inputs fail with an actionable message.

Output example

Actual field excerpt from the real dataset saved in sample_output.json; the file contains the complete rows:

{
  "status": "ok",
  "documentNumber": "2025-24102",
  "title": "Effluent Limitations Guidelines and Standards for the Steam Electric Power Generating Point Source Category-Deadline Extensions",
  "documentType": "RULE",
  "type": "Rule",
  "publicationDate": "2025-12-31",
  "agencies": ["Environmental Protection Agency"],
  "citation": "90 FR 61328",
  "effectiveDate": "2026-03-02",
  "pdfUrl": "https://www.govinfo.gov/content/pkg/FR-2025-12-31/pdf/2025-24102.pdf",
  "errors": []
}

Pricing

Proposed pay-per-event fee: $0.001 per document event, or $1 per 1,000 successful documents. Error rows and zero-match searches have no document event charge. Apify platform costs depend on your plan. This build is unpublished; the owner must configure the proposed event before paid publication.

The dated Store comparison and event setup are in PRICING.md. On 2026-10-05 the most-used directly comparable Federal Register Rules & Regulations Search charged $0.002 per document; the proposed document fee is 50% lower.

Limits

• The actor caps a run at 10,000 result rows. Split larger searches into narrower date ranges. It follows official next-page links, including source continuation tokens, rather than relying solely on the reported page count.
• Filters describe publication dates, not effective dates. The source can include scheduled documents; set dateTo when you need a closed historical period.
• The actor returns metadata and document links. It does not download full document bodies or PDF files.
• Source availability, throttling, schema changes and record corrections can affect runs. Transport failures and HTTP 408/429/500/502/503/504 retry with bounded backoff; other HTTP failures surface immediately.
• A search/agency-directory failure fails the run. A document-level parsing/detail failure yields an uncharged error row and other documents continue. Review status, errors and SUMMARY before using results.
• One detail batch can finish after the spending limit is reached, but no additional rows are emitted. Do not configure a second paid default-dataset-item event alongside document.
• Deduplication applies within a run. Reusing a local storage directory appends to its dataset; use fresh storage when testing another run.

FAQ

What limits a search? maxItems caps document and error rows; narrow publication dates for broader searches. Detail enrichment makes extra requests.

Do I need a proxy, login, or source API key? No. The actor calls the public Federal Register API without source credentials or a configured proxy.

How is it priced? The proposed pay-per-event model charges the document event for each successfully saved document. Error rows and empty searches have no custom document charge; platform usage may be separate.

How are source rate limits handled? Requests that receive retryable responses use bounded retries and honor Retry-After when present. Detail requests are bounded by concurrency.

How fresh is a record? Results come from the source API at run time. Compare publicationDate with retrievedAt; publication dates are distinct from effective dates.

Why are citations or deadlines missing? Enable includeDetails; the source may still omit them. A failed detail lookup produces an error row.

Use with AI agents / MCP

After the owner publishes this actor, connect it through Apify's Actors MCP server. Give the agent a keyword, publication window, agency and result cap. Have it cite htmlUrl or pdfUrl, retain documentNumber, and exclude rows whose status is error. Treat a missing abstract or deadline as missing source information. This actor does not implement its own MCP server or call a model.

Local development and checks

The prescribed environment already contains the dependencies; do not install into the shared virtual environment. Runtime requirements are apify and httpx; pytest is needed for development tests.

On Windows, from this actor directory:

$env:APIFY_LOCAL_STORAGE_DIR = Join-Path $PWD 'storage'
C:/GritWork/earn/.venv/Scripts/python -m src
C:/GritWork/earn/.venv/Scripts/python -m pytest -q tests/test_config.py tests/test_parsing.py tests/test_crawler.py
C:/GritWork/earn/.venv/Scripts/python -m pytest -q -m live
apify validate-schema

Place the input JSON at storage/key_value_stores/default/INPUT.json before running. A real input and its dataset are retained there. scripts/run_local.py prepares fresh named storage, runs python -m src without the Apify CLI and captures receipts in evidence/. The live smoke test requests the official API; unit fixtures are saved official responses. Docker uses apify/actor-python:3.12 and starts python3 -m src.