Federal Register Document Scraper

What it does

Scrapes official documents from the Federal Register API, the daily journal of the United States Government. Access rules, proposed rules, notices, and presidential documents with full metadata.

This actor connects to a public API, fetches structured data based on your search criteria, and stores the results in a clean, normalized dataset on the Apify platform. It handles pagination automatically so you can collect large volumes of results without worrying about API limits or offsets. The actor is designed to be robust with built-in error handling, request timeouts, and input validation to ensure reliable data collection every time you run it.

Why use this actor

Manually querying APIs and handling pagination, rate limits, and data normalization is tedious and error-prone. This actor automates the entire process. Simply provide your search parameters, set the maximum number of results you want, and let the actor handle the rest. The data is stored in a structured dataset that you can export as JSON, CSV, or Excel. You can integrate this actor into larger workflows using the Apify API, schedule it for recurring data collection, or trigger it from your own applications via webhooks.

Input parameters

• searchQuery (string, required): The search term to query the Federal Register. Default: "environmental regulation".
• maxResults (integer, optional): Maximum number of results to return. Default: 100. Range: 1-1000.

All inputs are validated at startup with sensible defaults applied when values are missing. The actor will log warnings for any misconfigured options and continue with safe defaults rather than failing outright.

Output data

Each result in the dataset contains the following fields:

• documentNumber: Federal Register document number
• title: Document title
• type: Document type (Rule, Proposed Rule, Notice, Presidential Document)
• abstractText: Document abstract
• publicationDate: Date of publication
• agencies: Primary agency name
• htmlUrl: URL to the HTML version
• pdfUrl: URL to the PDF version

All string fields are null-checked to ensure consistent data quality. Missing or undefined values are stored as null rather than empty strings or undefined values.

Example output

{
    "documentNumber": "2023-12345",
    "title": "Clean Air Standards Update",
    "type": "Rule",
    "abstractText": "The EPA is amending...",
    "publicationDate": "2023-06-15",
    "agencies": "Environmental Protection Agency",
    "htmlUrl": "https://www.federalregister.gov/d/2023-12345",
    "pdfUrl": "https://www.govinfo.gov/content/pkg/FR-2023-06-15/pdf/2023-12345.pdf"
}

Pricing

This actor is available on the Apify platform with transparent usage-based pricing. Each run incurs a small startup cost of approximately $0.005 per start, plus roughly $0.01 per result collected. Actual costs depend on the number of results, API response times, and memory allocation. You can control costs by setting the maxResults parameter to limit the number of results collected per run. For high-volume use cases, consider running the actor on a schedule during off-peak hours to optimize platform resource usage.

More scrapers from brave_paradise

Check out these other useful data collection actors by brave_paradise:

• arXiv Research Paper Scraper
• PubMed Medical Article Scraper
• Semantic Scholar Paper Scraper

Visit the brave_paradise profile on Apify to explore the full collection of specialized data scrapers and automation tools.