FERC eLibrary Energy Docket & Interconnection Scraper

Track, monitor, and extract regulatory filings, interconnection agreements, tariff amendments, and commission orders from the Federal Energy Regulatory Commission (FERC) eLibrary system.

---

Features

• Multi-Docket & Prefix Filtering: Search by specific docket numbers (e.g., ER24-100-000, CP24-50-000) or standard FERC prefixes (ER - Electric Rate/Interconnection, CP - Pipeline Certificates, RP - Gas Rates, RM - Rulemaking, EL - Electric Licensing).
• Keyword & Full-Text Search: Filter filings by description summary, tariff title, or submittal keywords (e.g. Large Generator Interconnection Agreement, PJM, MISO, battery storage).
• Comprehensive Metadata Extraction: Captures accession numbers, docket numbers, filing category, filing type, applicants/authors, recipients, filing dates, and file manifests.
• Security & Privacy Safe: Automatically verifies security classifications and drops Non-Public / CEII (Critical Energy/Electric Infrastructure Information) records. PII (emails, telephone numbers) in filing summaries is redacted. Applicant and author entities are organization names only, for both entity shapes the source uses: affiliation objects contribute only their organization field, and string entries are kept only when they carry an organization indicator (LLC, Inc, Corp, Corporation, Corporate, Company, Co or Co., Ltd or Ltd., Limited, PLC, GmbH, AG, BV, NV, SARL, Authority, Commission, Department, Administration, Association, Cooperative, LP, LLP, PC, PLLC, University, City of, County of, State of, Tribe, Nation). String-form extraction is indicator-only: a natural person's name carries no organization indicator in any encoding, script, or punctuation form, so person names are excluded by construction, with no name-shape matching. The deliberate trade-off, disclosed here and in TEST_NOTES: an indicator-less organization in string form (for example "Duke Energy" or "Acme S.A.") is dropped along with the person names.
• Outage & Maintenance Sentinel: Proactively monitors FERC eLibrary index health. Distinguishes weekend maintenance windows from true zero search results.
• No API Key Required: Operates directly against the public FERC eLibrary search infrastructure without requiring credentials or paid subscriptions.

---

Typical Use Cases

• Renewable & Battery Interconnection Tracking: Monitor regional transmission organization (RTO/ISO) queues, interconnection agreements (LGIA/SGIA), and study reports in electric rate (ER) dockets.
• Energy Regulatory Compliance & Legal Research: Track rulemakings (RM), enforcement orders, pipeline certificates (CP), and rate cases (RP) across federal energy jurisdictions.
• Corporate Intelligence & M&A: Follow market participants, submittals, tariff changes, and intervention petitions as they are filed and indexed.

---

Input Parameters

Field	Type	Required	Default	Description
docketPrefixes	Array of strings	No	["ER", "CP", "EL", "QF"]	Major FERC docket prefixes to search (e.g., ["ER", "CP", "RP"]).
docketNumbers	Array of strings	No	[]	Specific full or base docket numbers (e.g., ["ER24-100-000"]).
searchKeywords	String	No	"Large Generator Interconnection Agreement"	Keywords to filter filings by title or description summary. An explicitly empty string clears the keyword filter.
filedDateFrom	String	No	Run date minus 7 days (dynamic)	Earliest filing date to include (YYYY-MM-DD).
filedDateTo	String	No	Run date	Latest filing date to include (YYYY-MM-DD).
maxItems	Integer	No	1000	Maximum number of filing records to return.

Only filings whose security level is Public are requested (the search asks the source for Public availability only) and emitted. Non-Public, CEII, Protected, and Privileged records are dropped and counted, never emitted.

Source cadence: FERC eLibrary accepts filings continuously, and the search index is updated as filings are processed. FERC runs search index maintenance during some weekend windows; during those windows the actor reports a PARTIAL outage status instead of a false zero result (see Troubleshooting).

Canary input

Labeled canary (safe to run as-is; the dynamic date default keeps it valid without edits):

{
  "docketPrefixes": ["ER", "CP", "EL", "QF"],
  "docketNumbers": [],
  "searchKeywords": "Large Generator Interconnection Agreement",
  "filedDateFrom": "",
  "filedDateTo": "",
  "maxItems": 1000
}

---

Output Format

Each record in the dataset represents a public FERC filing:

{
  "accessionNumber": "20261029-5100",
  "docketNumbers": ["ER27-200-000", "ER27-200-001"],
  "filedDate": "2026-10-29",
  "documentDate": "2026-10-29",
  "filingCategory": "Submittal",
  "filingType": "Tariff Amendment",
  "applicantOrAuthorEntities": [
    "Midcontinent Independent System Operator, Inc."
  ],
  "recipientEntity": "Federal Energy Regulatory Commission",
  "descriptionSummary": "Large Generator Interconnection Agreement submittal for Solar Project. Contact: [email-redacted] or [phone-redacted].",
  "securityLevel": "Public",
  "fileList": [
    {
      "fileId": "123456",
      "fileName": "LGIA_Agreement.pdf",
      "fileType": "PDF",
      "fileSizeBytes": 1048576
    }
  ],
  "fercFilelistUrl": "https://elibrary.ferc.gov/eLibrary/filelist?accession_Number=20261029-5100"
}

---

Integrity & Quality Guarantees

• No Silent Failures: Core fields (accessionNumber, docketNumbers, filedDate, securityLevel) are verified on every record. If more than 20% of records fail validation due to API drift, the actor halts immediately with an informative diagnostic error.
• Direct Link to Files: Every record contains a direct URL (fercFilelistUrl) to view and download all associated public submittal documents.

Troubleshooting

• PARTIAL status saying the search index is unavailable or undergoing maintenance: FERC runs weekend maintenance on the eLibrary search index. The actor checks index health before searching (a 7-day ER docket search must return results) and reports this PARTIAL status with 0 rows instead of presenting an outage as an empty result. Wait and run again after the maintenance window.
• Complete with 0 rows: The source was healthy and no filings matched your filters. Widen the date window, clear searchKeywords, or add docket prefixes.
• Run stopped at maxItems: A PARTIAL status names the cap. Raise maxItems to collect more of the reported total.
• Repeated source errors: If this keeps happening, please open an issue on this actor's Issues tab and we'll fix it fast.

Changelog

• 0.2.5 (2026-11-02): M1 structural closure and N4 indicator additions from the Red Team final GO review of 0.2.4. String-form entity extraction is now indicator-only, the structural fix the round-4 reviewer recommended: a string entry is kept only when it carries an organization indicator, so a natural person's name drops in any encoding, script, or punctuation form (the 0.2.4 matcher still leaked names written with non-ASCII commas, typographic joiners, format characters, or Tibetan syllable marks), and the name-shape matcher is removed. The indicator set gains AG, BV, NV, SARL, and Corporate. Disclosed cost: indicator-less organizations in string form (for example "Duke Energy", "Acme S.A.") now drop along with the person names. The structured affiliation-object path is unchanged. Shipped fixture cases cover each codepoint family, unit level and end to end, plus the N4 keeps and the adjusted keep and drop matrices.
• 0.2.4 (2026-11-02): M1 class closure and N3 indicator additions from the Red Team final GO review of 0.2.3. String-form person detection now closes the defect class instead of enumerating shapes: entries are normalized to composed Unicode form and matched on a diacritic-folded copy, so decomposed (NFD) accented names drop like their precomposed forms and names in scripts whose marks have no precomposed form are covered; name tokens accept dotted initials (for example "J.R. Doe" and "Cronin, A.J."); and a comma entry whose first two segments are name-like is a person whatever its trailing segments hold (for example "Doe, John, V", "Doe, John, Esq", "Doe, Jane, Jr., MD"). The organization indicator set gains Ltd, Ltd., Limited, PLC, and GmbH, so entries carrying those suffixes are kept. Shipped fixture cases cover each family, unit level and end to end, including an NFD-encoded name, plus the full organization keep and prior drop matrices.
• 0.2.3 (2026-11-02): M1 closure and N1 indicator fix from the Red Team final GO review of 0.2.2. String-form person detection now accepts Unicode letters in name tokens (accented names such as "García, José" and "José García" are excluded), covers comma forms without a token cap on the surname segment (for example "De La Cruz Martinez, Juan"), handles a trailing suffix segment in the comma form ("Doe, John, Jr", with Jr, Sr, II, III, IV and an optional trailing period) and a trailing suffix token in the plain form, and covers plain forms at two to four parts (for example "Jose De La Cruz" and "Mary Ann Van Horn"); plain names longer than four parts are a documented heuristic limit, disclosed in the dataset schema. The documented Co. organization indicator now actually keeps entries (for example "Acme Co.", "Acme Co", "Acme Co., Ltd", and "The Williams Co."). Shipped fixture cases cover each new format, unit level and end to end, plus the full organization keep and prior drop matrices.
• 0.2.2 (2026-11-02): M1 extension and TEST_NOTES repoint from the Red Team final GO review of 0.2.1. String-form person detection now treats a single-letter token (with or without a trailing period) as a name token in both comma and plain forms, allows surnames with internal spaces and apostrophe-initial surnames in the comma form, and matches case-insensitively, so forms such as "Doe, J", "J Doe", "Van Horn, John", "De La Cruz, Juan", "O'Brien, Conan", and "doe, john" are excluded from applicantOrAuthorEntities; shipped fixture cases cover each new format. TEST_NOTES.md now points at the shipped fixture suite (tests/test_fixtures.py, run from the package root), reports its current result count, uses the current "X of Y matches" status wording, and states explicitly that the indicator-less organization-name drop in string form is a known, disclosed trade-off.
• 0.2.1 (2026-11-02): Privacy and drift fixes from the Red Team review of 0.2. String-form applicant/author entities that look like natural person names are now dropped unless they carry an organization indicator, so natural person contacts are excluded for both entity shapes. Docket number entries are type-validated: a dict entry yields its docketNumber value, and any other non-string entry counts the row invalid instead of being stringified into the output. Rows are pushed per page instead of being held in memory to the end. Status counts now read "X of Y matches", state the skipped-row count, and print "reported total: unknown" when no total was reported. The sample output fercFilelistUrl now shows the query form the code emits, the "near real-time" wording is removed, and the fixture suite ships in tests/ (see TEST_NOTES.md).
• 0.2 (2026-11-01): Fixed the search request and response handling against the live eLibrary backend: empty sortBy (the UI default), Public-only availability filter, response envelope validation so backend errors can never read as zero results, real hit field names (including the backend's acesssionNumber spelling and availCode security codes), transmittals mapped into fileList, retry standard applied to all search calls, and corrected input defaults in this README.
• 0.1 (2026-11-01): Initial build.