NCES K-12 Schools & Districts Intelligence

Apify Actor for reproducible, auditable public-school and LEA intelligence built from official NCES CCD data, with optional NCES EDGE geography and ArcGIS district geometry.

What it returns

The Actor can return:

• public K-12 school records;
• public LEA/district records;
• school-to-LEA relationship records;
• multiple school years in one request;
• preliminary, final, or revised release selection;
• normalized fields plus complete official source rows;
• optional EDGE geographic enrichment;
• optional ArcGIS district geometry;
• year-over-year ADDED, REMOVED, and CHANGED records;
• RUN_SUMMARY in the default Key-Value Store.

The default Dataset contains one JSON item per selected entity or relationship. Official data is never silently discarded: normalized fields are separate from raw_record, and component-level rows are retained in raw_records_by_component.

Minimal input

{
  "entity_types": ["schools"],
  "school_years": ["2023-24"],
  "ncessch_ids": ["010000500870"],
  "include_raw_record": true,
  "max_results": 100
}

For a joined school/district/relationship result:

{
  "entity_types": ["schools", "districts", "relationships"],
  "school_years": ["2023-24", "2024-25"],
  "states": ["AL"],
  "include_raw_record": true,
  "include_edge": true,
  "include_geometry": true,
  "output_mode": "normalized",
  "max_results": 100
}

The exact accepted fields, enumerations, and limits are defined in .actor/input_schema.json and are validated again at runtime. A schema-valid request can still produce an explicit business failure when an official source is unavailable or a requested release is not found.

Output modes

• normalized: stable normalized fields plus source and raw-field metadata;
• raw: normalized identity and complete original component rows;
• flat: flattened output for spreadsheet-oriented workflows;
• summary: a compact run-level summary item.

Every successful run writes RUN_SUMMARY to the default Key-Value Store. It includes status, record counts, source metadata, release metadata, component success/failure, warnings, truncation state and reasons, and change counts. If changes are enabled, the CHANGES key contains the change records.

Data integrity and semantics

• School identity is NCESSCH + school_year.
• District identity is LEAID + school_year.
• Relationship identity is NCESSCH + LEAID + school_year.
• CCD values are not overwritten by EDGE values; enrichment uses explicit edge_* fields.
• School and district EDGE source labels are kept distinct.
• Missing, suppressed, not applicable, not reported, and numeric zero values are not collapsed into one value.
• A limit is reported as truncation only when records were actually cut off. Targeted ID early-stop is reported separately as target_early_stop.
• Empty results, component failures, partial success, and source errors remain visible in the summary; they are not reported as ordinary successful data runs.

Official sources

• NCES CCD Data Tables API: https://nces.ed.gov/ccd/datatables/api
• NCES CCD downloadable files: https://nces.ed.gov/ccd/data.html
• NCES EDGE geospatial data: https://nces.ed.gov/programs/edge/
• ArcGIS geometry is queried only when geometry enrichment is requested and the district identifier is available.

Each downloaded source retains its official URL, filename, SHA-256, school year, release version/status and retrieval metadata. The Actor uses bounded retries for 429, 5xx and recoverable network failures; permanent HTTP errors fail explicitly.

Local verification

This project is tested before deployment and does not require an Apify run for the local acceptance matrix:

pytest -q
python3 -m compileall -q src tests scripts
python3 scripts/run_local_acceptance.py
python3 scripts/run_local_resource_benchmark.py
python3 scripts/run_local_merge_benchmark.py
python3 scripts/run_local_docker_contract.py
python3 -m json.tool .actor/actor.json >/dev/null
python3 -m json.tool .actor/input_schema.json >/dev/null
python3 -m json.tool .actor/output_schema.json >/dev/null
apify validate-schema .actor/input_schema.json

The local suite covers output modes, multi-year behavior, changes, partial failures, truncation, invalid input, empty results, release selection, retries, download limits, ZIP/CSV edge cases, EDGE/ArcGIS no-match behavior, source hashes, Docker startup, and representative synthetic resource measurements.

The resource reports are engineering measurements on synthetic CCD-like fixtures. They are not a nationwide cost estimate and must not be used alone to set pricing.

Limitations

• Cross-run change detection requires a previous baseline supplied through the programmatic pipeline; a previous Dataset is not automatically imported as a baseline.
• Full nationwide requests can require substantial download time and storage because official CCD components are distributed as downloadable files.
• max_results is a safety limit, not a guarantee that every source file is scanned when targeted identifiers allow deterministic early stopping.
• EDGE and ArcGIS enrichment is optional and can legitimately have no matching geography for a valid CCD entity.
• Official-source availability, release catalog changes, and upstream file revisions can affect a run; those conditions are recorded in RUN_SUMMARY rather than hidden.

Release status

The code and local test suite are maintained separately from Store visibility, pricing, and public listing configuration. Deployment is performed only after the local gate passes, followed by one minimal online regression that reads the Run, Dataset, and RUN_SUMMARY instead of relying on process exit code alone.