Drug Label Diff Tracker (DailyMed)

Watch FDA drug labels (SPL) and get structured, section-level before/after diffs when the label changes — built exclusively on the official NIH/NLM DailyMed Web Services API. No scraping, no fragile selectors, no fabricated data.

When an FDA-approved reference-listed drug (RLD) label is revised, generic manufacturers, RA consultancies and CROs must find out exactly what changed in safety-relevant sections (boxed warning, contraindications, warnings & precautions, adverse reactions) and drive that change into their own labeling review. DailyMed's native alerts notify you that something changed; this actor characterizes the change: which LOINC-coded section changed, what text was added or removed, how large the delta is, and a conservative severity heuristic for triage.

Decision-support only. Diffs and severity levels are machine-generated heuristics with no measured semantic precision. They are not a regulatory, compliance, or clinical determination, and they are not legal advice. Always verify against the official DailyMed label (linked in every record) before acting. Users remain responsible for their own labeling decisions.

---

Who is this for (ICP) & use cases

• Regulatory affairs teams at generic drug manufacturers — track RLD labels on your watchlist; get an auditable, structured change trail to support 21 CFR 314.94(a)(8)(ii)-driven labeling reviews.
• RA consultancies & CROs servicing ANDA holders — monitor dozens of client molecules from one place; export diffs as evidence of systematic diligence.
• Pharmacovigilance / drug safety — early visibility of boxed-warning, contraindication and warnings-and-precautions edits.
• Quality & compliance — demonstrate a dated watchlist process instead of ad-hoc manual label checks.

Typical workflow: run daily on a schedule with your SETID/NDC/RxNorm watchlist in incremental mode and route new section_diff records to Slack, email, a webhook, a database or a spreadsheet via Apify's dataset integrations.

---

What it does

1. Resolves identifiers — SETIDs are watched directly; NDCs and RxNorm RxCUIs are resolved to labels through the official DailyMed /spls search (ndc= / rxcui= parameters). Resolution is deterministic: newest-published first with a SETID tiebreak, capped per identifier (the full upstream match count stays reported in the run summary).
2. Checks version history — the official /spls/{SETID}/history endpoint lists every published label version.
3. Fetches both versions — the current SPL XML comes from /services/v2/spls/{SETID}.xml; the prior version comes from DailyMed's documented ZIP download endpoint (getFile.cfm?type=zip&…&version=N), extracted binary-safely.
4. Normalizes LOINC-coded sections — parses the HL7 v3 SPL document, extracts top-level LOINC sections (subsections fold into their parent), normalizes text conservatively (NFC, zero-width removal, whitespace collapsing only — wording, numbering and punctuation preserved exactly).
5. Diffs section by section — line-level LCS diff over normalized paragraph/list/table-cell blocks with bounded memory guards; emits capped added/removed lines plus true counts, similarity and char-delta metrics.
6. Classifies severity conservatively — a documented heuristic combining static section criticality and textual magnitude (see below). Every input to the decision is reported on the record so a human can audit or override it.
7. Keeps state safely — per-label snapshots live in a named key-value store; incremental runs skip only labels whose current version is unchanged; dataset writes happen strictly before the durable state commit, so any failure replays rather than skips.

---

Output schema (schemaVersion 1.0.0)

Every dataset record carries stable core fields; two record types share the schema:

label_status — one per watched label per run (heartbeat + state):

{
  "recordId": "…40-char sha1…",          // deterministic: sha1("label_status"|schemaVersion|setid|version)
  "type": "label_status",
  "setid": "073cf6d9-7142-4d8b-898e-174c62a1f76f",
  "labelTitle": "METFORMIN HYDROCHLORIDE TABLET, FILM COATED [REMEDYREPACK INC.]",
  "currentVersion": 15,
  "priorVersion": 14,
  "changedSinceLastRun": true,           // false on heartbeat runs
  "baseline": false,                     // true on first-ever check
  "requestedSectionsMissing": [],        // LOINC codes not found in either version (explicit unknowns)
  "note": undefined | "no_prior_version_available",
  "resolvedFrom": [{ "kind": "ndc", "raw": "70518-1149-1" }],
  "sourceUrls": { "labelPage": "https://dailymed.nlm.nih.gov/dailymed/drugInfo.cfm?setid=…", "currentXml": "…", "historyJson": "…", "priorVersionZip": "…" },
  "provenance": { "source": "NIH NLM DailyMed Web Services v2", "attribution": "…", "notice": "…", "endpoints": ["…actually-fetched URLs"], "fetchedAt": "2026-08-25T18:00:00.000Z" },
  "checkedAt": "2026-08-25T18:00:00.000Z"
}

section_diff — one per changed section between the two latest versions:

{
  "type": "section_diff",
  "fromVersion": 14, "toVersion": 15,
  "sectionKey": "warnings_precautions",
  "loincCode": "43685-7",
  "sectionTitle": "5 WARNINGS AND PRECAUTIONS",
  "presentInPrior": true, "presentInCurrent": true,
  "severity": {
    "normalized": "medium",              // high | medium | low | unknown — HEURISTIC, not regulatory
    "raw": "sectionWeight=2; deltaChars=214; changedLines=6",   // auditable inputs to the heuristic
    "material": true
  },
  "diff": {
    "addedLines": ["…"], "removedLines": ["…"],   // capped (counts remain exact)
    "addedCount": 3, "removedCount": 1,
    "similarity": 0.87,                  // 2·LCS/(lenA+lenB) over normalized blocks
    "deltaChars": 214,
    "quality": "exact"                   // "degraded" when size guards forced a coarse diff
  },
  "baseline": false,
  "recordId": "…sha1(setid|14|15|sectionKey) — stable across replays…"
}

Because recordIds are derived from content coordinates (setid, version pair, section), a replayed run re-emits identical ids — downstream consumers can deduplicate idempotently.

Severity heuristic (conservative)

Level	When
high	Material change (≥60 normalized chars or ≥4 changed lines) in boxed warning or contraindications; or such a section appearing/disappearing
medium	Material change in warnings & precautions / adverse reactions / drug interactions; any other tracked section appearing/disappearing
low	Minor edits anywhere (below both thresholds); material changes in non-safety catalog sections
unknown	Reserved for cases the actor cannot assess

This is deliberately not semantic classification: no claim is made about what the changed text means, and no precision/recall figure is asserted because none has been measured. Treat levels as a review-prioritization signal only.

Tracked sections (LOINC)

Key	Section	LOINC
boxed_warning	BOXED WARNING	34066-1
contraindications	CONTRAINDICATIONS	34070-3
warnings_precautions	WARNINGS AND PRECAUTIONS	43685-7
adverse_reactions	ADVERSE REACTIONS	34084-4
drug_interactions	DRUG INTERACTIONS	34073-7
indications_usage	INDICATIONS AND USAGE	34067-9
dosage_administration	DOSAGE AND ADMINISTRATION	34068-7
overdosage	OVERDOSAGE	34088-5
patient_counseling	PATIENT COUNSELING INFORMATION	34076-0

Subsections (which often reuse generic LOINC codes) fold into their enclosing top-level section so the parent's diff covers them verbatim.

---

Input

All fields optional; defaults ship in the input schema. The example/prefill input is a bounded real-source smoke (two multi-version labels, full mode) that always produces a non-empty dataset well within five minutes.

{
  "setids": ["073cf6d9-7142-4d8b-898e-174c62a1f76f"],
  "ndcs": [],
  "rxcuis": [],
  "mode": "incremental",
  "sections": ["boxed_warning", "contraindications", "warnings_precautions", "adverse_reactions"],
  "maxItems": 200,
  "includeDiffLines": true,
  "maxDiffLinesPerSection": 40,
  "resolveLimitPerIdentifier": 25,
  "minRequestIntervalMs": 1100
}

• mode: incremental (default) — fetches history for every label, but re-fetches XML and re-diffs only labels whose current version differs from stored state (or was never checked). Unchanged labels still emit a label_status heartbeat row every run.
• mode: full — re-diffs every label against its prior version regardless of stored state. Use for baselines and after changing sections. The Store example input uses full so the daily platform health check always has output.
• Empty watchlist → hard error. The actor never invents work silently.

---

Modes, deduplication & crash safety

• Stable IDs: status rows are keyed by (setid, currentVersion); diffs by (setid, fromVersion, toVersion, sectionKey).
• Write-before-state: dataset records and the run summary (OUTPUT, LAST_RUN_SUMMARY) are written first; the snapshot state commits last, and never for a label whose records were cut off by maxItems. Any crash/failure therefore causes a replay of the same window on the next run — never a skip.
• Bounded state: snapshots hold only version numbers, dates and a content hash (~300 bytes/label); the store is pruned to the most recently checked 2,000 labels. A pruned label simply re-baselines once if watched again — it is never silently suppressed.
• Replay duplicates: dataset writes are append-only, so a crashed run's records may appear again after replay; identical recordIds make downstream dedup trivial.

---

Sources, rate limits & attribution

Endpoint	Use
GET /services/v2/spls.json?ndc=… / ?rxcui=…	identifier → SETID resolution
GET /services/v2/spls/{SETID}/history.json	published version list + title
GET /services/v2/spls/{SETID}.xml	current SPL document
GET /dailymed/getFile.cfm?type=zip&setid=…&version=…	prior-version ZIP (official, linked from the web-services page)

All endpoints are official NIH/NLM DailyMed Web Services v2 resources used as designed — zero scraping. DailyMed publishes no formal client quota; this actor paces itself at ~1 request/second (minRequestIntervalMs, configurable), retries transient failures (408/429/5xx/network) with capped exponential backoff honoring Retry-After, enforces per-request timeouts and hard response-size ceilings, and fails individual labels explicitly without poisoning the rest of the run.

Attribution: data © the respective label authors, disseminated by the NIH National Library of Medicine / FDA Structured Product Labeling program. This actor is an independent tool, not affiliated with or endorsed by NIH, NLM or FDA. Every record embeds attribution, a decision-support notice, deep links to the official label page and the exact fetched endpoints.

Known source limitations (explicit, not hidden)

• RxNorm ingredient codes match many labels (e.g. metformin's ingredient RxCUI matches hundreds); results are capped per identifier newest-first and the truncation is reported in the run summary. Prefer product-level RxCUIs or SETIDs for precision.
• Versioned XML ships as ZIP: older versions come from the official ZIP endpoint (contains the SPL XML plus label images); the largest .xml member is extracted with binary-safe handling.
• Highlights-section boxed-warning summaries are not tracked separately; only the dedicated boxed-warning section (34066-1) is diffed where present as a top-level section. Sections genuinely absent from a label are reported under requestedSectionsMissing rather than guessed.
• Incremental trust boundary: incremental mode trusts version numbers from the history endpoint; content-level drift without a version bump is out of scope (use full periodically if you want belt-and-braces).
• No historical backfill beyond DailyMed's archive: only version pairs still listed by /history can be diffed.

---

Cost & performance

The actor is free to run; you pay Apify platform usage for your own runs. Runtime scales with watchlist size at ~1 request/second:

• Example run (2 labels): ~6 requests ≈ 10–20 s including startup.
• Sustained rule of thumb: ~3–4 s per label (history + current XML + prior ZIP).
• Memory: 1024 MB default is ample; parsing is streaming-friendly and bounded (32 MiB response ceiling, LCS cell budget with explicit degraded-quality fallback).

Cost-control levers: keep the watchlist focused, use incremental mode for schedules, raise minRequestIntervalMs if you prefer extra politeness over speed.

---

Development

npm install
npm run lint            # ESLint (flat config)
npm run typecheck       # tsc --noEmit
npm run build           # dist/
npm test                # unit + integration + schema tests (no network)
npm run test:smoke      # SMOKE=1 gated live-source tests against DailyMed
npm run smoke:local     # full actor locally against the real API via local storage

Architecture: src/main.ts (Apify wiring) → src/run.ts (orchestration, injectable IO) → src/dailymed.ts (official endpoints, paced client) · src/splxml.ts (ordered-tree LOINC extraction) · src/differ.ts (bounded LCS) · src/severity.ts (documented heuristic) · src/state.ts (snapshot store) · src/http.ts (retries/backoff/byte-caps/binary-safe fetch) · src/zip.ts.

Concurrency note: run schedules non-overlapping (maxConcurrency = 1). Overlapping runs merge snapshots best-effort (KV has no compare-and-swap).

---

Changelog

See ./CHANGELOG.md. Semantic versioning; the dataset schemaVersion bumps on any breaking output change.