RAG Chunk Quality Auditor

Overview

Audit up to 500 supplied chunks before ingestion. The report checks empty or invalid text, configured character thresholds, repeated IDs, exact duplicate content and source reference format. Each record retains its original inputIndex. Chunk boundaries are preserved.

A successful audit can return qualityStatus ERRORS_FOUND. That means the report found data issues. The Actor returns observations for every supported record. Structural batch contract violations fail before delivery.

Input example

{
  "chunks": [
    {
      "id": "chunk-001",
      "text": "A source paragraph with enough context for a structural chunk quality review.",
      "sourceUrl": "https://example.com/guide"
    },
    {
      "id": "chunk-001",
      "text": "A different source paragraph explains the service terms and the delivery process.",
      "sourceId": "terms"
    },
    {
      "id": "chunk-003",
      "text": "A source paragraph with enough context for a structural chunk quality review.",
      "sourceUrl": "https://example.com/copy"
    },
    {
      "id": "",
      "text": ""
    },
    {
      "id": "0005",
      "text": "Short note",
      "sourceUrl": "relative/path"
    },
    {
      "id": "chunk-006",
      "text": "Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text Long text ",
      "sourceId": "long-source"
    }
  ],
  "minimumCharacters": 40,
  "maximumCharacters": 1600,
  "maxItems": 1
}

The default example intentionally includes conflicting IDs, repeated text, missing values and a malformed source URL. examples/input-clean.json supplies a passing example. Only id, text, sourceId and sourceUrl are accepted within a record. Field values may be missing or malformed and become findings. Map extra metadata before upload.

Output

One complete report is one dataset item and RAG_QUALITY JSON. RAG_CSV has one row per original record, and REPORT summarizes the findings. Per-record metadata includes ID, characterCount, textSha256, source references, qualityStatus and findings. Original text is represented by its hash and length.

IDs and duplicate text use literal case-sensitive equality. Duplicate IDs distinguish identical, conflicting and unverifiable text. Duplicate text groups retain every inputIndex, ID counts, missing references and distinct full valid reference tuples. Review repeated text in its intended context before deciding which copies to retain.

Counts of rows with errors and warnings can overlap. errorFindings and warningFindings count individual findings. passedRows contains records with zero findings. Character measurements use Unicode code points, including combining characters individually. Model token limits require separate evaluation.

A source ID or structurally valid HTTP or HTTPS URL establishes a provided reference. URL credentials are rejected and omitted from output. Query strings and fragments are removed from exported URLs, while fingerprints distinguish full valid reference tuples. Source availability and factual citation support require separate verification. Remove secrets and unnecessary personal information before upload. The original INPUT follows your Apify storage retention settings.

Existing RAG Delta workflow

Map an existing RAG Delta export into the supported input shape.

const chunks = delta.upserts.map(({ id, text, sourceUrl }) => ({ id, text, sourceUrl }));
const input = { chunks, minimumCharacters: 40, maximumCharacters: 1600, maxItems: 1 };

Pricing and cost

The unit is one completed rag-quality-report for up to 500 supplied chunks. The factory proposal is USD 0.03 per report. Release receipts and the Store specify active pricing. Direct demand and matched competitor pricing are pending research, with competitorMedianUsd explicitly unknown in PRICING.json. The proposal remains at a partial commercial stage.

The audit uses supplied records and standard algorithms. No AI service, embedding API, proxy or content request is required. Cloud execution cost and margin require separate measurement.

Limits and known scope

Input is limited to 2 MB and 500 objects. IDs and source IDs support up to 160 Unicode code points, and source URLs up to 2048. Invalid field values become findings. Review thresholds range up to 20000 characters. The application budget is cooperative and checked by the shared runner. Acceptance also applies a process timeout and measures RSS. Semantic quality and model-specific ingestion suitability require their own evaluation.

Development and recovery

Use Node 22 or newer and the pinned lock. Install with npm ci --offline --ignore-scripts --omit=dev --no-audit --no-fund when cache is available, then npm test and npm start. Normal start and test:local use --max-old-space-size=128 --max-semi-space-size=8. Docker uses npm start. Shared libraries are vendored from _factory/lib.

PREPARED precedes report effects. COMPLETE requires one physical report and the expected named event count. Replay checks the complete effective input fingerprint, report and export hashes, runtime, pricing and physical dataset before reusing the result. The input fingerprint also binds malformed values represented by generic findings. Markers store hashes. Inconsistent effects require reconciliation. Preserve storage and use a single writer per job. A new intentional job uses fresh storage. CSV cells are quoted, formula prefixes neutralized and numeric IDs retained as text.