RAG Dataset Linter

What is RAG Dataset Linter?

RAG Dataset Linter audits chunks from an Apify Dataset, an Actor Run's default Dataset, or an inline sample. It checks text, identifiers, provenance, ordering, duplication, and overlap, then returns a PASS, WARN, or FAIL summary with row-level findings. Use it before embedding or indexing without changing the source.

What data can it extract or produce?

• Detects missing, non-string, empty, undersized, and oversized chunk text.
• Finds exact duplicates, likely near duplicates, and duplicate chunk IDs.
• Checks duplicate or missing chunk indexes, broken per-document sequences, and excessive adjacent overlap.
• Reports missing document provenance, source URLs, stable chunk IDs, and heading context.
• Optionally flags repeated boilerplate, HTML leakage, and contextless references; these heuristic checks are warnings only.
• Auto-detects common flat and nested RAG fields, with explicit dot-path overrides for custom schemas.
• Produces a one-row audit summary, a separate findings Dataset, a JSON report, and an optional Markdown report.

At least one error yields FAIL, warning-only findings yield WARN, and no findings yields PASS. Linguistic heuristics never produce errors.

What can you use the data for?

• Gate a chunking pipeline before paying for embeddings.
• Audit output from a crawler, parser, converter, or text splitter.
• Find duplicated content that could inflate a vector index.
• Verify stable IDs, document provenance, and sequence metadata before vector upserts.
• Tune chunk-size and overlap settings using issue counts and token percentiles.

How much does it cost?

RAG Dataset Linter charges the chunk-audited pay-per-event event once per source record admitted to the audit. Platform usage is included. Current Pricing tab rates are:

Apify tier	Price per 1,000 audited chunks
Free	$1.50
Bronze	$1.00
Silver	$0.75
Gold and above	$0.50

Charge = itemsAudited / 1,000 x tier rate. For example, 10,000 records cost $15 on Free or $5 on Gold and above. A spending limit can stop a run partway through a batch.

How to use RAG Dataset Linter

1. Choose exactly one source: Dataset, Actor Run, or inline records.
2. Leave fieldMapping empty for common schemas, or provide dot-path overrides.
3. Test thresholds and limits on a representative sample.
4. Review the summary and findings Dataset. Automations should inspect gateStatus or enable failRunOnError.

This minimal valid input uses the first record shape from the current Input schema:

{
  "records": [
    {
      "chunkId": "guide-0",
      "documentId": "guide",
      "sourceUrl": "https://example.com/guide",
      "chunkIndex": 0,
      "title": "Getting started",
      "headingPath": ["Getting started"],
      "chunkText": "Install the package, create a client, and verify the connection before indexing production content."
    }
  ],
  "minTokens": 0
}

What input does the Actor accept?

Provide exactly one of datasetId, runId, or records. A Dataset integration can supply payload.resource.defaultDatasetId; explicit datasetId takes precedence. Do not combine that Dataset source with runId or records.

Field	Default and limits	Behavior
datasetId	Optional string	Accessible Dataset ID or name; the picker requests read access.
runId	Optional string	Reads an accessible Run's default Dataset.
records	Optional object array, max 1,000	Non-empty inline sample.
maxItems	1000, range 1-25,000	Maximum admitted records and the main control for runtime and PPE charges.
fieldMapping	{}	Dot-path overrides for text, chunkId, documentId, sourceUrl, title, headingPath, and chunkIndex.
minTokens	40, range 0-10,000	Warns below this estimated size. Must be lower than maxTokens.
maxTokens	2000, range 1-20,000	Reports an error above this estimated size.
nearDuplicateSimilarity	0.95, range 0.8-1	Minimum 64-bit SimHash similarity for a near-duplicate warning.
maxAdjacentOverlapRatio	0.35, range 0-1	Warns above this suffix/prefix token ratio for adjacent chunks.
enableHeuristicWarnings	true	Adds boilerplate, HTML, missing-context, and contextless-reference checks.
maxFindings	5000, range 1-20,000	Caps stored finding rows; aggregate counts continue beyond the cap.
includeMarkdownReport	true	Writes REPORT.md, previewing at most 50 findings.
failRunOnError	false	Writes reports first, then fails the run when gateStatus is FAIL.

Auto-detection examines the first 50 records and recognizes common RAG fields. Use explicit mappings for mixed schemas. Document checks group on documentId, then sourceUrl.

maxItems controls possible charges. maxFindings and heuristic checks can increase stored output or processing, but not the per-record event count.

What does the output look like?

The default Dataset always contains one audit_summary row after processing starts successfully. A compact excerpt for the inline example above is:

{
  "itemType": "audit_summary",
  "gateStatus": "PASS",
  "automationAction": "continue",
  "sourceType": "inline",
  "sourceId": null,
  "itemsAvailable": 1,
  "itemsAudited": 1,
  "findingCount": 0,
  "errorCount": 0,
  "warningCount": 0,
  "estimatedEmbeddingWasteTokens": 0,
  "warnings": [],
  "truncated": false
}

The full summary adds startedAt, finishedAt, actorName, sourceDatasetId, validTextItems, chargedItems, billingEnabled, issueCounts, duplicate counts and rates, tokenPercentiles, coverage, resolvedFieldMapping, limits, durationSeconds, and failure.

Finding fields are findingId, recordIndex, relatedRecordIndex, documentKey, chunkId, sourceUrl, code, severity, category, message, evidencePreview, metric, and threshold. Related indexes, provenance, evidence, and metrics are null when inapplicable or absent. failure is null on success or contains code and message. Inline sources have null sourceId and sourceDatasetId; itemsAvailable can also be null.

Find outputs in the run's Output and Storage tabs:

Location	Contents
Default Dataset	One summary row and the summary view.
findings Dataset	Stored row-level findings, with findings and errors views.
Default key-value store, OUTPUT	Complete machine-readable JSON summary.
Default key-value store, REPORT.md	Optional human-readable report.

Export either Dataset as JSON, JSONL, CSV, Excel, XML, HTML, or RSS. See the Input tab and the run's Output and Storage tabs for full schemas.

How are errors and partial results handled?

Invalid source selection, empty inline records, or minTokens >= maxTokens fails before reporting. Later operational failures write a FAIL summary and OUTPUT when possible, then fail the run.

With failRunOnError: true, outputs are written before a failing quality gate fails the run. With the default false, a run can succeed with gateStatus: "FAIL", so automations must read the gate.

For partial results, inspect truncated and warnings: MAX_ITEMS_APPLIED, MAX_TEXT_BUDGET_APPLIED, MAX_FINDINGS_APPLIED, CHARGE_LIMIT_REACHED, or SOURCE_TRUNCATED. maxFindings caps detail only; aggregate counts continue.

Troubleshooting:

• Empty source: verify the Dataset has items and the Run has a default Dataset.
• No text detected: set fieldMapping.text, especially for mixed rows.
• Unexpected duplicates: compare relatedRecordIndex; exact matching normalizes Unicode and whitespace but preserves case.
• Partial audit: reduce the source, raise an allowed limit or spending cap, and rerun. Findings cover admitted records only.
• Failed private Run lookup: select its default Dataset or use a Dataset integration.

What are the limits, performance characteristics, and best practices?

• Hard limits are 25,000 audited records, 50,000,000 mapped text characters, 20,000 stored findings, and 1,000 inline records.
• The Actor uses fixed 512 MB memory; no runtime or throughput guarantee is made.
• Token estimates use ceil(text.length / 4); calibrate thresholds against your tokenizer.
• Near-duplicate detection uses approximate SimHash candidate matching, not semantic similarity. Structural checks are deterministic for the same admitted records and order.
• Adjacent overlap is checked only for consecutively indexed chunks in a document group when every row in that group has a numeric index.
• Sample first, map production fields explicitly, and treat warnings as review signals.

The Actor does not crawl, use proxies, authenticate to external sites, embed, call an LLM, repair data, or copy complete source rows to output.

How can you use the API, schedules, webhooks, and integrations?

Run sebastian-actors/rag-dataset-linter through the Apify API or the official Node.js and Python apify-client libraries. Read the summary Dataset, findings Dataset, or OUTPUT record.

Schedules can repeat a saved audit. A Dataset integration supplies an upstream defaultDatasetId; webhooks can notify CI or a workflow after run success or failure. Set failRunOnError according to the webhook signal you need. A separate downstream step must perform embedding or indexing.

Frequently asked questions

Does PASS guarantee good retrieval quality?

No. It means the configured checks found no findings. Retrieval still depends on content, chunking, embeddings, index configuration, and evaluation.

Why is findingCount larger than the findings Dataset?

maxFindings limits stored detail rows, not aggregate counting. Check MAX_FINDINGS_APPLIED and truncated.

Can the Actor repair or deduplicate my Dataset?

No. Use finding codes and record indexes to fix the upstream pipeline.

Is using this Actor legal and responsible?

Only audit data you are authorized to process. You are responsible for applicable law, contracts, retention, and access controls. Outputs contain diagnostic previews, so protect them appropriately. This is not legal advice.

Which related Actors may help?

• Website to RAG Dataset creates chunks from public web pages for this Actor to audit.
• Website Crawl Preflight Auditor checks crawl and extraction risk before corpus creation.

How can you get support?

Open an item in the Actor's Issues tab. Include the run URL, gateStatus or failure.code, relevant warning codes, and a redacted input example. Do not post API tokens, private Dataset contents, or confidential evidence previews.