Answer Drift Planner: Knowledge Base Change Review

Know which answers need another look when your knowledge base changes.

Compare earlier and later inventories of your knowledge sources, then follow the relationships you supply to answers, citations and tests. Get a focused list of items to recheck, with missing coverage and missing links kept visible.

This is not source-code review. Here, knowledge sources mean help articles, policies, product documents and similar reference material—not software source code. You supply records describing those materials; the Actor compares their recorded versions, hashes and other evidence. It does not read the documents' meaning, discover dependencies or decide whether an answer is correct.

Try the example · Prepare your files · Returned-results example · Pricing · Help

What you receive

Open REPORT.html for the review. report.json retains structured findings and exact relationship references; summary.txt provides a portable text summary.

The review distinguishes Revalidation required, Source coverage incomplete, Dependency mapping missing and No change in mapped source fields. Findings can overlap. A lack of mapped change is not proof that an answer is correct or a test unnecessary.

For example, a changed policy revision can flag a linked support answer and its test for another look. A test with no supplied relationship remains a mapping question; it does not disappear simply because its dependency is missing. This is an illustrative use case, not a claim that the Actor understands the policy change.

The Actor does not connect to your chatbot or repository, call a model, assess answer truth, inspect software for defects, execute tests or decide which tests can safely be skipped.

Who this helps

Best suited to teams that already maintain knowledge-source inventories and answer/test relationships. A support or knowledge-operations manager can use the result to organize rechecks after a content update and see whether supplied follow-up results match the material under review.

Preparing those relationships is still your responsibility. A website, repository or folder of documents alone is not enough. For a small collection with few links, a maintained spreadsheet may be sufficient. The value of this review is in following supplied relationships consistently and keeping gaps and older results visible—not automatically validating your knowledge base.

Try the fictional example

Open the knowledge-base example, or select Try the fictional example in the form. Check the maximum cost, start the run and open REPORT.html from Output. No knowledge-system connection or model account is required.

The example uses fictional inventories and links. Normal pricing applies: $0.25 per completed report plus $0.00005 per start. It is not a free exception to that price.

Use your prepared inventories and links

Choose Review my files. Supply the earlier knowledge-source inventory, later inventory, items to assess and declared dependency links. An inventory lists document or section identities and recorded versions, hashes and other evidence; it is not the document text. Include each inventory's capture time, scope and completeness.

Prepared CSV templates and column mapping avoid hand-writing nested link objects. The strict JSON route is also available. Keep document, section, product version, locale and visibility identifiers explicit and consistent. Do not include protected source text, prompt bodies, personal information or credentials.

Ordinary evaluation exports may not contain the required source relationships. A dataset row, trace parent or retrieved document does not automatically establish which answer or test depends on a source section. Supply missing relationships separately. The tool does not invent links to make a report look complete.

A filtered or failed capture is not a complete inventory; absence from it does not establish document deletion. Hashes and capture declarations are supplied evidence, not independently authenticated facts. There is no blanket native compatibility promise for LangSmith, Copilot or another vendor's exports.

Review returned retest results

See which retest results belong to this change.

Bring back the results from the tests you chose to run. Match them to the knowledge-source changes and test versions under review, and see missing results, older results and disagreements without losing the original review questions.

This compares the records you supply. It does not run a model, verify how a test was performed, prove an answer correct or approve a release.

Choose Review my files and keep the prepared inventories and explicit dependencies for the change under review. Add the optional Retest evidence review object with contract: "answer-retest-evidence-v1", expectedTests and receipts. Each expected test uses its full scoped identity and an explicit definitionRevision; the product version, filename and timestamp are not test-definition revisions. Missing definitions remain unbound.

To prepare requests, supply an empty receipts array. RETEST_REQUEST.json lists the affected declared tests with their exact identities, expected definition revisions and per-test dependency-context fingerprints. It contains no submitted result. All declared and referenced tests remain visible in report.json, including unchanged tests and unresolved mappings.

After performing the tests separately, add returned receipts with a literal receiptId, the exact test, definitionRevision and contextFingerprint, the declared result (pass, fail, inconclusive or not_run), and executedAt when known. Run another review with the inventories and dependencies you intend those results to cover. The request template is a different contract; copying it unchanged cannot clear review work.

Applicability and outcome are separate. An older or differently scoped result is kept visible even if it says pass or has a recent execution time. A matching pass does not remove source-change, missing-coverage or missing-mapping findings. Conflicting receipts remain disagreements; no latest-result rule picks a winner. These fingerprints bind supplied context, not authenticated provenance or execution proof.

A later run reviewing returned results is another paid report at the normal price, not a free continuation of the first run.

Fictional returned results

In the Input JSON editor, replace the complete input with the JSON below. It selects Review my files with strict JSON preparation, so the form's CSV prefills do not apply. Check the maximum cost and start one report.

Open REPORT.html: T-A has a matching supplied pass, T-B has an older-context supplied pass, and T-gap still has a missing dependency mapping. The original review questions remain visible. These receipts and identifiers are fictional. Normal pricing applies: $0.25 per completed report plus $0.00005 per start.

{
  "mode": "review",
  "format": "strictJson",
  "before": "{\"capture_id\":\"fictional-before\",\"scope_id\":\"fictional-tenant-a-inventory\",\"captured_at\":\"2026-10-07T12:00:00Z\",\"capture_state\":\"complete\",\"records\":[{\"key\":{\"document\":\"fictional-policy-A\",\"section\":\"fictional-policy-section\",\"product\":\"fictional-handbook\",\"version\":\"kb-2026.10\",\"locale\":\"en-US\",\"visibility\":\"fictional-tenant-a-private\"},\"content_hash\":\"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\",\"evidence_hash\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\",\"source_revision\":\"source-A-r1\",\"extractor_version\":\"fictional-extractor-r1\",\"validity\":\"valid\"},{\"key\":{\"document\":\"fictional-policy-B\",\"section\":\"fictional-policy-section\",\"product\":\"fictional-handbook\",\"version\":\"kb-2026.10\",\"locale\":\"en-US\",\"visibility\":\"fictional-tenant-a-private\"},\"content_hash\":\"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\",\"evidence_hash\":\"BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB\",\"source_revision\":\"source-B-r1\",\"extractor_version\":\"fictional-extractor-r1\",\"validity\":\"valid\"}]}",
  "after": "{\"capture_id\":\"fictional-after\",\"scope_id\":\"fictional-tenant-a-inventory\",\"captured_at\":\"2026-10-09T12:00:00Z\",\"capture_state\":\"complete\",\"records\":[{\"key\":{\"document\":\"fictional-policy-A\",\"section\":\"fictional-policy-section\",\"product\":\"fictional-handbook\",\"version\":\"kb-2026.10\",\"locale\":\"en-US\",\"visibility\":\"fictional-tenant-a-private\"},\"content_hash\":\"cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc\",\"evidence_hash\":\"CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC\",\"source_revision\":\"source-A-r2\",\"extractor_version\":\"fictional-extractor-r1\",\"validity\":\"valid\"},{\"key\":{\"document\":\"fictional-policy-B\",\"section\":\"fictional-policy-section\",\"product\":\"fictional-handbook\",\"version\":\"kb-2026.10\",\"locale\":\"en-US\",\"visibility\":\"fictional-tenant-a-private\"},\"content_hash\":\"dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd\",\"evidence_hash\":\"DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD\",\"source_revision\":\"source-B-r2\",\"extractor_version\":\"fictional-extractor-r1\",\"validity\":\"valid\"}]}",
  "dependencies": "{\"items\":[{\"id\":\"T-A\",\"kind\":\"test\",\"product\":\"fictional-help-assistant\",\"version\":\"app-2026.10\",\"locale\":\"en-US\",\"visibility\":\"fictional-tenant-a-private\"},{\"id\":\"T-B\",\"kind\":\"test\",\"product\":\"fictional-help-assistant\",\"version\":\"app-2026.10\",\"locale\":\"en-US\",\"visibility\":\"fictional-tenant-a-private\"},{\"id\":\"T-gap\",\"kind\":\"test\",\"product\":\"fictional-help-assistant\",\"version\":\"app-2026.10\",\"locale\":\"en-US\",\"visibility\":\"fictional-tenant-a-private\"}],\"links\":[{\"from_source\":{\"document\":\"fictional-policy-A\",\"section\":\"fictional-policy-section\",\"product\":\"fictional-handbook\",\"version\":\"kb-2026.10\",\"locale\":\"en-US\",\"visibility\":\"fictional-tenant-a-private\"},\"from_item\":null,\"to\":{\"id\":\"T-A\",\"kind\":\"test\",\"product\":\"fictional-help-assistant\",\"version\":\"app-2026.10\",\"locale\":\"en-US\",\"visibility\":\"fictional-tenant-a-private\"}},{\"from_source\":{\"document\":\"fictional-policy-B\",\"section\":\"fictional-policy-section\",\"product\":\"fictional-handbook\",\"version\":\"kb-2026.10\",\"locale\":\"en-US\",\"visibility\":\"fictional-tenant-a-private\"},\"from_item\":null,\"to\":{\"id\":\"T-B\",\"kind\":\"test\",\"product\":\"fictional-help-assistant\",\"version\":\"app-2026.10\",\"locale\":\"en-US\",\"visibility\":\"fictional-tenant-a-private\"}},{\"from_source\":null,\"from_item\":{\"id\":\"fictional-undeclared-answer\",\"kind\":\"answer\",\"product\":\"fictional-help-assistant\",\"version\":\"app-2026.10\",\"locale\":\"en-US\",\"visibility\":\"fictional-tenant-a-private\"},\"to\":{\"id\":\"T-gap\",\"kind\":\"test\",\"product\":\"fictional-help-assistant\",\"version\":\"app-2026.10\",\"locale\":\"en-US\",\"visibility\":\"fictional-tenant-a-private\"}}]}",
  "retestReview": {
    "contract": "answer-retest-evidence-v1",
    "expectedTests": [
      {
        "test": {
          "id": "T-A",
          "kind": "test",
          "product": "fictional-help-assistant",
          "version": "app-2026.10",
          "locale": "en-US",
          "visibility": "fictional-tenant-a-private"
        },
        "definitionRevision": "test-definition-A-r7"
      },
      {
        "test": {
          "id": "T-B",
          "kind": "test",
          "product": "fictional-help-assistant",
          "version": "app-2026.10",
          "locale": "en-US",
          "visibility": "fictional-tenant-a-private"
        },
        "definitionRevision": "test-definition-B-r3"
      },
      {
        "test": {
          "id": "T-gap",
          "kind": "test",
          "product": "fictional-help-assistant",
          "version": "app-2026.10",
          "locale": "en-US",
          "visibility": "fictional-tenant-a-private"
        },
        "definitionRevision": "test-definition-gap-r2"
      }
    ],
    "receipts": [
      {
        "receiptId": "fictional-receipt-A-current",
        "test": {
          "id": "T-A",
          "kind": "test",
          "locale": "en-US",
          "product": "fictional-help-assistant",
          "version": "app-2026.10",
          "visibility": "fictional-tenant-a-private"
        },
        "definitionRevision": "test-definition-A-r7",
        "contextFingerprint": "answer-retest-context-v1:299e925a5fe075a31bbfdbd9d18172e8b4b85cb4d656b3082cc45fc3cc5e314a",
        "declaredResult": "pass",
        "executedAt": "2026-10-09T12:10:00Z"
      },
      {
        "receiptId": "fictional-receipt-B-old",
        "test": {
          "id": "T-B",
          "kind": "test",
          "locale": "en-US",
          "product": "fictional-help-assistant",
          "version": "app-2026.10",
          "visibility": "fictional-tenant-a-private"
        },
        "definitionRevision": "test-definition-B-r3",
        "contextFingerprint": "answer-retest-context-v1:8b713edc03806cedd874e488f286f585c04e23a2ce4b9758ff42f831557482c9",
        "declaredResult": "pass",
        "executedAt": "2026-10-09T12:11:00Z"
      }
    ]
  }
}

Pricing and limits

$0.25 per completed change-impact report, including this Actor's platform usage, plus $0.00005 per start. There is no additional fee per document, test, link or Dataset row. Event prices are the same across plan tiers. Subscriptions, taxes and unrelated services are separate; check the current Pricing tab before starting.

A completed review can consist entirely of unresolved coverage or relationships and is still billable. Invalid preparation and incomplete or unacknowledged report publication do not initiate the completed-report charge. Startup charges can still apply. An unknown charge request may already have settled.

Input ceilings are 500 source rows combined across both inventories, 5,000 dependency links, 5,000 declared or referenced items and 1,000 supplied retest receipts. Optional expected-test declarations also cannot exceed 5,000. Fields are limited to 128 UTF-8 bytes. Combined input, including mappings, capture metadata and optional retest evidence, is limited to 8 MiB; combined results, including the request template, are also limited to 8 MiB. A separate relationship-expansion bound prevents excessive output. Not every maximum-width combination fits.

Duplicate scoped identities, cycles and invalid mapping stop the affected job before an analysis report is published. Missing dependencies representable by the accepted contract remain visible in a completed report. Limits do not silently remove inconvenient tests.

Privacy, recovery and help

Reports contain selected identifiers, qualifiers and reasons rather than source-document bodies. Identifiers may still be sensitive. Supply only permitted records and retain source owners' access restrictions in your Apify storage settings. This tool is not an access-control service.

The Actor does not connect to private systems, execute or remove tests, call models, relay customer input to our workstation or automatically delete input/output storage.

A stopped run may already have a report or a charge. Inspect Output and retained completion/billing state before starting another. Unknown does not mean nothing was written or charged. Results are published before the completed-report charge is requested; interrupted effects are not blindly replayed.

Use the Issues tab for help, with a run ID and fictional example. Do not post private inventories, source text, credentials or access links.

Existing saved workflows

New users can start with the example. Numbered build 0.2.4 added the returned-retest comparison and request template, as verified on October 9, 2026. Earlier builds such as 0.2.2 provide the ordinary prepared-inventory review but do not gain the retest feature. A saved task can retain an older selection. Preserve original files and read the public release notes before changing it; use an available exact numbered build when a fixed selection is required. Movable tags such as latest and retest220 are not permanent version identifiers.

Prepared inputs and explicit dependency declarations remain the basis of the review. Updating the tool does not validate answer truth, authenticate lineage or clear tests to be skipped. Earlier local file-only prototypes are not selectable public releases.