# KB Refresh Coverage and Removal Review (`remarkable_whipscorpion/kb-refresh-review`) Actor

Compare supplied expected, prior and current URL inventories; review missing URLs with bounded safe HTTP checks.

- **URL**: https://apify.com/remarkable_whipscorpion/kb-refresh-review.md
- **Developed by:** [Kagerou AP](https://apify.com/remarkable_whipscorpion) (community)
- **Stats:** 2 total users, 1 monthly users, 86.7% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / completed valid coverage report

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## KB Refresh Coverage and Removal Review

Compare your expected URLs, previous inventory and current retrieval results in one report. Missing rows receive optional, bounded HTTP checks so you can distinguish an upstream retrieval gap, a current 404/410 observation and an unknown result before reviewing a KB refresh.

Built for agencies delivering knowledge base updates for voice and support agents. Bring the inventories you already own or have permission to use. The benefit is a fixed comparison and review format with fewer joins and HTTP guard settings to assemble; reduced preparation time and repeat paid demand are hypotheses, not measured outcomes.

### What the report means

| Status | Meaning |
|---|---|
| `fetched` | At least one current input row explicitly says `fetched`. This is the caller's statement. |
| `crawl_missing` | A missing supplied URL returned permitted static content with HTTP 2xx. It was not a successful current input row. |
| `observed_404_410` | This check observed HTTP 404 or 410. Review only; it does not establish deletion. |
| `unknown` | Checks were disabled, unsafe, restricted, unsuccessful or capped. The reason is retained. |

`removeReviewCandidate` is true only for a URL in the prior inventory with a new 404/410 observation. It is a human review queue, never an instruction to delete a KB entry. A transient response, redirect or partial retrieval can change the observation. No website completeness, document deletion or actual KB ingestion is proven. The Actor neither discovers URLs nor uploads, edits or deletes a KB.

### Input contract

Send a JSON object with `authorized: true`, `allowedOrigins` and up to three inventories: `expected`, `prior`, `current`. Each inventory is `{ "rows": [...], "complete": false, "sourceAt": null }`. `complete` is a required boolean when an inventory is supplied. `sourceAt` must be an ISO timestamp or null. Omitted inventories default to empty/partial. These fields are caller claims and are returned as such.

Expected/prior rows are `{ "url": "https://...", "id": "optional-original-id" }`. Current rows also require `status: "fetched"` or `"error"`; an optional `reason` is preserved. IDs/reasons are strings of at most 256 characters. Success is not inferred from an upstream tool's other field names; map your exported results explicitly. Only the known fields enter the report. Malformed rows or unsafe URLs reject the whole input and incur no custom event.

`allowedOrigins` lists exact permitted HTTP/HTTPS origins, such as `https://docs.example.com`, without paths, queries or fragments. There are at most 100 origins. Checking a URL or redirect on another origin yields `source_restricted`. Use only public static sources you may check; no credentials, cookies, authentication headers or bypass of access restrictions. Do not supply secret or authenticated URLs. Source permission does not follow automatically from a robots rule.

`checkMissing` defaults to true. False gives a comparison without network checks. It is useful for reviewing inventories whose sources cannot be fetched safely.

Limits: **1,000 total rows across all inventories, including repeated rows; 100 distinct missing URLs; concurrency 4; 5 seconds per DNS/request operation; 60-second HTTP run deadline; 3 redirect follows per missing URL; 2 MiB per response and 20 MiB total response data including robots and redirects.** Publisher run timeout is 60 seconds. Storage/SDK initialization may be interrupted by the platform timeout; unfinished work is not a completed report. At most 500 HTTP requests can result from 100 checks, their robots files and redirect follows; the shared deadline/bytes can stop much earlier. Caps cannot be changed by input.

URL identity is the WHATWG absolute URL minus fragment. Host case/default ports normalize; locale, path case, version, trailing slash and query order/values remain distinct. Identical URLs share one check within a run. Each output record retains original URLs, inventory, row indices and IDs; each inventory's input order is preserved in its references. A current success and error on the same URL yield `fetched` plus `conflictingCurrentRows: true`. One run produces at most one paid report; a fresh run is a new purchase.

### Example

[`sample-input.json`](sample-input.json) is the fixed prefilled demo: synthetic expected/prior English and Japanese URL variants, a caller-reported successful English row and a Japanese timeout. Checks are disabled and permitted origins are empty, so this useful inventory comparison needs no source requests. The UI prefill and Actor example run input acknowledge permission only for this demo. The authorization default remains false; explicit custom JSON that omits authorization or supplies false is rejected. A run with no explicit input uses the configured demo example input.

```json
{
  "schemaVersion": "1.0",
  "status": "valid_partial",
  "usefulComparison": true,
  "counts": {
    "inputRows": 6,
    "uniqueURLs": 2,
    "expectedFetched": 1,
    "expectedMissing": 1,
    "fetched": 1,
    "unknown": 1
  }
}
```

This is an excerpt. [`sample-output.json`](sample-output.json) contains the complete generated report, references and scope. Its `chargedCount: null` marks an uncharged example. Generate it with `npm run sample`.

### Output and API use

The full report is a single JSON record named **OUTPUT in the run's default key-value store**, which remains authoritative. A successfully delivered useful report is also mirrored unchanged as one item in the default dataset for normal dataset consumers and platform quality checks. Empty, failed-only and budget diagnostics have no report mirror. Its fields are `schemaVersion`, `generatedAt`, `status`, `usefulComparison`, `counts`, `records`, `scope` and `billing`. Each record includes `url`, `wasExpected`, `wasPrior`, `wasFetched`, `status`, `reason`, `observedAt`, `removeReviewCandidate`, `addedToCurrent`, `conflictingCurrentRows` and `refs`. HTTP observations also include `httpStatus` and `finalUrl`. `observedAt` is null for supplied retrieval success and unknown checks; caller `sourceAt` remains in `scope.callerMetadata`.

Counts include supplied/unique/repeated row totals, expected/prior matches and misses, unexpected fetched URLs, each result status, review candidates and actual HTTP requests/response bytes. `duplicateRows` is inputRows minus uniqueURLs across all three inventories, including references repeated across inventories. Counts do not extrapolate beyond supplied scope.

Use the native authenticated API with your existing client authorization:

1. `POST /v2/acts/{actorId}/runs?memory=256&timeout=60&maxTotalChargeUsd=1.00005` with the JSON input.
2. Poll `GET /v2/actor-runs/{runId}` until the run finishes; retain its `defaultKeyValueStoreId`.
3. `GET /v2/key-value-stores/{defaultKeyValueStoreId}/records/OUTPUT` returns the report. The output schema also declares this link as `output.report`.

An insufficient unit budget returns a `budget_exhausted` diagnostic without records and without a custom event. Invalid input fails before a custom event. Empty input produces `empty`; current errors alone without supplied inventories or current successes produce `failed_only`, both with custom charge zero. A valid supplied-inventory join can be `valid_partial` with unknowns even if all optional HTTP checks fail.

The documented synchronous alternative is `POST /v2/actors/{actorId}/run-sync?outputRecordKey=OUTPUT&memory=256&timeout=60&maxTotalChargeUsd=1.00005&restartOnError=false` with the same JSON input. It returns the OUTPUT record if completed within the API's connection/time constraints. This route is documented but has not been executed for this Actor; a one-call response is not yet a validated product claim. If the connection drops, inspect the existing run before starting another purchase. [Synchronous OUTPUT API](https://docs.apify.com/api/v2/actor-run-sync-post).

For native MCP, use `fetch-actor-details`, then `call-actor` with input and `callOptions` for memory/timeout/spending limits. `call-actor` returns status and storage IDs; if needed, poll `get-actor-run`, then read OUTPUT with `get-key-value-store-record`. Confirm exact argument names from the connected tool schema. This requires execution and record retrieval calls; it does not promise the report in one MCP call. API, MCP and output schema behavior in a cloud account are not yet validated. Normal dataset retrieval exposes the same useful report mirror after successful delivery; the dataset-only sync route has not been validated for this revision. [Native MCP tools](https://docs.apify.com/mcp), [API record retrieval](https://docs.apify.com/api/v2/key-value-store-record-get), [native output links](https://docs.apify.com/actors/development/actor-definition/output-schema).

### Pricing

Proposed Store price: **$1 per completed valid report** (`coverage_report`). One useful partial report also counts once: it must contain a supplied expected/prior inventory comparison or a caller-reported current success, with counts and unknown scope. Empty, invalid and failed-only current rows without useful coverage incur **zero custom events**. Rows, duplicates and HTTP retries never add custom events.

Apify also automatically charges the native Actor start event, currently **$0.00005 at the configured 256 MiB**, including zero-custom-event runs. The code never manually charges it. The publisher must remove paid default-dataset-item pricing; the report mirror uses a direct uncharged SDK Dataset write and the Actor checks the cloud PPE configuration before work. The mirror adds no custom event. The $1 price is a hypothesis awaiting Store configuration, not proof of realized revenue. Platform usage is a creator cost under PPE; the buyer's source preparation/upstream tools and account plan are separate. Free-plan runs do not establish eligible creator sales. Model fees and sustained margins are unmeasured. [Native PPE specification](https://docs.apify.com/actors/publishing/monetize/pay-per-event), [pricing and creator costs](https://docs.apify.com/actors/publishing/monetize/pricing-and-costs).

The wrapper checks remaining capacity and the unrounded spent-plus-unit amount before work and immediately before its serial one-unit charge. SDK capacity rounding alone can allow a tiny budget deficit; the amount check blocks that case. It examines `chargedCount`, not merely the limit flag, and preserves the final paid report when that charge exhausts the budget. Both OUTPUT and REPORT_STATE durably contain the comparison before charge; failure of either initial save incurs no custom event. OUTPUT remains readable through the same KVS/API/MCP record route after a charge response is lost, a post-charge checkpoint fails or the process aborts. A failed or aborted run can require direct KVS retrieval; the synchronous endpoint is not guaranteed to return an aborted run's record.

`billing.state` distinguishes `pending` (no confirmed charge result yet, possibly in flight), `ambiguous` (an attempt's outcome needs native event inspection), `confirmed` (SDK returned `chargedCount: 1`) and `not_charged` (SDK returned zero or no custom event was requested). Pending/ambiguous records have `chargedCount: null`; a valid comparison status alone does not mean payment completed. A checkpoint prevents charging the same saved unit twice. Normal ready/delivered units resume without another charge. Final OUTPUT and the delivered checkpoint are saved before the dataset projection; a lost/failed dataset write leaves the authoritative report accessible and is not automatically replayed. A crash at that boundary can leave the dataset empty. A charging checkpoint refuses automatic retry while retaining OUTPUT access; inspect native charged events before resolving that run. If the confirmation cannot be persisted, the earlier pending comparison stays available without a paid-completion claim. Restart recovery does not promise exactly-once external payment after arbitrary platform/storage failure. [SDK charge result](https://docs.apify.com/sdk/js/reference/interface/ChargeResult).

### Source safety and limitations

Only HTTP/HTTPS on standard ports are permitted. Every DNS answer must be a public unicast IPv4/IPv6 address; documentation ranges including `2001:db8::/32` and `3fff::/20` are rejected. Each connection is pinned to a validated address and its peer is checked; every redirect is revalidated. No proxy, browser or API fallback is used. Robots is checked and cached per origin; disallow, positive crawl-delay, unavailable or unsupported robots yields unknown. Robots 404/410 permits checking. A 429 response cancels in-flight requests and stops further requests for this run; there is no retry. Restricted 401/403 and unsupported content stay unknown. Explicit login/sign-in redirects return `unknown` with `auth_redirect`; direct authentication routes or a static HTML password form return `unknown` with `auth_required`. Ordinary permitted static redirects remain eligible. This bounded check cannot identify every custom authentication page. [IANA IPv6 registry](https://www.iana.org/assignments/iana-ipv6-special-registry).

Requests use `Accept-Encoding: identity`. Non-identity compression is refused instead of decompressed, closing decompression-bomb paths. Response bytes include robots and redirect bodies. HTML, XHTML and plain text are supported; source bodies are not included in output or routinely logged. JavaScript rendering, paywalls, authenticated sources and selective retries are outside scope. A reachability observation does not validate a page's content or factual accuracy.

### Run, validate and publish

Requires Node 24. Dependencies are pinned with the lockfile: Apify SDK 3.7.2 (Apache-2.0), ipaddr.js 2.2.0 (MIT), robots-parser 3.0.1 (MIT). Runtime dependencies come from those packages; no crawler framework is directly implemented. [IP parser](https://github.com/whitequark/ipaddr.js), [robots parser](https://github.com/samclarke/robots-parser).

Run `npm ci --ignore-scripts`, `npm test` and `npm run sample` in the product directory. On this Windows environment use `C:\Program Files\nodejs\npm.cmd` and `node.exe`; the approved normal test runner resolves sandbox child-process EPERM. `--test-isolation=none` can run unit tests but does not resolve SDK memory-probe EPERM. Do not treat a skipped or blocked check as a pass. SDK local test pricing is simulated at $1 and proves the wrapper path, not a configured Store price.

The separate authorized publisher must complete independent QA/simplification and market/TOS review, then build privately from these files with the existing approved CLI or Console Web IDE. `.actor/ppe.json` is a configuration contract, not an automatically applied monetization setting. Set `coverage_report` primary event to $1, keep native start, remove default dataset event, require limited permissions and 256 MiB/60 seconds, and run private 0/below-price/1-unit and useful-partial samples within the approved cash cap. Check native charged events, OUTPUT, output link and actual costs. Only then publish with the existing authorization. Login, billing details and KYC belong to the separate publisher/owner route; no key paste is needed here.

Cloud build/run, actual Store pricing, public listing, API/MCP calls and sales are unexecuted. Before publication, management must establish cumulative creator spend monitoring and an actual disable/unpublish control within the $10 initial/$5 post-listing loss caps; per-run caps alone cannot enforce a cumulative limit. No scheduled monitoring or automatic future continuation is promised by this product.

# Actor input Schema

## `authorized` (type: `boolean`):

Confirm you own or may use these inventories and check the specified public static sources. The prefilled permission applies to the synthetic no-network inventory demo below; confirm permission for your own input. No login, restricted source or secrets.

## `allowedOrigins` (type: `array`):

Up to 100 exact http/https origins, e.g. https://example.com. Redirects to any other origin stay unknown. Empty in the no-network demo.

## `checkMissing` (type: `boolean`):

At most 100 distinct missing URL checks; robots and source safety may refuse checks. Disabled in the synthetic demo.

## `expected` (type: `object`):

{rows:\[{url,id?}], complete:boolean, sourceAt:ISO timestamp or null}. Expected scope is supplied inventory only. Prefilled rows are synthetic.

## `prior` (type: `object`):

{rows:\[{url,id?}], complete:boolean, sourceAt:ISO timestamp or null}. Preserve original IDs. Prefilled rows are synthetic.

## `current` (type: `object`):

{rows:\[{url,id?,status:'fetched'|'error',reason?}],complete:boolean,sourceAt:ISO timestamp or null}. Success must be explicit; no guess from field names. Prefilled rows are synthetic.

## Actor input object example

```json
{
  "authorized": true,
  "allowedOrigins": [],
  "checkMissing": true,
  "expected": {
    "rows": [
      {
        "url": "https://example.com/en/v1?lang=en",
        "id": "expected-en"
      },
      {
        "url": "https://example.com/ja/v1?lang=ja",
        "id": "expected-ja"
      }
    ],
    "complete": true,
    "sourceAt": "2026-10-09T00:00:00Z"
  },
  "prior": {
    "rows": [
      {
        "url": "https://example.com/en/v1?lang=en",
        "id": "prior-en"
      },
      {
        "url": "https://example.com/ja/v1?lang=ja",
        "id": "prior-ja"
      }
    ],
    "complete": true,
    "sourceAt": "2026-10-09T00:00:00Z"
  },
  "current": {
    "rows": [
      {
        "url": "https://example.com/en/v1?lang=en",
        "id": "current-en",
        "status": "fetched"
      },
      {
        "url": "https://example.com/ja/v1?lang=ja",
        "id": "current-ja-error",
        "status": "error",
        "reason": "upstream_timeout"
      }
    ],
    "complete": false,
    "sourceAt": "2026-10-10T00:00:00Z"
  }
}
```

# Actor output Schema

## `report` (type: `string`):

One JSON KVS OUTPUT record: schemaVersion,status,counts,records,scope,billing. Inspect billing.state: pending/ambiguous is unconfirmed (chargedCount null); confirmed has chargedCount 1. Comparison is saved before charge and survives charge interruptions. Budget exhaustion returns diagnostic only; no default dataset items.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "authorized": true,
    "allowedOrigins": [],
    "checkMissing": false,
    "expected": {
        "rows": [
            {
                "url": "https://example.com/en/v1?lang=en",
                "id": "expected-en"
            },
            {
                "url": "https://example.com/ja/v1?lang=ja",
                "id": "expected-ja"
            }
        ],
        "complete": true,
        "sourceAt": "2026-10-09T00:00:00Z"
    },
    "prior": {
        "rows": [
            {
                "url": "https://example.com/en/v1?lang=en",
                "id": "prior-en"
            },
            {
                "url": "https://example.com/ja/v1?lang=ja",
                "id": "prior-ja"
            }
        ],
        "complete": true,
        "sourceAt": "2026-10-09T00:00:00Z"
    },
    "current": {
        "rows": [
            {
                "url": "https://example.com/en/v1?lang=en",
                "id": "current-en",
                "status": "fetched"
            },
            {
                "url": "https://example.com/ja/v1?lang=ja",
                "id": "current-ja-error",
                "status": "error",
                "reason": "upstream_timeout"
            }
        ],
        "complete": false,
        "sourceAt": "2026-10-10T00:00:00Z"
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("remarkable_whipscorpion/kb-refresh-review").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "authorized": True,
    "allowedOrigins": [],
    "checkMissing": False,
    "expected": {
        "rows": [
            {
                "url": "https://example.com/en/v1?lang=en",
                "id": "expected-en",
            },
            {
                "url": "https://example.com/ja/v1?lang=ja",
                "id": "expected-ja",
            },
        ],
        "complete": True,
        "sourceAt": "2026-10-09T00:00:00Z",
    },
    "prior": {
        "rows": [
            {
                "url": "https://example.com/en/v1?lang=en",
                "id": "prior-en",
            },
            {
                "url": "https://example.com/ja/v1?lang=ja",
                "id": "prior-ja",
            },
        ],
        "complete": True,
        "sourceAt": "2026-10-09T00:00:00Z",
    },
    "current": {
        "rows": [
            {
                "url": "https://example.com/en/v1?lang=en",
                "id": "current-en",
                "status": "fetched",
            },
            {
                "url": "https://example.com/ja/v1?lang=ja",
                "id": "current-ja-error",
                "status": "error",
                "reason": "upstream_timeout",
            },
        ],
        "complete": False,
        "sourceAt": "2026-10-10T00:00:00Z",
    },
}

# Run the Actor and wait for it to finish
run = client.actor("remarkable_whipscorpion/kb-refresh-review").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "authorized": true,
  "allowedOrigins": [],
  "checkMissing": false,
  "expected": {
    "rows": [
      {
        "url": "https://example.com/en/v1?lang=en",
        "id": "expected-en"
      },
      {
        "url": "https://example.com/ja/v1?lang=ja",
        "id": "expected-ja"
      }
    ],
    "complete": true,
    "sourceAt": "2026-10-09T00:00:00Z"
  },
  "prior": {
    "rows": [
      {
        "url": "https://example.com/en/v1?lang=en",
        "id": "prior-en"
      },
      {
        "url": "https://example.com/ja/v1?lang=ja",
        "id": "prior-ja"
      }
    ],
    "complete": true,
    "sourceAt": "2026-10-09T00:00:00Z"
  },
  "current": {
    "rows": [
      {
        "url": "https://example.com/en/v1?lang=en",
        "id": "current-en",
        "status": "fetched"
      },
      {
        "url": "https://example.com/ja/v1?lang=ja",
        "id": "current-ja-error",
        "status": "error",
        "reason": "upstream_timeout"
      }
    ],
    "complete": false,
    "sourceAt": "2026-10-10T00:00:00Z"
  }
}' |
apify call remarkable_whipscorpion/kb-refresh-review --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,remarkable_whipscorpion/kb-refresh-review"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/RtKVy7Hh0heTlWT4G/builds/MyNkGxiEvJyyKfakO/openapi.json