Corpus Coverage Gate

Check a website text export against expected URLs or an earlier crawl before updating a RAG corpus.

Quick start

Paste this input into the Apify JSON input editor and start a run. The example uses synthetic records for a reproducible first audit.

{
  "current": {
    "complete": true,
    "items": [
      {
        "url": "https://docs.example.org/start",
        "text": "Welcome to the documentation. Learn how to configure your project."
      }
    ]
  },
  "baseline": {
    "complete": true,
    "items": [
      {
        "url": "https://docs.example.org/start",
        "text": "Welcome to the documentation. Learn how to configure your project."
      },
      {
        "url": "https://docs.example.org/api",
        "text": "API reference and authentication instructions."
      }
    ]
  },
  "minimumCoverage": 0.98,
  "acceptBaseline": false
}

Results

Read OUTPUT.gate, reasons and coverage; use dataset rows and REPORT.csv for individual findings. Baseline acceptance is explicit, never automatic.

A platform run marked SUCCEEDED means execution finished. Always inspect the application status and unresolved rows before using the result.

Sources and limits

Use current and optional baseline as inline items, completed runId or datasetId. Configure expectedUrls when you have an authoritative list. Default limits are 50,000 records and 64 MiB per source, configurable up to 100,000 records and 256 MiB. A dataset without a completed run cannot independently establish completion.

Limitations

Reports URL coverage, missing pages and text-length regressions. A failing gate is a valid audit result. It neither crawls websites nor proves that a baseline represents the whole website.

Automation and data

Use Apify schedules or its API to run this Actor. Scheduling and external notifications are not enabled automatically. Results stay in the run dataset and key-value store; storage retention follows your Apify settings. Do not include passwords or session cookies in inputs.

Support

Report an issue with the Actor version, run ID, expected result and a small redacted input. Avoid including private customer data or credentials. Owlects is independent of the third-party platforms named here.

Pricing and run budget

Launch Actor fee: USD 0.1 per conclusive audit. Apify platform usage is charged separately. Unresolved or partial audits and blocked results do not generate an Actor fee. See the pricing panel for the active rates.

The maximum cost must cover all requested result fees before work starts. Use smaller batches to reduce the required budget. For remote Funda exports, the budget check uses the 10,000-record source limit.