# Dataset Drift & QA Monitor (`cynix_dev/dataset-drift-qa`) Actor

Stop finding out your scrapers broke three days late. Point this actor at any Apify dataset or JSON endpoint and it watches the data itself — not just whether the run succeeded.

- **URL**: https://apify.com/cynix\_dev/dataset-drift-qa.md
- **Developed by:** [Cynix Dev](https://apify.com/cynix_dev) (community)
- **Categories:** Developer tools, Automation, Agents
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.25 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Dataset Drift & QA Monitor

Stop finding out your scrapers broke three days late. Point this Actor at any **Apify dataset or JSON endpoint** and it watches **the data itself** — schema, row counts, null rates, numeric distributions and row-level changes — not just whether the run exited zero.

### What it does

A scraper that returns 200 OK and an empty `price` field on every row is broken, but every monitoring tool built around run status will call it healthy. This Actor profiles the output data and compares each run's fingerprint against the previous one.

It alerts on the failures that actually matter in production:

- **New or missing fields** — the site changed its markup and your selector silently stopped matching.
- **Row-count deltas** — yesterday 5,000 rows, today 12.
- **Null-rate jumps** — the field is still there but is now empty.
- **Numeric distribution shifts** — prices suddenly 200% higher, or parsed as strings.
- **New-value bursts** — a categorical field filling with unexpected values.
- **Row-level changes** — matched by your identity `keys`, so you see *which* records changed.

Alerts route to a webhook, Slack, Discord or email.

### Features

- **Watches data, not run status** — catches silent breakage that status monitors miss.
- **Six alert classes** — new/missing fields, row-count delta, null-rate delta, numeric-stats delta, new-value burst, row-level changes.
- **Tunable sensitivity** — every threshold is a percentage you set in `alertOn`.
- **Any JSON source** — an Apify `datasetId` or any URL returning a JSON array.
- **Row-level matching** — set identity `keys` and get per-record change detection.
- **Slack and Discord built in**, plus generic signed webhooks and email.
- **Field scoping** — `fieldsToWatch` limits profiling to the columns that matter.
- **Isolated state** — `storeName` keeps separate monitors from colliding.

### What people use it for

- Monitor your own scheduled scrapers for silent breakage.
- Data-quality gates in an ETL pipeline before loading a warehouse.
- Vendor API monitoring — catch upstream schema changes before they hit production.
- Regression detection after deploying a scraper change.
- SLA reporting — a durable record of data health over time.

### Tuning `alertOn`

Every threshold is a percentage, so you can set sensitivity per signal:

```json
{
  "alertOn": {
    "newFields": true,
    "missingFields": true,
    "rowCountDeltaPct": 20,
    "nullRateDeltaPct": 25,
    "newValueBurstPct": 40,
    "numericStatsDeltaPct": 30,
    "rowLevelChangesPct": 15
  }
}
```

Start permissive, watch a week of real runs, then tighten. Data that is naturally volatile (prices, inventory) needs looser numeric thresholds than a reference dataset that should barely move.

#### Row-level change detection

Set `keys` to the fields that identify a row — `["id"]`, `["productId"]`, `["sku", "region"]`. The Actor then matches rows across runs and reports which specific records changed, rather than just telling you the aggregate moved. Without `keys`, monitoring is aggregate-only.

#### The first run is a baseline

There's no previous fingerprint on run one, so it establishes the baseline and won't alert. Real comparisons begin on the second run.

### Input

Supply either `datasetId` (an Apify dataset) or `datasetUrl` (any JSON array endpoint). Set `keys` if you want row-level change detection.

| Field | Type | Default | What it does |
| --- | --- | --- | --- |
| `datasetId` | string | — | ID of an Apify dataset to monitor (e.g. the dataset produced by one of your scheduled actors). |
| `datasetUrl` | string | `https://jsonplaceholder.typicode.com/users` | OR a URL returning a JSON array (another monitored source). |
| `fieldsToWatch` | array | `[]` | Subset of fields to profile. Empty = all top-level fields. |
| `keys` | array | `[]` | For object datasets: stable identity keys for cross-run row matching (e.g. \['id'] or \['productId']). |
| `alertOn` | object | `{"newFields": true, "missingFields": true, "rowCountDeltaPct": 20, "nullRateDeltaPct": 25, "newValueBurstPct": 40, "numericStatsDeltaPct": 30, "rowLevelChangesPct": 15}` | Toggle individual alerts and set sensitivity (percent deltas). |
| `storeName` | string | `dataset-drift-qa-state` | Named key-value store holding the previous fingerprint. |
| `webhookUrl` | string | — | POST alert payloads to this URL on every run (or only when alerts fire). |
| `webhookEvents` | array | `["on_alert"]` | When to fire webhook: 'always' (every run), 'on\_alert' (only when status=ALERT), 'on\_change' (schema/rowcount changed). |
| `webhookSecret` | string | — | HMAC-SHA256 secret for verifying webhook payloads. If set, X-Signature header is included. |
| `slackWebhookUrl` | string | — | Incoming webhook URL for Slack. Sends formatted message on alert. |
| `discordWebhookUrl` | string | — | Discord webhook URL. Sends embed on alert. |
| `emailAlert` | object | `{"enabled": false, "to": [], "subjectPrefix": "[Dataset Drift Alert]"}` | Email notification settings. Requires APIFY\_TOKEN with email scope or external SMTP service. |

#### Input example

```json
{
  "datasetUrl": "https://api.apify.com/v2/key-value-stores/dcDAdW7LfWrrUKAIt/records/fixture?REDACTED",
  "keys": [
    "id"
  ],
  "alertOn": {
    "newFields": true,
    "missingFields": true,
    "rowCountDeltaPct": 20,
    "nullRateDeltaPct": 25,
    "newValueBurstPct": 40,
    "numericStatsDeltaPct": 30,
    "rowLevelChangesPct": 15
  },
  "storeName": "ddqa-proof-final",
  "webhookEvents": [
    "on_alert"
  ],
  "emailAlert": {
    "enabled": false,
    "to": [],
    "subjectPrefix": "[Dataset Drift Alert]"
  }
}
```

### Output

One record per check: when it ran, the source, row and field counts, an overall `status` (`OK` or `ALERT`), the alert count, and the individual alerts with previous versus current values and the percentage change.

Every dataset record contains: `checkedAt`, `source`, `rowCount`, `fieldCount`, `alertCount`, `status`, `alerts`, `fingerprint`, `deliveredTo`.

#### Output example

A real record from a run of this Actor:

```json
{
  "checkedAt": "2026-08-21T00:34:24.659Z",
  "source": "https://api.apify.com/v2/key-value-stores/dcDAdW7LfWrrUKAIt/records/fixture?REDACTED",
  "rowCount": 3,
  "fieldCount": 3,
  "alertCount": 3,
  "status": "ALERT",
  "alerts": [
    {
      "type": "numeric_stats_delta",
      "field": "price",
      "stat": "max",
      "prev": 30,
      "now": 99,
      "pct": 230
    },
    {
      "type": "numeric_stats_delta",
      "field": "price",
      "stat": "mean",
      "prev": 20,
      "now": 46.3333,
      "pct": 131.7
    },
    {
      "type": "row_level_changes",
      "changed": 1,
      "added": 0,
      "removed": 0,
      "pct": 33.3
    }
  ],
  "fingerprint": {
    "rowCount": 3,
    "fields": [
      "id",
      "name",
      "price"
    ],
    "nullRate": {
      "id": 0,
      "name": 0,
      "price": 0
    },
    "valueSets": {
      "id": [
        "1",
        "2",
        "3"
      ],
      "name": [
        "BETA-CHANGED",
        "alpha",
        "gamma"
      ],
      "price": [
        "10",
        "30",
        "99"
      ]
    },
    "numericStats": {
      "id": {
        "min": 1,
        "max": 3,
        "mean": 2
      },
      "price": {
        "min": 10,
        "max": 99,
        "mean": 46.3333
      }
    },
    "rowHashes": {
      "1": [
        "98a18e525b5ddb38"
      ],
      "2": [
        "59dd4a5192244817"
      ],
      "3": [
        "7a0f703b21de0954"
      ]
    }
  },
  "deliveredTo": []
}
```

Export the dataset as JSON, CSV, Excel, XML or JSONL from the Console, or pull it programmatically through the Apify API and any of the official clients.

### How to use it

1. Click **Try for free** (or **Start** if you already have an Apify account).
2. Fill in the input fields described above — the defaults already produce a working run.
3. Press **Start** and watch the log; results stream into the dataset as they are found.
4. When the run finishes, open the **Output/Storage** tab and export as JSON, CSV or Excel.

Runs can be scheduled (hourly, daily, weekly) and wired into Slack, Google Sheets, Zapier, Make, webhooks or your own backend through Apify integrations. Everything the Console does is also available over the [Apify API](https://docs.apify.com/api/v2).

### Pricing

This Actor is billed on Apify's **pay-per-event** model: a small charge when a run starts, plus a charge for each result written to the dataset. You only pay for records you actually receive — a run that finds nothing costs only the start event. Current rates are always shown on the **Pricing** tab of this page, and the run log prints your usage as it goes.

Free-plan credits from Apify cover a large amount of light usage, so you can evaluate the Actor before committing to anything.

### FAQ

#### Why isn't run status enough?

Because the worst scraper failures succeed. A site changes its markup, your selector stops matching, and the run finishes cleanly with empty fields. Only profiling the data catches that.

#### What can it monitor?

Any Apify dataset by `datasetId`, or any URL returning a JSON array via `datasetUrl` — including your own API endpoints and third-party APIs.

#### How do I connect it to Slack?

Paste an incoming-webhook URL into `slackWebhookUrl`. Discord works the same way via `discordWebhookUrl`. Both send formatted alerts.

#### I'm getting too many alerts. What should I change?

Raise the percentage thresholds in `alertOn`, narrow `fieldsToWatch` to the columns that matter, and switch webhook events to `on_alert` so healthy runs stay quiet.

#### Does it store my data?

It stores a statistical fingerprint — schema, counts, null rates, numeric summaries and row hashes — in the named key-value store, not your full dataset.

#### Can one Actor watch several datasets?

Run one task per dataset with its own `storeName`. That keeps each baseline clean and makes alerts unambiguous.

### Other Actors by cynix\_dev

| Actor | What it does |
| --- | --- |
| [Website to RAG Chunks](https://apify.com/cynix_dev/web-to-rag-chunks) | Crawl any website and turn its pages into clean, chunked, metadata-rich Markdown records ready for RAG pipelines, vector stores, … |
| [Page Change Monitor](https://apify.com/cynix_dev/page-change-monitor) | Monitor web pages for content changes. Diffs each run against the previous snapshot and emits structured change records with … |
| [OpenStreetMap Geocoder](https://apify.com/cynix_dev/osm-geocoder) | Forward and reverse geocoding via the free Komoot Photon / OpenStreetMap service. No API key, no scraping, ODbL data. |
| [Page Change Monitor](https://apify.com/cynix_dev/page-change-monitor) | Monitor web pages for content changes. Diffs each run against the previous snapshot and emits structured change records with … |
| [Website to RAG Chunks](https://apify.com/cynix_dev/web-to-rag-chunks) | Crawl any website and turn its pages into clean, chunked, metadata-rich Markdown records ready for RAG pipelines, vector stores, … |

### Legal and responsible use

This Actor collects only publicly available information. You are responsible for how you use the data, including compliance with the target site's Terms of Service, robots directives, copyright, and data protection law such as GDPR and CCPA. Do not use it to gather personal data without a lawful basis.

### Support and feedback

Found a bug, hit a site change, or need an extra field? Open a ticket on the **Issues** tab of this Actor — issues are read and fixed. Feature requests and custom-scraper enquiries are welcome through the same channel.

# Actor input Schema

## `datasetId` (type: `string`):

ID of an Apify dataset to monitor (e.g. the dataset produced by one of your scheduled actors).

## `datasetUrl` (type: `string`):

OR a URL returning a JSON array (another monitored source).

## `fieldsToWatch` (type: `array`):

Subset of fields to profile. Empty = all top-level fields.

## `keys` (type: `array`):

For object datasets: stable identity keys for cross-run row matching (e.g. \['id'] or \['productId']).

## `alertOn` (type: `object`):

Toggle individual alerts and set sensitivity (percent deltas).

## `storeName` (type: `string`):

Named key-value store holding the previous fingerprint.

## `webhookUrl` (type: `string`):

POST alert payloads to this URL on every run (or only when alerts fire).

## `webhookEvents` (type: `array`):

When to fire webhook: 'always' (every run), 'on\_alert' (only when status=ALERT), 'on\_change' (schema/rowcount changed).

## `webhookSecret` (type: `string`):

HMAC-SHA256 secret for verifying webhook payloads. If set, X-Signature header is included.

## `slackWebhookUrl` (type: `string`):

Incoming webhook URL for Slack. Sends formatted message on alert.

## `discordWebhookUrl` (type: `string`):

Discord webhook URL. Sends embed on alert.

## `emailAlert` (type: `object`):

Email notification settings. Requires APIFY\_TOKEN with email scope or external SMTP service.

## Actor input object example

```json
{
  "datasetUrl": "https://jsonplaceholder.typicode.com/users",
  "fieldsToWatch": [],
  "keys": [],
  "alertOn": {
    "newFields": true,
    "missingFields": true,
    "rowCountDeltaPct": 20,
    "nullRateDeltaPct": 25,
    "newValueBurstPct": 40,
    "numericStatsDeltaPct": 30,
    "rowLevelChangesPct": 15
  },
  "storeName": "dataset-drift-qa-state",
  "webhookUrl": "",
  "webhookEvents": [
    "on_alert"
  ],
  "webhookSecret": "",
  "slackWebhookUrl": "",
  "discordWebhookUrl": "",
  "emailAlert": {
    "enabled": false,
    "to": [],
    "subjectPrefix": "[Dataset Drift Alert]"
  }
}
```

# Actor output Schema

## `dataset` (type: `string`):

One record per run: schema fingerprint, row counts, null-rate and distribution deltas, and triggered alerts.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "alertOn": {
        "newFields": true,
        "missingFields": true,
        "rowCountDeltaPct": 20,
        "nullRateDeltaPct": 25,
        "newValueBurstPct": 40,
        "numericStatsDeltaPct": 30,
        "rowLevelChangesPct": 15
    },
    "emailAlert": {
        "enabled": false,
        "to": [],
        "subjectPrefix": "[Dataset Drift Alert]"
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("cynix_dev/dataset-drift-qa").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "alertOn": {
        "newFields": True,
        "missingFields": True,
        "rowCountDeltaPct": 20,
        "nullRateDeltaPct": 25,
        "newValueBurstPct": 40,
        "numericStatsDeltaPct": 30,
        "rowLevelChangesPct": 15,
    },
    "emailAlert": {
        "enabled": False,
        "to": [],
        "subjectPrefix": "[Dataset Drift Alert]",
    },
}

# Run the Actor and wait for it to finish
run = client.actor("cynix_dev/dataset-drift-qa").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "alertOn": {
    "newFields": true,
    "missingFields": true,
    "rowCountDeltaPct": 20,
    "nullRateDeltaPct": 25,
    "newValueBurstPct": 40,
    "numericStatsDeltaPct": 30,
    "rowLevelChangesPct": 15
  },
  "emailAlert": {
    "enabled": false,
    "to": [],
    "subjectPrefix": "[Dataset Drift Alert]"
  }
}' |
apify call cynix_dev/dataset-drift-qa --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,cynix_dev/dataset-drift-qa"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/H8hg9WEodqAF8Wif9/builds/Y46ptpSvlVv7wqWhU/openapi.json