# Dataset Change Detector: new, removed, changed items (`ucchiii/dataset-change-detector`) Actor

Compare the previous and current dataset of a scheduled scraper and output only what changed: new items, removed items and changed fields with before/after values and % change. Price, stock, jobs and listings monitoring. $0.50 per 1,000 changes.

- **URL**: https://apify.com/ucchiii/dataset-change-detector.md
- **Developed by:** [Actor Factory](https://apify.com/ucchiii) (community)
- **Categories:** Developer tools, Automation, E-commerce
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.35 / 1,000 changes

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Dataset Change Detector — new, removed and changed items between two runs

You run the same scraper every day or week. What you actually need is **what changed**: the price that dropped, the
product that went out of stock, the new job posting, the listing that disappeared. This Actor compares the
**previous dataset** with the **current dataset** and writes only the differences — no scraping, no proxies, no
code.

- **Added / removed / changed** items, one row per change
- **Field-level diff**: `field`, `before`, `after` and `deltaPct` (% change for numbers)
- **Smart matching**: match items by `url`, `id`, your own key fields (e.g. `asin`) or, as a last resort, a hash of the item
- **Watch or ignore fields**: track only `price` and `inStock`, ignore noise like `scrapedAt`; nested paths such as `offer.price` work
- **Numeric threshold**: ignore price moves smaller than, say, 1%
- Optionally output `unchanged` items too
- **$0.50 per 1,000 changes.** Reading datasets is free; only the items written are billed.

Typical uses: price monitoring, stock alerts, new job postings, real-estate listing changes, competitor catalog tracking.

### Use cases

- Price and stock monitoring: rerun an e-commerce scraper daily and get only price drops and out-of-stock items
- New and removed job postings between two runs of a job scraper
- Real-estate or marketplace listings that appeared or disappeared since last week
- Alert workflows: send only the changes to Slack, email or a sheet instead of the full dataset

### Input

```json
{
  "previousDatasetId": "<dataset ID of last week's run>",
  "currentDatasetId": "<dataset ID of this week's run>",
  "keyFields": ["url"],
  "watchFields": ["price", "inStock"],
  "ignoreFields": ["scrapedAt"],
  "numericChangeThresholdPct": 1,
  "outputTypes": ["added", "removed", "changed"]
}
```

Leave both dataset fields empty to see a free demo on sample (fictional) product data.

### Output (one row per change)

```json
{
  "changeType": "changed",
  "key": "https://shop.example.com/p/1",
  "changedFields": [{"field": "price", "before": 80, "after": 68, "deltaPct": -15}],
  "current": {"url": "https://shop.example.com/p/1", "name": "Trail Backpack 30L", "price": 68, "inStock": true},
  "previous": {"url": "https://shop.example.com/p/1", "name": "Trail Backpack 30L", "price": 80, "inStock": true}
}
```

- `added`: `previous` is `null`. `removed`: `current` is `null`. `unchanged`: only `current` is filled.
- Run counts (`added`, `removed`, `changed`, `unchanged`, `previousItems`, `currentItems`, `written`) are saved in the key-value store as `STATS`.

### Use it with a schedule

1. Schedule your scraper in Apify (e.g. daily). Each run creates a new default dataset.
2. Create a schedule or an integration (webhook on "run succeeded") that runs this Actor with the **last two** dataset IDs.
   With the Apify API or a small webhook handler you can pass `previousDatasetId` / `currentDatasetId` automatically.
3. Send the result to Slack, email, Google Sheets or your own system with an Apify integration.

Tip: if every run scrapes the same pages, put a stable field (`url`, `id`, `asin`) in `keyFields`.
If an item has no key, items are matched by a hash of their whole content (minus ignored fields), so any edit shows up as one removed and one added item.

### Notes

- **Memory:** the previous dataset is held in memory as a dictionary of key to (hash, item). Memory use grows with the
  size of the *previous* dataset (roughly the size of its JSON); the current dataset is streamed. For multi-million-item
  datasets, choose an Actor memory of several GB or slim the previous dataset down to the fields you need first.
- If the same key appears twice in one dataset, the last previous item and the first current item are used.
- If the charge limit you set for the run is reached, the Actor stops cleanly and `STATS.stoppedAtChargeLimit` is `true`.
- This Actor only reads datasets you already own; it does not access any website. You are responsible for how the
  underlying data was collected and used.

### More tools from the same developer

- [Dataset Merge & Dedup](https://apify.com/ucchiii/dataset-dedup-merge) — merge Apify datasets and remove duplicates by the fields you choose
- [Dataset Transform](https://apify.com/ucchiii/dataset-transform) — flatten, filter, rename and convert any Apify dataset, export CSV / JSON / JSONL with a link
- [Google Sheets Export](https://apify.com/ucchiii/google-sheets-export) — export an Apify dataset to Google Sheets
- [Google Maps Leads Cleaner](https://apify.com/ucchiii/maps-leads-cleaner) — dedupe Google Maps scraper results, format phones (E.164), CRM-ready leads
- [Job Postings Normalizer](https://apify.com/ucchiii/job-postings-normalizer) — unify fields, salaries and dates from any job scraper and remove duplicates

# Actor input Schema

## `previousDatasetId` (type: `string`):

Dataset ID of the earlier scraper run (its 'defaultDatasetId', e.g. 'WkzbQMuFYuamGv3YF'): the baseline. Required for real results; leave both datasets empty for a free demo on sample data.

## `currentDatasetId` (type: `string`):

Dataset ID of the latest scraper run (its 'defaultDatasetId', e.g. 'Hs8qLk2VnBd9RtXyz'). Items are compared against the previous dataset. Required for real results; leave empty for a free demo.

## `keyFields` (type: `array`):

Fields that identify the same item across runs, e.g. \['url'] or \['asin']. Dot paths allowed. Empty = automatic: url, then id, then a hash of the whole item.

## `watchFields` (type: `array`):

Only changes in these fields count (e.g. \['price', 'inStock', 'offer.price']). Empty = all fields.

## `ignoreFields` (type: `array`):

Fields never compared, such as scrapedAt or position. Nested paths allowed. If left empty, common timestamp fields (scrapedAt, crawledAt, fetchedAt, loadedAt, timestamp) are ignored automatically.

## `numericChangeThresholdPct` (type: `number`):

A numeric field that moved by less than this percentage is not treated as a change. 0 = report every change.

## `outputTypes` (type: `array`):

Which kinds of results to write: any of 'added' (new items), 'removed' (items gone), 'changed' (items with changed fields), 'unchanged'. Default: added, removed, changed. Every written item is billed as one change, including unchanged items if you select them.

## Actor input object example

```json
{
  "numericChangeThresholdPct": 0,
  "outputTypes": [
    "added",
    "removed",
    "changed"
  ]
}
```

# Actor output Schema

## `results` (type: `string`):

One item per change: changeType (added, removed, changed), key, changedFields (before/after values and % change) and the current and previous item.

## `stats` (type: `string`):

Record STATS: run counts (items compared, added, removed, changed).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("ucchiii/dataset-change-detector").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("ucchiii/dataset-change-detector").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call ucchiii/dataset-change-detector --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,ucchiii/dataset-change-detector"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/jMzIQwg3hLt6lEm78/builds/foaWE9M5CdVYjvVbD/openapi.json