Dataset Change Detector: new, removed, changed items
Pricing
from $0.35 / 1,000 changes
Dataset Change Detector: new, removed, changed items
Returns one row per added, removed or changed item between two Apify datasets (previous and current run of a scheduled scraper), with field, before, after and % change. Match by url, id or your key fields; watch or ignore fields. Price, stock and job monitoring. Empty input = free demo.
Pricing
from $0.35 / 1,000 changes
Rating
0.0
(0)
Developer
Actor Factory
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
a day ago
Last modified
Categories
Features
Share
New, removed and changed items between two runs
You run the same scraper every day or week. What you actually need is what changed: the price that dropped, the product that went out of stock, the new job posting, the listing that disappeared. This Actor compares the previous dataset with the current dataset and returns only the differences — no scraping, no proxies, no code.
- Added / removed / changed items, one row per change
- Field-level diff:
field,before,afteranddeltaPct(% change for numbers) - Smart matching: match items by
url,id, your own key fields (e.g.asin) or, as a last resort, a hash of the item - Watch or ignore fields: track only
priceandinStock, ignore noise likescrapedAt; nested paths such asoffer.pricework - Numeric threshold: ignore price moves smaller than, say, 1%
- Optionally output
unchangeditems too - $0.50 per 1,000 changes; the $5 monthly free credit covers about 10,000 changes. Reading datasets is free; only the items written are billed.
Typical uses: price monitoring, stock alerts, new job postings, real-estate listing changes, competitor catalog tracking.
Use cases
- Price and stock monitoring: rerun an e-commerce scraper daily and get only price drops and out-of-stock items
- New and removed job postings between two runs of a job scraper
- Real-estate or marketplace listings that appeared or disappeared since last week
- Alert workflows: send only the changes to Slack, email or a sheet instead of the full dataset
Input
{"previousDatasetId": "<dataset ID of last week's run>","currentDatasetId": "<dataset ID of this week's run>","keyFields": ["url"],"watchFields": ["price", "inStock"],"ignoreFields": ["scrapedAt"],"numericChangeThresholdPct": 1,"outputTypes": ["added", "removed", "changed"]}
Leave both dataset fields empty to see a free demo on sample (fictional) product data.
Output (one row per change)
{"changeType": "changed","key": "https://shop.example.com/p/1","changedFields": [{"field": "price", "before": 80, "after": 68, "deltaPct": -15}],"current": {"url": "https://shop.example.com/p/1", "name": "Trail Backpack 30L", "price": 68, "inStock": true},"previous": {"url": "https://shop.example.com/p/1", "name": "Trail Backpack 30L", "price": 80, "inStock": true}}
added:previousisnull.removed:currentisnull.unchanged: onlycurrentis filled.- Run counts (
added,removed,changed,unchanged,previousItems,currentItems,written) are saved in the key-value store asSTATS.
Use it with a schedule
- Schedule your scraper in Apify (e.g. daily). Each run creates a new default dataset.
- Create a schedule or an integration (webhook on "run succeeded") that runs this Actor with the last two dataset IDs.
With the Apify API or a small webhook handler you can pass
previousDatasetId/currentDatasetIdautomatically. - Send the result to Slack, email, Google Sheets or your own system with an Apify integration.
Tip: if every run scrapes the same pages, put a stable field (url, id, asin) in keyFields.
If an item has no key, items are matched by a hash of their whole content (minus ignored fields), so any edit shows up as one removed and one added item.
Notes
- Memory: the previous dataset is held in memory as a dictionary of key to (hash, item). Memory use grows with the size of the previous dataset (roughly the size of its JSON); the current dataset is streamed. For multi-million-item datasets, choose an Actor memory of several GB or slim the previous dataset down to the fields you need first.
- If the same key appears twice in one dataset, the last previous item and the first current item are used.
- If the charge limit you set for the run is reached, the Actor stops cleanly and
STATS.stoppedAtChargeLimitistrue. - This Actor only reads datasets you already own; it does not access any website. You are responsible for how the underlying data was collected and used.
Pricing
Pay per event: $0.0005 per change. Failed runs and the demo are free. Store discounts apply automatically: BRONZE −10%, SILVER −20%, GOLD −30%. On the Apify Free plan, the $5 of monthly platform credit covers about 10,000 changes a month — enough to try it on real data.
If this Actor saved you time, a rating on the Store helps others find it.
More tools from the same developer
- Remove Duplicates — merge Apify datasets and remove duplicates by the fields you choose
- Dataset Transform — flatten, filter, rename and convert any Apify dataset, export CSV / JSON / JSONL with a link
- Google Sheets Export — export an Apify dataset to Google Sheets
- Google Maps Leads Cleaner — dedupe Google Maps scraper results, format phones (E.164), CRM-ready leads
- Job Postings Normalizer — unify fields, salaries and dates from any job scraper and remove duplicates