Pricing

Pay per usage

Try for free

Go to Apify Store

Only New Items | Cross-Run Dedupe Ledger for Any Dataset

Try for free

Run your scraper on a schedule and get back only what you have never seen before. Name the field that identifies an item, pipe any dataset in, and this actor holds back everything already delivered. Records pass through untouched. The ledger is yours, not a database you rent.

Pricing

Pay per usage

Rating

0.0

(0)

Developer

Brandon Mensing

Actor stats

Bookmarked

Total users

Monthly active users

a month ago

Last modified

Categories

Developer tools

Automation

Agents

You can access the Only New Items | Cross-Run Dedupe Ledger for Any Dataset programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in API & Integrations in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1from apify_client import ApifyClient
2
3# Initialize the ApifyClient with your Apify API token
4# Replace '<YOUR_API_TOKEN>' with your token.
5client = ApifyClient("<YOUR_API_TOKEN>")
6
7# Prepare the Actor input
8run_input = {
9    "items": [
10        {
11            "jobId": "a1",
12            "title": "Senior Data Engineer",
13            "company": "Northwind",
14            "url": "https://example.com/jobs/a1",
15        },
16        {
17            "jobId": "b2",
18            "title": "Product Designer",
19            "company": "Northwind",
20            "url": "https://example.com/jobs/b2",
21        },
22        {
23            "jobId": "c3",
24            "title": "Support Analyst",
25            "company": "Contoso",
26            "url": "https://example.com/jobs/c3",
27        },
28    ],
29    "keyFields": ["jobId"],
30}
31
32# Run the Actor and wait for it to finish
33run = client.actor("broomwagon/only-new-items").call(run_input=run_input)
34
35# Fetch and print Actor results from the run's dataset (if there are any)
36print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
37for item in client.dataset(run.default_dataset_id).iterate_items():
38    print(item)
39
40# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

Only New Items: Cross-Run Dedupe for Any Dataset API in Python

The Apify API client for Python is the official library that allows you to use Only New Items | Cross-Run Dedupe Ledger for Any Dataset API in Python, providing convenience functions and automatic retries on errors.

Install the apify-client

$pip install apify-client

Other API clients include:

Only New Items | Cross-Run Dedupe Ledger for Any Dataset API in JavaScript

Only New Items | Cross-Run Dedupe Ledger for Any Dataset API through CLI

Only New Items | Cross-Run Dedupe Ledger for Any Dataset OpenAPI definition

Only New Items | Cross-Run Dedupe Ledger for Any Dataset API

RSS Feed Monitor - New Items Only

technicaldost/rss-feed-monitor

Watch any RSS or Atom feed and get back only the items published since your last run. Remembers what it has already seen, so you never process or pay for the same article twice.

Technical Dost Solutions

Dataset Deduper | Exact & Fuzzy Dedupe for Any Dataset

broomwagon/dataset-deduper

Collapse duplicates in any dataset, in two passes you control. Exact matching ignores case, punctuation, and word order; an optional fuzzy pass catches the near-duplicates normalization cannot. Every decision is explained in an audit trail, and a cross-run ledger remembers what you already got.

Brandon Mensing

Scheduled Job Heartbeat and Silence Ledger

kingii98/scheduled-job-heartbeat-and-silence-ledger

Your cron job sends one curl ping after each run. This Actor keeps the ledger in a named key-value store and reports every job that went silent, with an optional webhook alert. HTTP only, no browser, no proxy, no database.

kingii98

Daily RSS and Atom Monitor: Only Items You Have Not Seen

lotebo-lab/feed-change-watcher

Schedule it on your RSS and Atom feeds. The first run returns everything the feeds carry now, as a baseline. Every run after that returns only entries no earlier run returned, plus a summary row. Failed feeds and feeds robots.txt blocks are not charged.

Tarcio Elyakin Agra Diniz

App Store Review Radar

titan_coder/appstore-review-radar

Every App Store review scraper hands you the same reviews again on every run — and bills you for them. This one remembers what it already showed you and delivers only reviews you have never seen. A quiet day costs nothing at all.

Radu Furtuna

Twitter/X Account Radar

titan_coder/twitter-x-radar

X scrapers hand you the same timeline again on every run, and many want your login. This one needs no account at all, remembers what it already delivered, and charges only for tweets you have never seen. Baseline, repeats and errors are free.

Radu Furtuna

Geekpay Ledger Exporter

landgeek/geekpay-ledger-exporter

This tool is your go-to solution for automating the extraction of ledger entries from GeekPay.

LandGeek

Trustpilot Review Monitoring - New Reviews Only

vonsensey/trustpilot-review-monitoring-scraper-api

Watch any list of companies on Trustpilot and get only the reviews you have not already had. Remembers every review it delivered, so a daily or weekly schedule bills you for new feedback and nothing else. Ratings, full text, company replies, reviewer country. Pay per new review.

Blackcube

Instagram New Posts Monitor API - Schedule It, Pay for Changes

vonsensey/instagram-new-posts-monitor-api

Watch Instagram profiles on a schedule: only posts you have not seen, or whose counts moved, are billed; unchanged posts come back free. A last-7-days window, a feed-check tick per profile, optional monthly pass.

Blackcube

Amazon Product Monitoring Scraper - Change Alerts by ASIN

vonsensey/amazon-product-monitoring-scraper-api

Watch Amazon products and searches on a schedule and get only what changed since the last run - price, availability, sales rank or Buy Box seller. Already-seen products come back free, so a quiet re-run is nearly free. Mix ASINs and search URLs in one feed.

Blackcube