Merge & Dedup Datasets: Remove Duplicates, Only New Items
Pricing
from $1.90 / 1,000 batch of 1,000 output items
Go to Apify Store
Merge & Dedup Datasets: Remove Duplicates, Only New Items
Merge any number of Apify datasets and remove duplicates by the fields you choose (or the whole item). Keep the first or newest copy, or output only items not seen in earlier runs: ideal after a scheduled scraper. Streams millions of items. Duplicates are free; $0.0019 per 1,000 items output.