
Diff Datasets
Pricing
Pay per usage
Go to Store

Diff Datasets
Take one dataset on Apify platform, compare it to another, and output the missing ones. This can also be used to output only changed items, using a compound key
0.0 (0)
Pricing
Pay per usage
3
Total users
13
Monthly users
2
Runs succeeded
0%
Last modified
3 years ago
Take one dataset on Apify platform, compare to another, and output the missing ones. This can also be used to output only changed items, using a compound key.
Supports using whole nested objects as value, they are JSON.stringify
'd before being turned
into a small non-cryptographic space efficient hash
Example
await Apify.call('pocesar/diff-datasets', {baseDatasetId: 'LdNAlaOY1aKGhwAah', // place the datasets here. The order of "base" and "other" mattersotherDatasetId: 'Bzu1pgOjenN43VhPY', // existing items in "base" are not output from "other"uniqueFields: [// simple primitive field value, like string, number, boolean"pageUrl",// you can use lodash.get notation to get nested items,// in this case `sub.fields.0` works like `sub.fields[0]` and the object looks like// {// pageUrl: "https//pageurl",// sub: {// fields: [// {...},// {...}// ]// }// }"sub.fields.0",// you can also use .length to count arrays or string characters, as in"sub.fields.length","pageUrl.length"],});
Limitations
- Every value is kept in memory while reading from the
base
dataset, more items more memory needed. - The key value store might choke when trying to save the in-memory
Set
with too many items
License
Apache 2.0