Diff Datasets

Pricing

Pay per usage

Diff Datasets

Take one dataset on Apify platform, compare it to another, and output the missing ones. This can also be used to output only changed items, using a compound key

Pricing

Pay per usage

Rating

0.0

(0)

Developer

Paulo Cesar

Maintained by Community

Actor stats

Bookmarked

Total users

Monthly active users

3 years ago

Last modified

Example

await Apify.call('pocesar/diff-datasets', {
    baseDatasetId: 'LdNAlaOY1aKGhwAah', // place the datasets here. The order of "base" and "other" matters
    otherDatasetId: 'Bzu1pgOjenN43VhPY', // existing items in "base" are not output from "other"
    uniqueFields: [
        // simple primitive field value, like string, number, boolean
        "pageUrl",

        // you can use lodash.get notation to get nested items,
        // in this case `sub.fields.0` works like `sub.fields[0]` and the object looks like
        // {
        //    pageUrl: "https//pageurl",
        //    sub: {
        //      fields: [
        //        {...},
        //        {...}
        //      ]
        //    }
        //  }
        "sub.fields.0",

        // you can also use .length to count arrays or string characters, as in
        "sub.fields.length",
        "pageUrl.length"
    ],
});

Limitations

Every value is kept in memory while reading from the base dataset, more items more memory needed.
The key value store might choke when trying to save the in-memory Set with too many items

License

Apache 2.0

Diff Dataset Fields

lukaskrivka/diff-dataset-fields

Compare changes in text fields between two datasets. Monitor new and updated content on websites.

Lukáš Křivka

DIFF Eyewear Scraper

mshopik/diff-eyewear-scraper

Scrape DIFF Eyewear and extract data on eyewear from diffeyewear.com. Our DIFF Eyewear API lets you crawl product information and pricing. The saved data can be downloaded as HTML, JSON, CSV, Excel, and XML.

Mark Carter

Google Dataset Items Translator

web.harvester/google-dataset-items-translator

Translate any dataset field(s) to any of the supported languages using the Google Translate website, it goes through all the items in the dataset and translates all of the selected fields

Web Harvester

Website Screenshot Generator

apify/screenshot-url

Create a screenshot of a website based on a specified URL. The screenshot is stored as the output in a key-value store. It can be used to monitor web changes regularly after setting up the scheduler.

Apify

3.6K

4.1

Validate Dataset(s) with JSON Schema

jaroslavhejlek/validate-dataset-with-json-schema

This Actor validates items in one or more datasets against a provided JSON Schema. Use it if you planning to add a dataset validation schema to your actor and you want test it.

Jaroslav Hejlek

Monday.com Exporter (Pay per event)

odaudlegur/monday-com-exporter-ppe

Exports items from an Apify dataset to Monday.com as items or subitems. Designed to be triggered automatically when a source actor run succeeds, but can also run manually.

SLASH

Sort Dataset Items

lukaskrivka/sort-dataset-items

Add this actor as a webhook to your scraper to sort the dataset by index field

Lukáš Křivka

Yet Another Dataset Translator

mvolfik/yet-another-dataset-translator

Actor to translate datasets with field selection and source language detection. Requires Google Translate API Key.

Matěj Volf

Monday.com Exporter (Subscription)

odaudlegur/monday-com-exporter-subscription

Unlimited exports of items from an Apify dataset to Monday.com as items or subitems. Designed to be triggered automatically when a source actor run succeeds, but can also run manually. If you want to try it out, please see the pay per event version.

SLASH