You can access the Merge, Dedup & Transform Datasets programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

$echo '{
<  "preDedupTransformFunction": "async (items, { Apify }) => {\\n    return items;\\n}",
<  "postDedupTransformFunction": "async (items, { Apify }) => {\\n    return items;\\n}",
<  "customInputData": {}
<}' |
<apify call lukaskrivka/dedup-datasets --silent --output-dataset

Merge, Dedup & Transform Datasets API through CLI

The Apify CLI is the official tool that allows you to use Merge, Dedup & Transform Datasets locally, providing convenience functions and automatic retries on errors.

Install the Apify CLI

$npm i -g apify-cli
$apify login

Other API clients include:

Merge, Dedup & Transform Datasets API in Python

Merge, Dedup & Transform Datasets API in JavaScript

Merge, Dedup & Transform Datasets OpenAPI definition

Merge, Dedup & Transform Datasets API

Batch Runner

useful-tools/batch-runner

Actor for starting several Runs from single Input. It also has capability to merge the Dataset items from the created batch.

Useful tools

5.0

LLM Dataset Processor

dusan.vystrcil/llm-dataset-processor

Allows you to process output of other actors or stored dataset with single LLM prompt. It's useful if you need to enrich data, summarize content, extract specific information, or manipulate data in a structured way using AI.

Dušan Vystrčil

Storage Cleaner

useful-tools/storage-cleaner

Storage cleaner is a tool to delete all your Storage types (Datasets, Key-Value storages and Request queues) on your Apify user account based on the amount of days since they were last accessed.

Useful tools

Forward Dataset to Actor or Task

valek.josef/forward-dataset-to-actor-or-task

Forwards contents of specified dataset to a specified field on the input of another Actor or task.

Josef Válek

Google Sheets Import & Export

lukaskrivka/google-sheets

Import data from datasets or JSON files to Google Sheets. Programmatically process data in Sheets. Easier and faster than the official Google Sheets API and perfect for importing data from scraping.

Lukáš Křivka

1.5K

5.0

Apify Run Queue

lexis-solutions/apify-run-queue

Apify utility to queue runs until memory is available. Workaround to memory exceeded errors on Apify. This actor will retry starting actor starts with a delay. Open source and free!