# Dataset Merge & Dedup Datasets — Filter, Join CSV & Excel (`saliga/dataset-merge-dedupe`) Actor Merge Apify datasets, CSV, JSON and Excel files or Google Sheets into one table. Clean fields, normalise URLs, extract domains, filter, remove duplicates and join (VLOOKUP) in one run. Works with limited permissions. CSV and Excel exports free. $0.50 per 1,000 output rows. - **URL**: https://apify.com/saliga/dataset-merge-dedupe.md - **Developed by:** [Salih Küçükkayış](https://apify.com/saliga) (community) - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.50 / 1,000 output rows This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README Merge **Apify datasets, CSV, TSV, JSON, JSONL and Excel files, and Google Sheets** into one clean table, in one run: **clean fields, normalise URLs, extract domains, filter, remove duplicates, join (VLOOKUP)** with a second table, sort, and export to **CSV, Excel and JSON for free**. Use it on its own or as the last step after any scraper. **$0.50 per 1,000 output rows.** Rows removed by filters or as duplicates are free, saved duplicates are free, file exports are free. ### Quick start Press **Start** with the example input. It uses 11 fictional companies given inline (no account data needed), normalises the website URLs, extracts the domain, keeps companies with 10 or more employees, removes duplicates by domain, sorts by size and saves a CSV: ```json { "rows": [ {"company": "Acme Robotics", "website": "https://www.acme-robotics.example/?utm_source=newsletter", "country": "DE", "employees": 120}, {"company": "ACME Robotics GmbH", "website": "acme-robotics.example", "country": "DE", "employees": 120}, "... 9 more rows ..." ], "normalizeUrlFields": ["website"], "extractDomainFrom": "website", "filters": [{"field": "employees", "op": "gte", "value": 10}], "dedupeFields": ["domain"], "sortBy": "employees", "sortDirection": "desc", "exportFormats": ["csv"] } ``` The example run writes 6 rows from 11 (2 filtered out, 3 duplicates removed). First row: ```json { "company": "Granite Data Systems", "website": "https://granitedata.example?page=2", "country": "US", "employees": 1500, "industry": "Software", "domain": "granitedata.example" } ``` Merge the output of two Actor runs and a Google Sheet, remove duplicate companies by domain, and keep the removed ones: ```json { "datasetIds": ["WkzbQMuFYuamGv3YF", "Xq3vLmN8pRtY2sKdE"], "fileUrls": ["https://docs.google.com/spreadsheets/d//edit#gid=0"], "renameFields": {"Company Name": "company", "URL": "website"}, "extractDomainFrom": "website", "dedupeFields": ["domain"], "outputDuplicates": true, "addSourceColumn": true, "exportFormats": ["csv", "xlsx"] } ``` Add columns from a second table, like VLOOKUP (here: technologies found by a tech-stack run, matched on domain): ```json { "datasetIds": [""], "extractDomainFrom": "website", "joinDatasetId": "", "joinOn": "domain", "joinFields": ["technologies", "cms"], "joinType": "left" } ``` ### What it does Every step is optional. They run in this order: 1. **Merge** any number of sources: Apify datasets (picked in the input, by ID, or as `username~dataset-name`), public file links (CSV, TSV, JSON, JSONL / NDJSON, Excel .xlsx; Google Sheets, Google Drive and Dropbox share links are converted to download links) and rows given inline as JSON. `addSourceColumn` tags each row with where it came from (`_source`). 2. **Clean fields**: rename columns so different sources line up, keep only some fields (dots pull nested values into their own column, e.g. `company.website`), trim spaces, lowercase fields, **normalise URLs** (lowercase host, no `utm_*` or click-ID parameters, no fragment, no trailing slash) and **extract the domain** of a website field into a `domain` column (`https://www.Example.com/about` becomes `example.com`). 3. **Filter** with conditions `{"field", "op", "value"}`: `eq`, `neq`, `contains`, `not_contains`, `regex`, `gt`, `gte`, `lt`, `lte`, `empty`, `not_empty`, `in`. All conditions (AND) or any (OR). Text comparisons ignore case by default; `gt`/`lte` compare numbers when both sides are numbers (`"1,200"` counts as 1200) and text otherwise, so ISO dates work. List fields match when any element matches. 4. **Remove duplicates** by one or more fields, or by the whole row. Keep the first or the last row of each group, ignore case and extra spaces (on by default). Rows whose dedupe fields are all empty are never removed, so rows without a website do not collapse into one. With `outputDuplicates`, the removed rows go to a separate dataset of the run instead of being dropped. 5. **Join (VLOOKUP)** with a dataset, a file or inline rows on a key field. Left join keeps every row, inner join keeps only rows with a match. Bring all columns or only the ones you list; a column your rows already have is added with a prefix (`right_` by default), never overwritten. The first matching row of the join table is used. 6. **Sort** by a field (numbers as numbers, empty values last) and **limit** with `maxRows`. 7. **Output**: rows go to the dataset. With `exportFormats`, the same rows are saved as `merged.csv` (UTF-8 with BOM, opens cleanly in Excel), `merged.xlsx` and `merged.json` in the key-value store, with download links in the `OUTPUT` record. Lists and objects become JSON text in CSV and Excel cells. Exports are safe to open in Excel: values that look like formulas (starting with `=`, `+`, `-`, `@`) are written as text, with a `'` in front, so nothing in the data runs as a formula. The `OUTPUT` record also lists rows read per source, rows filtered out, duplicates removed, join matches, rows written, the link to the removed duplicates and warnings (for example a filter field that no row has). ### Use it after a scraper The Actor reads any dataset, so it fits after any Actor run. Add it as an integration on another Actor or task ("Run Actor" after a successful run) with the input below, and it cleans each new result automatically: ```json { "datasetIds": ["{{resource.defaultDatasetId}}"], "dedupeFields": ["url"], "normalizeUrlFields": ["url"], "exportFormats": ["csv"] } ``` Ideas with other Actors by the same developer: - [Career Site Jobs](https://apify.com/saliga/career-site-jobs): merge several runs, remove jobs already seen by `url`, keep only roles matching `regex` on the title. - [Tech Stack Detector](https://apify.com/saliga/tech-stack-detector): join its results onto your lead list by `domain` to add CMS and ecommerce platform columns. - [UK Companies House](https://apify.com/saliga/uk-companies-house): combine daily new-company lists, dedupe by `companyNumber`, filter by SIC code or town. - [Sitemap Extractor](https://apify.com/saliga/sitemap-extractor): merge the sitemaps of several sites, filter URLs with `regex`, compare two runs with an inner join. ### Limited permissions The Actor is built to run with **limited permissions**: it reads only the datasets you give it in the input (the dataset fields are declared as dataset inputs, so the platform grants the run read access to them) and writes only to its own run storages. If a dataset cannot be read, the run stops before writing anything and says which one, and how to pass it. ### Input | Field | Default | What it does | |---|---|---| | `datasetIds` | | Apify datasets to merge (ID, picker, or `username~dataset-name`) | | `fileUrls` | | Public CSV, TSV, JSON, JSONL, XLSX links; Google Sheets, Drive and Dropbox links | | `rows` | | Rows as a JSON array (the example rows are ignored when datasets or files are given, see below) | | `fileFormat`, `excelSheet` | auto, first sheet | Force a file format; pick an Excel sheet | | `addSourceColumn` | false | Add `_source` to each row | | `fields`, `renameFields` | | Keep only these fields; rename old -> new | | `trimStrings` | true | Trim spaces in text values | | `lowercaseFields`, `normalizeUrlFields` | | Lowercase or normalise these fields | | `extractDomainFrom` | | Add a `domain` column from a website field | | `filters`, `filterMode` | all | Filter conditions and how they combine | | `dedupe`, `dedupeFields` | false | Remove duplicates by these fields (or the whole row) | | `keep` | first | Keep the first or last row of each duplicate group | | `dedupeIgnoreCase` | true | Ignore case and extra spaces when comparing | | `outputDuplicates` | false | Save removed duplicates to a separate dataset (free) | | `joinDatasetId` / `joinFileUrl` / `joinRows` | | The table to look values up in | | `joinOn`, `joinRightOn` | | Key field of your rows, and of the join table | | `joinType` | left | `left` keeps rows without a match, `inner` drops them | | `joinFields`, `joinPrefix` | all, `right_` | Columns to bring over; prefix for clashing names | | `sortBy`, `sortDirection` | asc | Sort field and direction | | `maxRows` | 0 (all) | Stop after this many output rows | | `exportFormats` | | `csv`, `xlsx`, `json` files in the key-value store (free) | ### What it costs | Run | Price | |---|---| | The example run (6 output rows) | $0.003 | | Two scraper runs of 5,000 rows each, 3,000 duplicates removed (7,000 output rows) | $3.50 | | A 50,000-row list filtered down to 8,000 rows, with CSV and Excel files | $4.00 | You pay only for rows written to the output dataset. Rows read, filtered out or removed as duplicates, the duplicates dataset and the CSV / Excel / JSON files are free. `maxRows` caps the price; the run also stops cleanly at your spending limit and still writes the files and summary for the rows it wrote. ### Limits and good to know - **Size**: without sorting and without "keep last", rows stream through and only a small fingerprint per dedupe key stays in memory, so large datasets (millions of rows) are limited mainly by run time. Sorting and keeping the last duplicate hold all rows in memory: in a local test, 300,000 rows of 10 short fields took about 360 MB, so plan on roughly 500,000 such rows per GB of memory (default 1 GB; raise the memory for more). The join table is held in memory too. - **Files** are downloaded to disk (up to 100 MB each) and read in full before anything is written, so a broken file stops the run before any row is charged. Text files can be UTF-8, UTF-16 or Windows-1252. Excel files that unpack to more than about 1 GB are refused. Excel files hold at most 1,048,576 rows; bigger outputs are complete in the dataset, CSV and JSON. - **Datasets** are read in pages of 5,000 rows. The run stops about 3 minutes before its timeout and keeps everything written so far; raise the timeout for very large merges. - **Google Sheets** must be shared as "Anyone with the link can view" (or published to the web). The tab in the link (`gid`) is the one read. - Values from CSV and Google Sheets are text; filters and sorting still compare numbers as numbers. - Only public file links are downloaded; private and internal network addresses are blocked. Links to (and redirects through) YouTube, TikTok, Instagram, Facebook, X/Twitter, LinkedIn, Reddit and the Amazon store are refused; files on Amazon S3 / AWS are fine. - The `regex` filter gives up on a value after 0.05 seconds (a pattern with nested repeats like `(a|aa)+` can take forever) and counts it as no match; the `OUTPUT` warnings say how often that happened. - **The example settings**: the input form opens with example rows and example settings (URL clean-up and a `domain` column from `website`, a filter on `employees`, dedupe by `domain`, sort by `employees`, a CSV file). When you add your own datasets or file links and leave the example rows in place, the example rows **and every example setting still at its example value** are ignored, and the log says which. To use one of those settings on your own data, clear the example rows first. ### FAQ **Can it merge datasets with different column names?** Yes: rename the columns with `renameFields` so they line up. Columns that exist in only some sources stay, and are empty in the other rows. **Which duplicate is kept?** The first one in source order (sources are read in the order: datasets, files, inline rows), or the last one with `keep: "last"`. Put the source you trust most first. **Can an AI agent use it?** Yes, through the Apify MCP server: give it rows inline or dataset IDs and ask for "dedupe by domain and save as CSV". ### More from this developer - [Career Site Jobs](https://apify.com/saliga/career-site-jobs): open jobs from company career sites (Greenhouse, Lever, Ashby, Workable, Recruitee, Workday). - [Tech Stack Detector](https://apify.com/saliga/tech-stack-detector): CMS, ecommerce platform, analytics and hosting of a list of websites. - [UK Companies House](https://apify.com/saliga/uk-companies-house): UK company profiles and new-company lists from the official API. - [Sitemap Extractor](https://apify.com/saliga/sitemap-extractor): every URL a website lists in its sitemaps. - [Bulk HTTP Requests](https://apify.com/saliga/bulk-http-requests): call any API once for every row of a CSV, JSON list or Apify dataset. Code examples (curl, Python, Node.js, n8n) for this and the other Actors: [github.com/msalihk/apify-actor-examples](https://github.com/msalihk/apify-actor-examples). Questions or a missing feature: open an issue on the Actor page. # Actor input Schema ## `datasetIds` (type: `array`): Datasets to merge, for example the output of other Actor runs. Pick them here (or paste their IDs) so the run gets permission to read them, also when this Actor runs with limited permissions. A named dataset can also be given as username~dataset-name. Datasets are read in pages of 5,000 rows. ## `fileUrls` (type: `array`): Public links to CSV, TSV, JSON, JSONL / NDJSON or Excel (.xlsx) files, up to 100 MB each. Google Sheets links work when the sheet is shared as "Anyone with the link" (the tab in the link is used); Google Drive and Dropbox share links work too. ## `rows` (type: `array`): Rows given directly as a JSON array of objects. Handy for tests and AI agents. The example rows are fictional. When you also give datasets or file links, they are ignored, and so is every example setting of this form still at its example value (clear the example rows to keep those settings). ## `fileFormat` (type: `string`): Format of the file links. Detected from the link, the server's content type and the first bytes when left on automatic. ## `excelSheet` (type: `string`): Sheet to read in Excel files. Empty: the first sheet. ## `addSourceColumn` (type: `boolean`): Tag each row with where it came from: dataset:, the file link, or rows. ## `fields` (type: `array`): Empty: keep all fields. Names after renaming; company.website pulls a nested value into its own column. The domain and \_source columns are kept automatically when you create them. ## `renameFields` (type: `object`): Old name -> new name, e.g. {"Company Name": "company", "URL": "website"}. Use it to line up columns from different sources before merging. ## `trimStrings` (type: `boolean`): Remove spaces at the start and end of every text value. ## `lowercaseFields` (type: `array`): Text in these fields becomes lowercase, e.g. email domains or country codes. ## `normalizeUrlFields` (type: `array`): URLs in these fields get a lowercase host, no utm\_\* or click-ID parameters (gclid, fbclid, msclkid ...), no #fragment and no trailing slash; https:// is added to bare domains. Good before deduplicating by URL. ## `extractDomainFrom` (type: `string`): A URL or website field. Adds a domain column (www. removed, lowercase): https://www.Example.com/about -> example.com. Dedupe or join on it. ## `filters` (type: `array`): List of {"field", "op", "value"}. Ops: eq, neq, contains, not_contains, regex, gt, gte, lt, lte, empty, not_empty, in (value is a list). Text comparisons ignore case unless the condition has "caseSensitive": true. gt/gte/lt/lte compare numbers when both sides are numbers (1,200 counts), text otherwise (ISO dates work). The example keeps companies with 10 or more employees; clear it for your own data. ## `filterMode` (type: `string`): How the conditions combine. ## `dedupe` (type: `boolean`): Remove duplicate rows. With no dedupe fields, rows count as duplicates only when every field is equal (the \_source column is ignored). ## `dedupeFields` (type: `array`): Rows with the same values in these fields are duplicates, e.g. domain, or name + city. Rows where all of these fields are empty are never removed. ## `keep` (type: `string`): Which row of each duplicate group stays. Keeping the last row holds all rows in memory. ## `dedupeIgnoreCase` (type: `boolean`): "Acme Inc" and " acme inc" count as the same value. ## `outputDuplicates` (type: `boolean`): Write the removed duplicates to a separate dataset of this run (linked in the OUTPUT record) instead of dropping them. Free. ## `joinDatasetId` (type: `string`): A dataset to look values up in. ## `joinFileUrl` (type: `string`): Public CSV, TSV, JSON, JSONL or Excel link, or a Google Sheet. ## `joinRows` (type: `array`): A JSON array of objects to look values up in. ## `joinOn` (type: `string`): Field of your rows to match on, e.g. domain. ## `joinRightOn` (type: `string`): Empty: the same name as above. ## `joinType` (type: `string`): Left join keeps every row; inner join keeps only rows with a match. Only rows written to the output are charged. ## `joinFields` (type: `array`): Empty: every column of the join table except its key. ## `joinPrefix` (type: `string`): When your rows already have a column of the same name, the joined value is added as , so nothing is overwritten. ## `joinIgnoreCase` (type: `boolean`): "ACME.example" matches "acme.example". The first matching row of the join table is used. ## `sortBy` (type: `string`): Numbers sort as numbers, text alphabetically; empty values go last. Sorting holds all rows in memory. ## `sortDirection` (type: `string`): Ascending (A to Z, small to large) or descending. ## `maxRows` (type: `integer`): Stop after this many output rows. 0: no limit. Also a cap on the price. ## `exportFormats` (type: `array`): Save the output rows as files in the key-value store, with download links in the OUTPUT record. Free. ## Actor input object example ```json { "rows": [ { "company": "Acme Robotics", "website": "https://www.acme-robotics.example/?utm_source=newsletter", "country": "DE", "employees": 120, "industry": "Robotics" }, { "company": "ACME Robotics GmbH", "website": "acme-robotics.example", "country": "DE", "employees": 120, "industry": "Robotics" }, { "company": "Bluefin Analytics", "website": "https://bluefin-analytics.example/about", "country": "GB", "employees": 45, "industry": "Software" }, { "company": "Cedar & Stone Builders", "website": "http://cedarstone.example", "country": "US", "employees": 8, "industry": "Construction" }, { "company": "Delta Freight Partners", "website": "https://www.deltafreight.example/contact/", "country": "NL", "employees": 310, "industry": "Logistics" }, { "company": "Bluefin Analytics Ltd", "website": "https://BLUEFIN-ANALYTICS.example/?utm_campaign=q3", "country": "GB", "employees": 47, "industry": "Software" }, { "company": "Evergreen Solar Co", "website": "https://evergreen-solar.example", "country": "ES", "employees": 64, "industry": "Energy" }, { "company": "Fjord Bio", "website": "https://fjordbio.example/en/", "country": "NO", "employees": 5, "industry": "Biotech" }, { "company": "Granite Data Systems", "website": "https://granitedata.example/?utm_medium=cpc&page=2", "country": "US", "employees": 1500, "industry": "Software" }, { "company": "Harbor Lane Coffee", "website": "", "country": "US", "employees": 22, "industry": "Food" }, { "company": "Delta Freight Partners B.V.", "website": "deltafreight.example", "country": "NL", "employees": 305, "industry": "Logistics" } ], "fileFormat": "auto", "addSourceColumn": false, "trimStrings": true, "normalizeUrlFields": [ "website" ], "extractDomainFrom": "website", "filters": [ { "field": "employees", "op": "gte", "value": 10 } ], "filterMode": "all", "dedupe": false, "dedupeFields": [ "domain" ], "keep": "first", "dedupeIgnoreCase": true, "outputDuplicates": false, "joinType": "left", "joinPrefix": "right_", "joinIgnoreCase": true, "sortBy": "employees", "sortDirection": "desc", "maxRows": 0, "exportFormats": [ "csv" ] } ``` # Actor output Schema ## `results` (type: `string`): The merged, cleaned, filtered, deduplicated and joined rows. ## `summary` (type: `string`): Rows read per source, filtered out, duplicates removed, join matches, rows written, links to the CSV / Excel / JSON files and to the removed duplicates. ## `csv` (type: `string`): Present when CSV export is on. ## `xlsx` (type: `string`): Present when Excel export is on. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "rows": [ { "company": "Acme Robotics", "website": "https://www.acme-robotics.example/?utm_source=newsletter", "country": "DE", "employees": 120, "industry": "Robotics" }, { "company": "ACME Robotics GmbH", "website": "acme-robotics.example", "country": "DE", "employees": 120, "industry": "Robotics" }, { "company": "Bluefin Analytics", "website": "https://bluefin-analytics.example/about", "country": "GB", "employees": 45, "industry": "Software" }, { "company": "Cedar & Stone Builders", "website": "http://cedarstone.example", "country": "US", "employees": 8, "industry": "Construction" }, { "company": "Delta Freight Partners", "website": "https://www.deltafreight.example/contact/", "country": "NL", "employees": 310, "industry": "Logistics" }, { "company": "Bluefin Analytics Ltd", "website": "https://BLUEFIN-ANALYTICS.example/?utm_campaign=q3", "country": "GB", "employees": 47, "industry": "Software" }, { "company": "Evergreen Solar Co", "website": "https://evergreen-solar.example", "country": "ES", "employees": 64, "industry": "Energy" }, { "company": "Fjord Bio", "website": "https://fjordbio.example/en/", "country": "NO", "employees": 5, "industry": "Biotech" }, { "company": "Granite Data Systems", "website": "https://granitedata.example/?utm_medium=cpc&page=2", "country": "US", "employees": 1500, "industry": "Software" }, { "company": "Harbor Lane Coffee", "website": "", "country": "US", "employees": 22, "industry": "Food" }, { "company": "Delta Freight Partners B.V.", "website": "deltafreight.example", "country": "NL", "employees": 305, "industry": "Logistics" } ], "normalizeUrlFields": [ "website" ], "extractDomainFrom": "website", "filters": [ { "field": "employees", "op": "gte", "value": 10 } ], "dedupeFields": [ "domain" ], "sortBy": "employees", "sortDirection": "desc", "exportFormats": [ "csv" ] }; // Run the Actor and wait for it to finish const run = await client.actor("saliga/dataset-merge-dedupe").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "rows": [ { "company": "Acme Robotics", "website": "https://www.acme-robotics.example/?utm_source=newsletter", "country": "DE", "employees": 120, "industry": "Robotics", }, { "company": "ACME Robotics GmbH", "website": "acme-robotics.example", "country": "DE", "employees": 120, "industry": "Robotics", }, { "company": "Bluefin Analytics", "website": "https://bluefin-analytics.example/about", "country": "GB", "employees": 45, "industry": "Software", }, { "company": "Cedar & Stone Builders", "website": "http://cedarstone.example", "country": "US", "employees": 8, "industry": "Construction", }, { "company": "Delta Freight Partners", "website": "https://www.deltafreight.example/contact/", "country": "NL", "employees": 310, "industry": "Logistics", }, { "company": "Bluefin Analytics Ltd", "website": "https://BLUEFIN-ANALYTICS.example/?utm_campaign=q3", "country": "GB", "employees": 47, "industry": "Software", }, { "company": "Evergreen Solar Co", "website": "https://evergreen-solar.example", "country": "ES", "employees": 64, "industry": "Energy", }, { "company": "Fjord Bio", "website": "https://fjordbio.example/en/", "country": "NO", "employees": 5, "industry": "Biotech", }, { "company": "Granite Data Systems", "website": "https://granitedata.example/?utm_medium=cpc&page=2", "country": "US", "employees": 1500, "industry": "Software", }, { "company": "Harbor Lane Coffee", "website": "", "country": "US", "employees": 22, "industry": "Food", }, { "company": "Delta Freight Partners B.V.", "website": "deltafreight.example", "country": "NL", "employees": 305, "industry": "Logistics", }, ], "normalizeUrlFields": ["website"], "extractDomainFrom": "website", "filters": [{ "field": "employees", "op": "gte", "value": 10, }], "dedupeFields": ["domain"], "sortBy": "employees", "sortDirection": "desc", "exportFormats": ["csv"], } # Run the Actor and wait for it to finish run = client.actor("saliga/dataset-merge-dedupe").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "rows": [ { "company": "Acme Robotics", "website": "https://www.acme-robotics.example/?utm_source=newsletter", "country": "DE", "employees": 120, "industry": "Robotics" }, { "company": "ACME Robotics GmbH", "website": "acme-robotics.example", "country": "DE", "employees": 120, "industry": "Robotics" }, { "company": "Bluefin Analytics", "website": "https://bluefin-analytics.example/about", "country": "GB", "employees": 45, "industry": "Software" }, { "company": "Cedar & Stone Builders", "website": "http://cedarstone.example", "country": "US", "employees": 8, "industry": "Construction" }, { "company": "Delta Freight Partners", "website": "https://www.deltafreight.example/contact/", "country": "NL", "employees": 310, "industry": "Logistics" }, { "company": "Bluefin Analytics Ltd", "website": "https://BLUEFIN-ANALYTICS.example/?utm_campaign=q3", "country": "GB", "employees": 47, "industry": "Software" }, { "company": "Evergreen Solar Co", "website": "https://evergreen-solar.example", "country": "ES", "employees": 64, "industry": "Energy" }, { "company": "Fjord Bio", "website": "https://fjordbio.example/en/", "country": "NO", "employees": 5, "industry": "Biotech" }, { "company": "Granite Data Systems", "website": "https://granitedata.example/?utm_medium=cpc&page=2", "country": "US", "employees": 1500, "industry": "Software" }, { "company": "Harbor Lane Coffee", "website": "", "country": "US", "employees": 22, "industry": "Food" }, { "company": "Delta Freight Partners B.V.", "website": "deltafreight.example", "country": "NL", "employees": 305, "industry": "Logistics" } ], "normalizeUrlFields": [ "website" ], "extractDomainFrom": "website", "filters": [ { "field": "employees", "op": "gte", "value": 10 } ], "dedupeFields": [ "domain" ], "sortBy": "employees", "sortDirection": "desc", "exportFormats": [ "csv" ] }' | apify call saliga/dataset-merge-dedupe --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,saliga/dataset-merge-dedupe" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/iUWm9bLqFzMMAnXDg/builds/pglEz0WAxFfN5dVjs/openapi.json