Dataset Transformer - Filter, Dedupe, Merge & Export CSV
Pricing
$1.00 / 1,000 row outputs
Filter, deduplicate, merge, sort, rename and flatten Apify datasets from any scraper, then export clean rows to a dataset, CSV or JSON file.
You can access the Dataset Transformer - Filter, Dedupe, Merge & Export CSV programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in API & Integrations in Apify Console.
{ "openapi": "3.0.1", "info": { "version": "1.0", "x-build-id": "ZCTpp5JBpTQoebTtJ" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/seemuapps~dataset-transformer/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-seemuapps-dataset-transformer", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/seemuapps~dataset-transformer/runs": { "post": { "operationId": "runs-sync-seemuapps-dataset-transformer", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/seemuapps~dataset-transformer/run-sync": { "post": { "operationId": "run-sync-seemuapps-dataset-transformer", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "datasetIds" ], "properties": { "datasetIds": { "title": "Datasets to transform", "type": "array", "description": "One or more dataset IDs, dataset names ('my-leads' or 'username~my-leads'), run IDs (the run's default dataset is used) or dataset/run URLs. Several datasets are merged in the order listed. When chaining from another actor via an integration, use {{resource.defaultDatasetId}}.", "items": { "type": "string" } }, "filters": { "title": "Filters", "type": "array", "description": "Keep only rows that match these rules. Each rule: {\"field\": \"dotted.path\", \"operator\": \"...\", \"value\": ...}. Operators: equals, notEquals, contains, notContains, regex, notRegex, gt, gte, lt, lte, exists, notExists, in, notIn. Text matching is case-insensitive unless the rule sets \"caseSensitive\": true. Numbers stored as text (\"1,234\", \"$19.99\", \"10.5K\") and ISO dates compare correctly. If a path points into a list (e.g. \"posts.likes\"), the rule matches when any item matches. Leave empty to keep every row.", "default": [] }, "filterLogic": { "title": "Combine filters with", "enum": [ "AND", "OR" ], "type": "string", "description": "AND keeps rows that match every rule; OR keeps rows that match at least one rule.", "default": "AND" }, "dedupeBy": { "title": "Remove duplicates by", "type": "array", "description": "Field(s) that identify a duplicate, e.g. 'phone', 'url' or 'username'. With several fields, rows are duplicates only when all of them match. Use '*' to remove rows that are identical in every field. The first occurrence is kept (after sorting, if a sort is set). Rows where all of these fields are empty are always kept. Leave empty to skip deduplication.", "default": [], "items": { "type": "string" } }, "dedupeNormalize": { "title": "Loose duplicate matching", "type": "boolean", "description": "Ignore case and extra spaces, treat '1,000' and 1000 as equal, and compare phone numbers by digits only ('(555) 123-4567' = '555.123.4567'). Turn off for exact matching.", "default": true }, "fields": { "title": "Keep only these fields", "type": "array", "description": "Output only these fields, in this order. Dotted paths reach into nested objects ('address.city'); a path through a list collects every item's value ('images.url'). Leave empty to keep all fields.", "default": [], "items": { "type": "string" } }, "omitFields": { "title": "Remove these fields", "type": "array", "description": "Drop these fields from the output. Dotted paths are supported ('address.geo').", "default": [], "items": { "type": "string" } }, "renameFields": { "title": "Rename fields", "type": "object", "description": "JSON object of old name to new name, e.g. {\"title\": \"businessName\", \"address.city\": \"city\"}. Renaming a dotted path moves that value to the new top-level name.", "default": {} }, "flatten": { "title": "Flatten nested objects", "type": "boolean", "description": "Turn nested objects into dotted columns ({\"address\": {\"city\": \"Austin\"}} becomes \"address.city\": \"Austin\") in the dataset output too. The CSV file is always flat.", "default": false }, "arrayMode": { "title": "Lists when flattening", "enum": [ "join", "index", "json" ], "type": "string", "description": "How lists become columns when flattening (and in the CSV file). Join: lists of text or numbers become 'a, b, c' and lists of objects become JSON text. Index: every item gets its own column (tags.0, tags.1, images.0.url). JSON: every list becomes JSON text.", "default": "join" }, "sortBy": { "title": "Sort by field", "type": "string", "description": "Sort the output by this field (dotted paths allowed). Numbers stored as text and ISO dates sort correctly; rows without the field go last. Leave empty to keep the input order." }, "sortOrder": { "title": "Sort order", "enum": [ "asc", "desc" ], "type": "string", "description": "Ascending (A to Z, smallest first) or descending (Z to A, largest first).", "default": "asc" }, "offset": { "title": "Skip first N rows", "minimum": 0, "type": "integer", "description": "Skip this many rows of the final result (after filtering, deduplication and sorting).", "default": 0 }, "maxItems": { "title": "Max rows", "minimum": 0, "type": "integer", "description": "Stop after this many output rows. 0 means no limit.", "default": 0 }, "addSourceField": { "title": "Add source dataset column", "type": "boolean", "description": "Add a '_sourceDataset' field with the name or ID of the dataset each row came from. Useful when merging several runs.", "default": false }, "pushToDataset": { "title": "Save rows to the output dataset", "type": "boolean", "description": "Write the transformed rows to this run's dataset, so you can export them in any format or chain another actor.", "default": true }, "exportCsv": { "title": "Save a CSV file", "type": "boolean", "description": "Also save all rows as OUTPUT.csv in the key-value store (always flat, with a header row).", "default": true }, "exportJson": { "title": "Save a JSON file", "type": "boolean", "description": "Also save all rows as a single OUTPUT.json array in the key-value store.", "default": false }, "csvDelimiter": { "title": "CSV delimiter", "enum": [ ",", ";", "tab", "|" ], "type": "string", "description": "Column separator for OUTPUT.csv. Use semicolon for Excel in locales that use a decimal comma.", "default": "," }, "csvBom": { "title": "Excel-friendly CSV (UTF-8 BOM)", "type": "boolean", "description": "Start OUTPUT.csv with a UTF-8 byte-order mark so Excel shows accents, emoji and non-Latin text correctly when the file is double-clicked.", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 }, "PROXY_UNBLOCKER_UNITS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 }, "PROXY_UNBLOCKER_UNITS": { "type": "integer", "example": 0 } } } } } } } } }}OpenAPI is a standard for designing and describing RESTful APIs, allowing developers to define API structure, endpoints, and data formats in a machine-readable way. It simplifies API development, integration, and documentation.
OpenAPI is effective when used with AI agents and GPTs by standardizing how these systems interact with various APIs, for reliable integrations and efficient communication.
By defining machine-readable API specifications, OpenAPI allows AI models like GPTs to understand and use varied data sources, improving accuracy. This accelerates development, reduces errors, and provides context-aware responses, making OpenAPI a core component for AI applications.
You can download the OpenAPI definitions for Dataset Transformer - Filter, Dedupe, Merge & Export CSV from the options below:
If you’d like to learn more about how OpenAPI powers GPTs, read our blog post.
You can also check out our other API clients: