Dataset Transformer – Filter, Dedupe, Flatten & JSON to CSV

Dataset Transformer cleans and reshapes tabular data in one step: filter rows, remove duplicates, flatten nested JSON, pick or rename fields, sort, and export CSV. Feed it an Apify dataset ID (for example the output of another scraper), a pasted JSON array, or a public JSON / JSONL / CSV file URL.

Store page: apify.com/chris_elder/dataset-transformer

✅ Pay per result: $0.50 per 1,000 output rows · 🆓 Dropped / filtered rows are never charged · 🔑 No API key

▶ Try it with one click · 📄 Real example output · 💲 Pricing · 🔌 API & integrations · ❓ FAQ · 🛟 Support

What does Dataset Transformer do?

• 🧹 Filter rows with 14 operators (eq, ne, gt, gte, lt, lte, contains, notContains, startsWith, endsWith, exists, notExists, in, regex) combined with AND or OR
• 🔁 Remove duplicates by one or more fields (dot paths like meta.city); the first occurrence is kept
• 📦 Flatten nested JSON into dotted keys ({ "meta": { "city": "London" } } → meta.city)
• ✏️ Pick, omit or rename fields
• ↕️ Sort ascending or descending by any field
• 📄 JSON to CSV – optional OUTPUT.csv in the Key-Value store; CSV files you load come back as JSON dataset rows (CSV to JSON)
• 🆓 Dropped rows free – only rows written to the output dataset are charged

Use cases

• Scraper users – clean and dedupe a noisy dataset before it goes to Google Sheets, Airtable or a warehouse
• Agents & n8n/Make – “filter + dedupe this run’s dataset” as one step after another Actor
• Developers – flatten nested JSON and convert JSON to CSV without writing a script
• Ops / QA – keep only rows that pass rules (e.g. price exists, rating ≥ 4) after a crawl

How to use it

1. Click Try for free – the prefill filters score ≥ 5, removes the duplicate Ada/London row, flattens and renames meta.city → city, sorts by score and exports CSV (2 output rows ≈ $0.0015 including the start fee).
2. To clean real data, replace items with datasetId (any dataset in your account, or a public one) or fileUrl (JSON array, JSONL or CSV).
3. Download the cleaned dataset (JSON, CSV, Excel from the Storage tab) or OUTPUT.csv from the run’s Key-Value store.

Input example

{
  "items": [
    { "id": 1, "name": "Ada", "meta": { "city": "London" }, "score": 9 },
    { "id": 2, "name": "Ada", "meta": { "city": "London" }, "score": 9 },
    { "id": 3, "name": "Bob", "meta": { "city": "Berlin" }, "score": 4 },
    { "id": 4, "name": "Cara", "meta": { "city": "Paris" }, "score": 7 }
  ],
  "filters": [{ "field": "score", "op": "gte", "value": 5 }],
  "dedupeBy": ["name", "meta.city"],
  "flatten": true,
  "rename": { "meta.city": "city" },
  "sortBy": "score",
  "sortDir": "desc",
  "exportCsv": true,
  "maxRows": 10
}

Other options: filterMode (and/or), pick, omit, csvKey.

Example: 4 rows in → 2 clean rows out

Step	Rows	What happened
Input	4	Ada (9), Ada (9, duplicate), Bob (4), Cara (7)
Filter score ≥ 5	3	Bob dropped (free)
Dedupe by name + meta.city	2	second Ada/London dropped (free)
Flatten + rename + sort desc	2	meta.city → city, highest score first

Output example (from a real public run)

Public run data (no login needed): dataset JSON · dataset CSV · OUTPUT.csv · RUN_SUMMARY

Dataset rows:

[
  { "id": 1, "name": "Ada", "score": 9, "city": "London" },
  { "id": 4, "name": "Cara", "score": 7, "city": "Paris" }
]

OUTPUT.csv:

id,name,score,city
1,Ada,9,London
4,Cara,7,Paris

RUN_SUMMARY:

{ "inputCount": 4, "outputCount": 2, "droppedFilter": 1, "droppedDedupe": 1, "skippedBudget": 0, "budgetHit": false, "exportCsv": true, "csvKey": "OUTPUT.csv", "source": "items" }

How much does it cost to clean a dataset?

Event	Free / Bronze	Silver	Gold+
Actor start (× memory GB, min 1)	$0.0005	$0.0005	$0.0005
Row output	$0.0005	$0.0004	$0.0003

Filtered, deduped and invalid rows are free. Only rows written to the output dataset are charged.

Job	Approx. cost
Prefill (2 output rows, 512 MB)	2 × $0.0005 + $0.0005 start = $0.0015
1,000 output rows	$0.50 + start
10,000 output rows (Gold)	10,000 × $0.0003 = $3.00 + start

Use Max cost per run to cap spend; the Actor stops writing rows when the cap is reached (budgetHit: true in RUN_SUMMARY).

Integrations (API, AI agents, Make, Zapier)

Clean another Actor’s output: pass that run’s defaultDatasetId as datasetId.

curl -X POST "https://api.apify.com/v2/acts/chris_elder~dataset-transformer/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"datasetId":"YOUR_DATASET_ID","dedupeBy":["url"],"flatten":true,"exportCsv":true}'

Python (pip install apify-client):

from apify_client import ApifyClient

client = ApifyClient("YOUR_TOKEN")
run = client.actor("chris_elder/dataset-transformer").call(run_input={
    "datasetId": "YOUR_DATASET_ID",
    "filters": [{"field": "price", "op": "exists"}],
    "dedupeBy": ["url"],
    "exportCsv": True,
})
rows = client.dataset(run["defaultDatasetId"]).list_items().items

MCP (AI agents): https://mcp.apify.com/?tools=chris_elder/dataset-transformer

Make / Zapier / n8n: add the Apify Run Actor step after your scraper, map its dataset ID into datasetId, and read the cleaned rows. Works with Apify schedules and webhooks.

Limits and good to know

• One source per run (items, datasetId or fileUrl). It does not merge several datasets.
• maxRows default 10,000 (max 100,000). For very large datasets, filter first or raise memory.
• CSV input is parsed with a header row and values arrive as text; gt/gte/lt/lte convert values to numbers, eq compares loosely ("9" equals 9).
• Excel (XLSX) files are not supported as input yet.

FAQ

How do I convert JSON to CSV? Load the JSON (pasted, dataset or file URL), turn on flatten for nested fields and exportCsv, then download OUTPUT.csv.

How do I remove duplicates from an Apify dataset? Set datasetId and list the fields that identify a duplicate in dedupeBy (e.g. ["url"]).

Does it modify my source dataset? No. It only reads, and writes a new dataset on this run.

Nested fields? Use dot paths (meta.city) and/or turn on flatten.

Why was I charged less than the input size? Filtered and deduped rows are free; you pay only for output rows.

Support, reliability and feature requests

• Dropped rows are free.
• Every release is tested on real inputs; the one-click example works.
• Issues answered within 2 business days via the Actor Issues tab.
• Breaking input or price changes are announced in this README first.

Built by Chris Elder with AI-assisted development; every release is reviewed and tested.

Need the data done for you?

If you'd rather not set up runs yourself, the UK company behind these tools (GPT Shop Ltd) offers a fixed-price done-for-you scrape. Tell us the website and the fields you need, and you get a clean CSV or Google Sheet within 48 hours of the confirmed brief, up to 5,000 rows, for £149 or $179. If we can't get the data, you're refunded in full. See the done-for-you scraper service

More tools by Chris Elder

Bulk File Downloader · HTTP Request Sender · Sitemap URL Extractor · PDF to Markdown · all tools

More use cases (FAQ)

• How do I convert an Apify dataset or JSON to CSV/Excel? Set exportCsv: true.
• How do I remove duplicate rows from scraped data? Use dedupeBy with one or more fields (nested paths such as meta.city work).
• Can I flatten nested JSON for Google Sheets? Yes. Set flatten: true, then rename columns with rename.