# JSON Transformer - jq & JMESPath for Datasets, APIs, Files (`gazidev/json-transformer`) Actor

Reshape any JSON with a jq or JMESPath expression: an Apify dataset, a JSON or JSON Lines URL, a key-value record or pasted JSON. Flatten, cast, rename, then save to a dataset or a CSV, XLSX, JSON or JSONL file. The glue step for n8n, Make and Apify integrations.

- **URL**: https://apify.com/gazidev/json-transformer.md
- **Developed by:** [Cemal Atakli](https://apify.com/gazidev) (community)
- **Security:** Limited permissions
- **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.20 / 1,000 output rows

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## JSON Transformer - jq & JMESPath for Apify Datasets, APIs and JSON Files

Reshape any JSON with a **jq** or **JMESPath** expression, without a server, a Lambda or a code node. Point it at an
**Apify dataset**, a **JSON / JSON Lines URL** (any REST API), a **key-value store record**, or **paste JSON**, write
one expression, and get clean rows in a dataset plus an optional **CSV, Excel (XLSX), JSON or JSONL** file with a
public download link.

It is the glue step between Actors and tools such as **n8n, Make, Zapier and Apify integrations**: pick the fields you
need, rename nested keys, filter, group and aggregate, explode arrays into rows, then hand the result to the next step.

### What it does

- **Real jq** (the official libjq C library through the `jq` Python binding): `select`, `map`, `group_by`, `reduce`, `to_entries`,
  `split`, `test` (regex), `@csv`, `$variables` and the rest of the language.
- **JMESPath** (the AWS CLI `--query` language) for simple projections and filters.
- **Two modes:** whole document (like `jq` on a file; a dataset becomes one array) or **per item** (like `jq -c` on
  JSON Lines; datasets are streamed page by page with low memory).
- **After the expression (optional):** flatten nested objects (`address.city`), convert types (`"$1,299"` -> `1299`),
  rename fields, keep only selected fields, remove nulls.
- **Output:** dataset rows and/or one file (CSV, XLSX, JSON, JSONL) in the key-value store, with its public URL in the
  `OUTPUT` record.
- **Errors never charged:** a bad expression fails fast; an item the expression fails on is skipped, counted and
  sampled in `OUTPUT.errorSamples`.

#### JSON Transformer or Dataset Filter & Transform?

| You want...                                                  | Use |
|--------------------------------------------------------------|-----|
| Expressions (jq / JMESPath), reshaping nested JSON, aggregates, JSON from APIs or KV records | **JSON Transformer** (this Actor) |
| No-code filter rules, dedupe, "only new since last run", CSV / XLSX input files | [Dataset Filter & Transform](https://apify.com/gazidev/dataset-filter-transform) |

### Use cases

- Turn a scraper's nested output into a flat table for Google Sheets or a CRM import.
- Call a JSON API and keep only what you need: `.data.items[] | {id, title, price: .pricing.amount}`.
- Aggregate a dataset: `group_by(.category) | map({category: .[0].category, count: length})`.
- Convert JSON Lines to CSV / Excel, or JSON to JSONL, in one step.
- Extract the useful part of another Actor's `OUTPUT` record for the next Actor in a pipeline.
- Give AI agents a safe, deterministic JSON reshaping tool.

### Input example

```json
{
  "jsonUrl": "https://jsonplaceholder.typicode.com/users",
  "engine": "jq",
  "expression": ".[] | {id, name, email, city: .address.city, lat: .address.geo.lat, company: .company.name, domain: (.email | split(\"@\")[1])}",
  "castMap": { "lat": "number" },
  "outputFormat": "csv"
}
```

Per-item jq with a variable on a dataset (e.g. from a webhook integration):

```json
{
  "datasetId": "{{resource.defaultDatasetId}}",
  "perItem": true,
  "engine": "jq",
  "jqVariables": { "minRating": 4.5 },
  "expression": "select(.totalScore >= $minRating) | {title, phone, website, rating: .totalScore}"
}
```

JMESPath:

```json
{ "inlineJson": "{\"orders\":[{\"id\":1,\"country\":\"PT\"},{\"id\":2,\"country\":\"US\"}]}",
  "engine": "jmespath", "expression": "orders[?country != 'US']" }
```

Use exactly one source: `jsonUrl`, `datasetId`, `kvStoreId` + `kvRecordKey`, or `inlineJson`.

### Output example

```json
[
  { "id": 1, "name": "Leanne Graham", "email": "Sincere@april.biz", "city": "Gwenborough", "lat": -37.3159, "company": "Romaguera-Crona", "domain": "april.biz" },
  { "id": 2, "name": "Ervin Howell", "email": "Shanna@melissa.tv", "city": "Wisokyburgh", "lat": -43.9509, "company": "Deckow-Crist", "domain": "melissa.tv" }
]
```

How results become rows: each object the expression emits is one row; an emitted array is split into one row per
element (turn off "One row per array element" to keep it); a number or string `x` becomes `{"value": x}`; `null` is
dropped. The `OUTPUT` record holds counts, error samples, the file link and the charges.

### Pricing

Pay per event, no subscription:

| Event | Price |
|---|---|
| Output row | **$0.0002** ($0.20 per 1,000 rows) |
| Output file saved (CSV / XLSX / JSON / JSONL) | $0.002 |
| Actor start | $0.00005 |

| Actor | Price |
|---|---|
| **JSON Transformer (this)** | **$0.0002 per row, $0.002 per file** |
| humble-echidna/dataset-transform | $0.001 per row |
| rl1987/jq-helper | $0.01 per conversion |

Set a maximum cost per run: the Actor stops cleanly at that limit, keeps the delivered rows and still saves the file.

### FAQ

**Can it read any dataset ID?** It reads datasets in your own account, datasets shared with you, and public
datasets. Another user's private dataset is not accessible (the Actor runs with limited permissions and only gets
access to storages you select or own). Use the dataset picker or `{{resource.defaultDatasetId}}` in integrations.

**Which jq?** The official libjq bundled in the `jq` Python package (1.13). All standard filters work. Reading
other inputs (`input`, `inputs`), modules (`import`) and `env` / `$ENV` are disabled; pass values with `jqVariables`.

**How big can the input be?** Per-item mode streams datasets, so millions of items work. Whole-document mode loads
everything into memory: raise the run memory for very large inputs, or use `maxInputItems` for a preview.
URL downloads are capped by `maxFileSizeMb` (default 200 MB).

**My expression returns nothing.** In whole-document mode the input is the full array, so start with `.[]`, or
switch on "Apply to each item separately".

**Private APIs?** Add `httpHeaders` (e.g. an `Authorization` header) for your own endpoints. Only fetch data you are
allowed to use.

### Use with AI agents / Apify MCP

The Actor is available as a tool in the [Apify MCP server](https://mcp.apify.com) (`gazidev/json-transformer`).
An agent can paste JSON into `inlineJson` (or pass a dataset ID from a previous tool call) with a jq expression and
get deterministic, typed rows back, instead of reshaping data inside the language model.

# Actor input Schema

## `jsonUrl` (type: `string`):

Public URL of a JSON document, a REST API endpoint or a JSON Lines (NDJSON) file. GitHub 'blob', Dropbox and Google Drive file links are turned into direct downloads. Use exactly one source.

## `datasetId` (type: `string`):

An Apify dataset ID, e.g. the output of a scraper run. In an integration or webhook use {{resource.defaultDatasetId}}. The Actor can read your own datasets and public ones; another user's private dataset is not accessible.

## `kvStoreId` (type: `string`):

Key-value store that holds a JSON record (e.g. the OUTPUT of another Actor run). Use together with 'Record key'.

## `kvRecordKey` (type: `string`):

Key of the JSON record in the key-value store above, e.g. OUTPUT.

## `inlineJson` (type: `string`):

Paste a JSON document or JSON Lines directly (handy for quick tests, n8n / Make and AI agents).

## `engine` (type: `string`):

jq (full jq language via libjq: select, map, group_by, reduce, string functions, $variables) or JMESPath (the AWS CLI query language).

## `expression` (type: `string`):

jq example: '.\[] | select(.age > 30) | {name, city: .address.city}'. JMESPath example: "\[?age > `30`].{name: name, city: address.city}". Empty = pass the input through ('.' / '@').

## `perItem` (type: `boolean`):

Off (default): the expression gets the whole document (for a dataset: the array of all items), like 'jq' on a file. On: the expression runs once per item (array element / dataset item / JSON line), like 'jq -c' on JSON Lines; datasets are then streamed page by page with low memory. Items for which the expression returns nothing (e.g. select() fails) are dropped.

## `jqVariables` (type: `object`):

Named values available as $name in a jq expression, e.g. {"minPrice": 10} -> 'select(.price >= $minPrice)'.

## `onItemError` (type: `string`):

Skip the item (errors are counted and sampled in OUTPUT, never charged) or fail the run.

## `explodeArrays` (type: `boolean`):

When the expression returns an array (e.g. 'map(...)' or a JMESPath projection), output each element as its own row. Off = keep the array in one row under the value field.

## `valueField` (type: `string`):

Dataset rows must be objects, so a number / string / array result x becomes {"value": x}.

## `flatten` (type: `boolean`):

Turn {"address": {"city": "Paris"}} into {"address.city": "Paris"} after the expression.

## `flattenSeparator` (type: `string`):

Separator between nested key names.

## `flattenArrays` (type: `string`):

Keep arrays as they are, join simple arrays into 'a, b, c', or expand to indexed columns (tags.0, tags.1).

## `castMap` (type: `object`):

Field -> string | number | integer | boolean | json | array, e.g. {"lat": "number"}. Text like '$1,299' or '15%' becomes a number. Uses names after flattening, before renaming.

## `castFailure` (type: `string`):

Set it to null, keep the original value, or drop the row.

## `renameMap` (type: `object`):

Old name -> new name, e.g. {"company": "companyName"}. Applied after casting.

## `selectFields` (type: `array`):

Output only these fields, in this order (names after renaming). Empty = keep all.

## `removeNullFields` (type: `boolean`):

Drop fields whose value is null.

## `outputToDataset` (type: `boolean`):

Write the result rows to this run's dataset (download as JSON, CSV, Excel or use in integrations).

## `outputFormat` (type: `string`):

Save all result rows as one file in the key-value store; its public link is in the OUTPUT record. CSV and XLSX write nested values as JSON text (turn on 'Flatten' for columns).

## `fileName` (type: `string`):

Base name of the file (the extension is added).

## `maxItems` (type: `integer`):

Stop after this many output rows (0 = no limit).

## `maxInputItems` (type: `integer`):

Read at most this many dataset items / array elements (0 = all). Useful for previews of big datasets.

## `httpHeaders` (type: `object`):

Optional request headers for the JSON URL, e.g. {"Authorization": "Bearer ..."} for your own API. Stored in the run input; use a secret-free token where possible.

## `maxFileSizeMb` (type: `integer`):

Larger URL downloads are refused (nothing is charged).

## Actor input object example

```json
{
  "jsonUrl": "https://jsonplaceholder.typicode.com/users",
  "engine": "jq",
  "expression": ".[] | {id, name, email, city: .address.city, lat: .address.geo.lat, company: .company.name, domain: (.email | split(\"@\")[1])}",
  "perItem": false,
  "onItemError": "skip",
  "explodeArrays": true,
  "valueField": "value",
  "flatten": false,
  "flattenSeparator": ".",
  "flattenArrays": "keep",
  "castMap": {
    "lat": "number"
  },
  "castFailure": "null",
  "removeNullFields": false,
  "outputToDataset": true,
  "outputFormat": "none",
  "fileName": "transformed",
  "maxItems": 0,
  "maxInputItems": 0,
  "maxFileSizeMb": 200
}
```

# Actor output Schema

## `rows` (type: `string`):

No description

## `files` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "jsonUrl": "https://jsonplaceholder.typicode.com/users",
    "expression": ".[] | {id, name, email, city: .address.city, lat: .address.geo.lat, company: .company.name, domain: (.email | split(\"@\")[1])}",
    "castMap": {
        "lat": "number"
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("gazidev/json-transformer").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "jsonUrl": "https://jsonplaceholder.typicode.com/users",
    "expression": ".[] | {id, name, email, city: .address.city, lat: .address.geo.lat, company: .company.name, domain: (.email | split(\"@\")[1])}",
    "castMap": { "lat": "number" },
}

# Run the Actor and wait for it to finish
run = client.actor("gazidev/json-transformer").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "jsonUrl": "https://jsonplaceholder.typicode.com/users",
  "expression": ".[] | {id, name, email, city: .address.city, lat: .address.geo.lat, company: .company.name, domain: (.email | split(\\"@\\")[1])}",
  "castMap": {
    "lat": "number"
  }
}' |
apify call gazidev/json-transformer --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,gazidev/json-transformer"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/9doY5s94VrLd3r5WX/builds/WKOYMTYsyeTFaNo4d/openapi.json