jq Helper – Transform, Filter & Reshape JSON with jq

Run any jq program over your JSON and get clean, structured results back. Reshape, filter, flatten, enrich, and aggregate JSON — from a pasted snippet or straight from another Apify Actor's dataset. The fast, no-code way to add a JSON transformation step to any scraping or data pipeline.

Map · filter · select · flatten · rename keys · merge fields · group & aggregate — all with battle-tested jq 1.8 syntax.

---

What does jq Helper do?

jq Helper is a tiny, blazing-fast Apify Actor that puts the full power of jq — the most popular command-line JSON processor — into a hosted, chainable step. Instead of writing a throwaway script every time a scraper returns messy JSON, drop in a jq filter and let this Actor do the transformation in the cloud.

It's built to slot directly into the Apify ecosystem: point it at the output dataset of any scraper and use it as a JSON post-processing, cleanup, and enrichment stage in your automation.

What can it do?

• Full jq 1.8 support — the complete language: select, map, group_by, reduce, sort_by, unique, string ops, math, conditionals, and more.
• Two input sources — paste inline JSON / JSONL, or read items from a linked Apify dataset by ID (perfect for chaining Actors).
• Two apply modes — transform each item independently (mapping/filtering/enrichment) or process the whole array at once (aggregations, grouping, dedupe).
• Pipeline-ready — writes results to its own dataset so the next Actor or integration (Zapier, Make, Google Sheets, webhooks…) can pick them up.
• Robust by default — errors on a single bad record are skipped and logged (or fail-fast if you prefer), and results are pushed in efficient batches.
• Zero setup — no servers, no jq install, no dependencies to manage.

What do people use jq Helper for?

• Clean up scraped data — drop nulls, rename fields, normalize values, flatten nested objects.
• Filter datasets — keep only the records that match your criteria (select(.price < 100)).
• Reshape for export — turn a verbose API response into a tidy, flat table for CSV/Excel/Sheets.
• Enrich records — derive new fields (domains from emails, full location strings, computed flags).
• Aggregate & summarize — group by a field, count, average, sum across an entire dataset.
• Glue Actors together — sit between a scraper and your destination as a no-code transformation step.

How do I transform JSON with jq Helper?

1. Choose your data source: paste JSON into jsonInput, or set datasetId to an existing dataset (e.g. an upstream scraper's run).
2. Write your transformation in the filter field using jq syntax.
3. Pick an applyMode (perItem or wholeInput).
4. Run it — transformed records land in the Actor's default dataset, ready to export or pass downstream.

Per-item vs. whole-input (read this first)

This is the one thing to get right:

Mode	What the filter receives	Reach a field with
perItem (default)	each array element, one at a time, as a single object	.websiteUrl
wholeInput	the entire array as one value	.[].websiteUrl

In per-item mode the array is iterated for you, so don't prefix with .[] — .[].websiteUrl will fail with "Cannot index string with string". Use a leading .[] only in whole-input mode.

What inputs does it take?

Field	Type	Description
filter	string (required)	The jq program. jq 1.8 syntax. May emit 0, 1, or many values per input — each becomes one output item.
applyMode	perItem | wholeInput	perItem (default): run on each element. wholeInput: feed the whole array as a single value.
datasetId	string	Apify dataset to read items from (provide either this or jsonInput, not both). Map this field to chain Actors in an integration. Paid: $0.01 per successful run. Your API token must have access to the dataset (owned by you, or explicitly shared) — otherwise the run fails with an insufficient permissions error.
jsonInput	string	Raw JSON (object/array/scalar) or JSONL (one value per line). Free — for prototyping.
wrapKey	string	Non-object outputs are wrapped as { <wrapKey>: value } (dataset items must be objects). Default value.
failOnError	boolean	true aborts on any jq runtime error; default false skips + logs the bad record.

What does the output look like?

Each value emitted by your filter becomes one item in the Actor's default dataset. Object outputs are stored as-is (their keys become columns); scalars and arrays are wrapped under wrapKey. Export the dataset as JSON, CSV, Excel, HTML, or RSS, or hand it to the next step in your workflow.

What do common jq filters look like?

Reshape & build a location string (per-item):

{fullName, currentEmployer, location: ([.city, .state, .country] | map(select(.)) | join(", "))}

Keep only verified records (per-item):

select(.clearVerified == true)

Pull one field as a clean column (per-item):

{websiteUrl}

Group and summarize the whole dataset (whole-input):

group_by(.state) | map({state: .[0].state, count: length}) | .[]

Deduplicate by a key (whole-input):

unique_by(.profileUrl) | .[]

Which jq function do I need?

Pick the task, copy the function. Everything here is standard jq and runs as-is.

I want to…	jq	Mode
Keep only some keys	{name, price, url}	per item
Drop keys	del(.rawHtml, .debug)	per item
Rename a key	{title: .name} + del(.name)	per item
Keep matching records	select(.price < 100)	per item
Drop empty/null fields	with_entries(select(.value != null and .value != ""))	per item
Derive a new field	. + {domain: (.email | split("@")[1])}	per item
Flatten a nested object	. + .address | del(.address)	per item
Join an array into a string	. + {tags: (.tags | join(", "))}	per item
Explode an array into rows	.variants[] + {parentId: .id}	per item
Numbers from strings	.price |= (gsub("[^0-9.]";"") | tonumber)	per item
Group and count	group_by(.state) | map({state: .[0].state, count: length}) | .[]	whole input
Sum or average	[{total: (map(.price) | add), avg: (map(.price) | add / length)}] | .[]	whole input
Deduplicate	unique_by(.url) | .[]	whole input
Sort	sort_by(-.price) | .[]	whole input
Top N	sort_by(-.rating) | .[:10] | .[]	whole input
Count records	[{count: length}] | .[]	whole input

Aggregations end in .[] because each emitted value becomes one dataset item — without it you get a single item holding the whole array.

How do I chain it after another Actor?

Run jq Helper automatically whenever an upstream scraper finishes, and pass it that run's dataset.

In the Console (no code):

1. Open the upstream Actor's run config → Integrations tab → Connect Actor or Task.
2. Choose jq Helper to run on success (event ACTOR.RUN.SUCCEEDED).
3. In jq Helper's input, set the datasetId field to the upstream run's dataset using the variable:

   {{resource.defaultDatasetId}}
   

   Leave jsonInput empty (the two sources are mutually exclusive), then set your filter and applyMode.

The transformed items land in jq Helper's own dataset — ready for the next Actor, a webhook, or an export to Sheets/Make/Zapier.

Other variables from the same run you can drop into string fields: {{resource.id}} (run ID), {{resource.actId}}, {{resource.defaultKeyValueStoreId}}, {{resource.status}}.

In code (parent orchestrator):

run = await Actor.call(actor_id='you/scraper', run_input={...})
await Actor.call(
    actor_id='rl1987/jq-helper',
    run_input={
        'filter': '.websiteUrl',
        'applyMode': 'perItem',
        'datasetId': run.default_dataset_id,
    },
)

Chaining always uses the datasetId path, so each chained run is a paid $0.01 conversion. Inline jsonInput stays free for prototyping.

FAQ

Do I need to install jq, and which version is it? Nothing to install — the engine is bundled with the Actor and runs in the cloud. It is jq 1.8, the full language, including 1.8 additions like abs and pick.

What does it cost? Runs on inline jsonInput are free, so you can prototype a filter at no cost. A run that reads a dataset (datasetId) is a flat $0.01, whatever the dataset's size — one cent to transform ten rows or a million.

Does it handle JSONL? Yes. jsonInput is parsed as a single JSON document first; if that fails it is parsed as JSONL, one JSON value per non-empty line.

My filter returns a string/number — why is it {"value": ...}? Dataset items must be JSON objects, so scalars and arrays are wrapped. Set wrapKey to rename the key, or emit an object (e.g. {websiteUrl}) for named columns.

How do I aggregate across all records? Use applyMode: wholeInput and reach elements with .[]. Per-item mode never sees more than one record at a time, so group_by, unique_by, add and sort_by need whole-input mode.

What if one record is malformed? By default it's skipped, logged as a warning, and counted in the run summary, so one bad row never loses the rest. Set failOnError: true to abort the run instead.

Why does my run fail with an insufficient-permissions error? The API token running this Actor must be able to read the dataset you passed as datasetId — your own dataset, or one explicitly shared with you. A dataset ID alone is not access.

Can I schedule it or trigger it automatically? Yes. Use Apify Schedules for a recurring run, or the upstream Actor's Integrations tab to run it on every successful scrape — see the chaining section above.

Can I use it from n8n?

Use the n8n-nodes-jq-helper community node (source) to run this Actor directly from an n8n workflow.

How do I run it locally?

pip install -r requirements.txt
apify run   # reads storage/key_value_stores/default/INPUT.json

---

Built with the Apify SDK for Python and the jq bindings (bundles libjq — no system jq required).

Disclaimer: This is an independent, unofficial project. It is not affiliated with, endorsed by, or otherwise associated with the jq project or any of its developers. "jq" is used here only to describe the JSON-processing language this Actor runs; all rights to jq belong to its respective authors.

Data pipeline toolkit

Part of the Data pipeline toolkit — small, chainable Actors for cleaning, transforming, and generating data inside a larger pipeline:

• DuckDB Helper – SQL over CSV, JSON, Parquet, Excel, SQLite — Run a DuckDB SQL query over remote/local files, push results to a dataset.
• Regex Helper — Apply named regular expressions to strings, extract structured matches.
• URL Wrangler — Join, decompose, and rewrite URLs and query params in batch.
• ZIP Code Helper — Resolves US ZIP codes into city, state, county, and more.
• Postal Address Normaliser — Parses and normalises postal addresses using libpostal.
• Phone Number Wrangler — Validate, format, and parse phone numbers using libphonenumber.
• UUID Generator — Generate bulk UUIDs (v1, v3, v4, v5, v7) on demand.
• Secure Password & Passphrase Generator — Generate secure passwords and diceware passphrases per NIST guidance.
• Thumbnail Maker — Generates thumbnails from image URLs using ImageMagick.
• Katana Web Crawler (ProjectDiscovery) — Crawl websites with Katana, stream results as JSONL.
• ProjectDiscovery Notify — Stream records to Slack, Discord, Telegram, Email, and more.

Did you find this useful?

Rate this actor on Apify! Your feedback helps other users find it and helps us keep improving it.