DuckDB Helper – Run SQL over CSV, JSON, Parquet, Excel & SQLite

Turn any messy export into clean, structured data — no code, no database to set up. Give DuckDB Helper a link to your file(s) (or another Actor's scrape results) and a SQL query, and get back tidy rows, ready to filter, download, or pipe into your next automation.

---

What does DuckDB Helper do?

DuckDB Helper runs your SQL query over data files hosted anywhere on the web — CSV, JSON, Parquet, Excel, and more — using the DuckDB engine, without you needing to install DuckDB, write a script, or manage a database. Point it at one or more files, write a query in plain SQL, and it does the rest: downloads the files, loads them, runs the query, and hands you the result as dataset rows or a single downloadable file.

It's also built to slot straight into the Apify ecosystem as a post-processing step: connect it to any scraper Actor and it can clean up, filter, deduplicate, or reshape that Actor's output automatically every time a run finishes.

Why use DuckDB Helper?

• Clean up scraper output — dedupe rows, drop unwanted columns, filter out junk records, straight after a scrape finishes.
• Combine multiple sources — join a CSV price list against an Excel product catalog against a scraped dataset, all in one query.
• Convert formats — pull data out of Excel or SQLite and get back CSV, JSON, or Parquet (or the reverse).
• Aggregate & summarize — group, count, sum, and average across thousands of rows without opening a spreadsheet.
• Skip the database setup — no server, no connection strings, no install; just a query and a link to your data.

How to use DuckDB Helper

DuckDB Helper has two explicit modes, chosen with the Mode field — there's no guessing which run is billable:

• Free (the default): try out SQL syntax against literals, a URL read directly inside the query itself, or data you paste straight into Paste data. Input files and Dataset ID are disabled in this mode. Never charged.
• Paid: load Input files and/or a chained Dataset ID. $0.01 per successful run, regardless of data size.

1. Pick Mode. Leave it on free to experiment; switch to paid once you're ready to process real files or a dataset.
2. In free mode, either write literals or paste some CSV/JSON/TSV/NDJSON text into Paste data to test against real content, no cost, no URL needed. In paid mode, point it at your data: add file links under Input files (each gets a name you reference in your query), and/or set Dataset ID to pull in another Actor's scrape results.
3. Write your SQL query referencing those names, e.g. SELECT * FROM sales WHERE amount > 100.
4. Optionally cap the row count and choose whether you also want a single downloadable file (CSV, JSON, Parquet, or Excel).
5. Run it. Your results appear as dataset rows, a downloadable file, or both.

A quick example

Say you have a CSV of Titanic passengers hosted online and want everyone over 60:

• Mode: paid
• Input files: { "name": "titanic", "url": "https://.../titanic.csv" }
• SQL query: SELECT Name, Age FROM titanic WHERE Age > 60 ORDER BY Age

Run it, and the matching passengers land in your dataset — no download, no spreadsheet, no script.

Try it for free with pasted data

No URL, no charge — just paste some data straight into the input:

• Mode: free
• Paste data: name,age\nAlice,30\nBob,16
• SQL query: SELECT * FROM data WHERE age > 18

More examples

Every example below is a real, working input — copy it in and run it as-is.

Filter a CSV

• Input files: { "name": "titanic", "url": "https://raw.githubusercontent.com/datasciencedojo/datasets/master/titanic.csv" }
• SQL query: SELECT Name, Age FROM titanic WHERE Age > 60 ORDER BY Age

Query a JSON API response

• Input files: { "name": "users", "url": "https://jsonplaceholder.typicode.com/users", "format": "json" }
• SQL query: SELECT name, email, company.name AS company FROM users

Read an NDJSON/JSONL file

• Input files: { "name": "movies", "url": "https://raw.githubusercontent.com/duckdb/duckdb/main/data/json/example_rn.ndjson", "format": "ndjson" }
• SQL query: SELECT * FROM movies LIMIT 10

Aggregate a Parquet file

• Input files: { "name": "people", "url": "https://raw.githubusercontent.com/duckdb/duckdb/main/data/parquet-testing/userdata1.parquet", "format": "parquet" }
• SQL query: SELECT first_name, last_name, salary FROM people ORDER BY salary DESC LIMIT 10

Pull rows out of an Excel workbook

• Input files: { "name": "sales", "url": "https://go.microsoft.com/fwlink/?LinkID=521962", "format": "excel" }
• SQL query: SELECT Segment, Country, SUM(Profit) AS total_profit FROM sales GROUP BY Segment, Country ORDER BY total_profit DESC

Query a SQLite database

• Input files: { "name": "chinook", "url": "https://raw.githubusercontent.com/lerocha/chinook-database/master/ChinookDatabase/DataSources/Chinook_Sqlite.sqlite", "format": "sqlite" }
• SQL query: SELECT Name, Composer FROM chinook.Track LIMIT 10 — note the chinook. prefix: SQLite sources attach as a database, so tables are reached as <name>.<table>, not through a single view.

Read an Avro file

• Input files: { "name": "weather", "url": "https://raw.githubusercontent.com/apache/avro/main/share/test/data/weather.avro", "format": "avro" }
• SQL query: SELECT * FROM weather LIMIT 10

Chain after another Actor and export a clean file

• Dataset ID: {{resource.defaultDatasetId}} (set via Integrations, or a real dataset ID for a one-off test)
• SQL query: SELECT * FROM dataset WHERE Age > 70 ORDER BY Age
• Export result as: excel, with skipDatasetPush enabled — produces only a ready-to-download spreadsheet, no dataset rows.

Input

Field	What it's for
Mode	free (default) — literals, pasted data, or a URL read directly in SQL; never charged. paid — process Input files/Dataset ID; $0.01/run.
SQL query	The query to run. Reference your pasted data/input files/dataset by name, or point it straight at a URL in the query itself.
Paste data	Free-mode only. Paste CSV/TSV/JSON/NDJSON text directly — no URL needed — to test against real content at no cost.
Input files	Paid mode only. The file(s) to load — each just needs a name and a link; the format (CSV, JSON, Parquet, Excel, Avro, SQLite) is detected automatically.
Dataset ID	Paid mode only. ID of an Apify dataset to load and query — this is what you map when chaining DuckDB Helper after another Actor.
Max output rows	Optional safety cap on how many rows come back.
Also export result file	Get the full result as one downloadable CSV, JSON, Parquet, or Excel file, in addition to (or instead of) dataset rows.

Full field-level details, including advanced options like extra DuckDB extensions for Postgres/MySQL/Iceberg/Delta sources, are in the Input tab.

Output

By default, every matching row becomes one item in the dataset, with your query's columns as fields:

[
    { "Name": "Barkworth, Mr. Algernon Henry Wilson", "Age": 80 },
    { "Name": "Svensson, Mr. Johan", "Age": 74 }
]

Download the dataset as JSON, CSV, Excel, HTML, or RSS — or hand it straight to the next Actor in your workflow.

If you turned on Also export result file, you'll additionally get a direct download link (in the run log and in the run's Output) pointing straight to a single CSV/JSON/Parquet/Excel file — handy when you just want a file to save or share rather than dataset rows to page through.

Post-processing another Actor's results

The most common way to use DuckDB Helper: run it automatically right after a scraper finishes, to clean up or filter that Actor's output.

1. Open your scraper Actor's run → Integrations tab → Connect Actor or Task.
2. Choose DuckDB Helper, set it to run on success.
3. In DuckDB Helper's input, set Dataset ID to the upstream run's dataset using the variable {{resource.defaultDatasetId}}.
4. Write your SQL query against the dataset view, e.g. SELECT * FROM dataset WHERE price > 100.

The cleaned-up results land in DuckDB Helper's own dataset, ready for the next step — a webhook, a Google Sheet, or another Actor.

Pricing

Mode decides the price, explicitly — not what fields happen to be filled in. free mode (literals, pasted data, or a URL read directly inside the SQL) never charges, and disables Input files/Dataset ID. Switch to paid mode to load Input files and/or a chained Dataset ID; each successful run is then a flat $0.01, regardless of how much data it processes.

FAQ

Do I need to know DuckDB specifically? No — if you know standard SQL, you're covered. DuckDB's syntax is very close to PostgreSQL.

What file formats are supported? CSV/TSV, JSON, NDJSON/JSONL, Parquet, Excel (.xlsx/.xls), Avro, and SQLite out of the box. Postgres, MySQL, Iceberg, and Delta sources are supported for advanced users via the Input tab's extensions option.

My file didn't load — why? Most often the URL isn't reachable, or its format couldn't be guessed from the link. Set the format explicitly in the Input files list to skip auto-detection.

Can I query more than one file at once? Yes — add each as its own entry under Input files (and/or set a Dataset ID), then join or union them in a single SQL query.

A column in my output looks like garbled binary text (e.g. b'\x01\x00...') — this happens with non-JSON DuckDB types such as GEOMETRY or BLOB (e.g. from the spatial extension). Cast them to text before selecting, e.g. ST_AsText(geom_column).

---

Built with the Apify SDK for Python and DuckDB.

Disclaimer: This is an independent, unofficial project. It is not affiliated with, endorsed by, or otherwise associated with the DuckDB project or any of its developers. "DuckDB" is used here only to describe the SQL engine this Actor runs; all rights to DuckDB belong to its respective authors.