JSON Dataset Merge & Deduplicator

Merge inline JSON datasets by explicit composite keys. Keep the first or last duplicate, preserve missing-key records and value types, and export full records with provenance diagnostics.

When to use it

Combine exports from several sources while keeping key handling explicit and preserving records that have no usable key.

Price

$0.03 per completed report, for supported paying Apify users. One report-generated event, after files and a summary are stored. No per-row, per-URL or dataset-item charge. Invalid input and processor timeouts do not trigger the report event. Apify’s free-user rules and any separately applicable platform charges are governed by Apify.

Use a maximum charge limit that covers at least one report. Keep a smaller diagnostic detailLimit for large inputs; all output records and counts remain complete. No external AI models or API keys are needed.

Example input

{
  "recordsJson": "[[{\"id\":\"001\",\"name\":\"old\"},{\"name\":\"no key\"}],[{\"id\":\"001\",\"name\":\"new\"},{\"id\":\"002\",\"name\":\"second\"}]]",
  "keyColumns": [
    "id"
  ],
  "keep": "last",
  "detailLimit": 100
}

Results

• REPORT.json: complete summary, full records and bounded details.
• RECORDS.csv: all output records. Formula-like cells are escaped for spreadsheets; JSON is the authoritative typed result.
• REPORT.html: self-contained escaped report, with at most 20 output records in the preview.

The default dataset contains a single operational summary; it is not the records export. A low detail limit truncates diagnostics only. Download JSON or CSV for full output.

Limits and behavior

Combined input envelope: 5 MiB; 10,000 records total; 100 unique columns; 32,768 UTF-8 bytes per cell; 256 UTF-8 bytes per field name. Records must be flat objects with finite JSON scalar values. Duplicate object fields are rejected. Key comparison distinguishes true, 1, 1.0 and "1". No fuzzy matching or coercion.

All products have a 25-second processor deadline, 256 MiB container memory, a 60-second run timeout and a combined 10 MiB export limit. Large or wide inputs may hit the export limit before the row limit. Failed exports do not charge the report event. No automatic restarts are enabled.

Privacy and operation

Use only data you are authorized to process. Inline data and outputs are stored in your run’s Apify storage under its configured access and retention settings. This Actor does not fetch external resources or invoke a model, and does not log source values. Review your own storage sharing and retention settings for sensitive inputs.

Billing uses a durable checkpoint. If a charge response is uncertain, the same run stops for ledger review instead of retrying payment automatically. Start a new run only after inspecting the earlier attempt. Report a problem through Apify’s Actor issue tracker with sanitized examples; do not post private records or tokens.