CSV and JSON Integrity Parser

Introduction

CSV and JSON Integrity Parser performs robust CSV and JSON parsing with integrity hashes and manifests for data ingestion pipelines. It processes remote HTTP/HTTPS exports or inline text snippets, validating header consistency, column counts, and JSON structure. Each parsed record receives a deterministic SHA-256 hash, and each document produces a manifest hash for complete data lineage and tamper verification.

Use cases

• Ingestion pipeline validation: Check exported CSV headers, row widths, and JSON objects before loading into operational databases or data warehouses.
• Data integrity verification: Generate SHA-256 integrity hashes and manifests for audit trails and ETL verification.
• Batch normalization: Normalize varying CSV rows and JSON structures into uniform records with consistent field ordering and contact scrubbing.

Tutorial

Step-by-step guide

1. Prepare a JSON input with file_urls (direct HTTP/HTTPS links) or inline_documents (objects with name, format, and content).
2. Run the Actor on the Apify platform or invoke it through the Apify API.
3. The Actor validates row integrity, parses each record, generates SHA-256 hashes, and scrubs sensitive contact patterns.
4. Export the normalized dataset rows in JSON, CSV, or Excel format.

Pricing

The Actor is billed per parsed document under the Pay-Per-Event model using the csv-json-records event.

Tier	USD per csv-json-records event	Max bill (180 documents)
FREE	$0.006	$1.08
BRONZE	$0.0054	$0.972
SILVER	$0.0048	$0.864
GOLD	$0.004	$0.72
PLATINUM	$0.004	$0.72
DIAMOND	$0.004	$0.72

For the largest allowed input of 180 documents (MAX_DOCUMENTS = 180), the maximum bill per run is capped at $1.08 on FREE, $0.972 on BRONZE, $0.864 on SILVER, and $0.72 on GOLD, PLATINUM, and DIAMOND tiers.

Charge contract

Each row is pushed first and charged right after a successful push; a refused charge stops work and that row stays delivered and uncharged; a row pushed but not charged because of a crash or error is not charged on restart either; a reached limit stops work after that row was charged; failed items are neither pushed nor charged. Invalid CSV headers, malformed JSON, or failed HTTP downloads produce no dataset row and are not charged. A document with zero records, such as a header-only CSV or JSON [], is not pushed and not charged, returning INVALID_INPUT with the fixed reason code no_records. Inputs with the same source_id, meaning exact duplicate URLs or identical inline documents, are removed before fetching, keeping the first occurrence, so each is charged once. Two inline documents with the same name and format but different content have distinct source_id values and are both delivered, including across restarts. Rows already in the dataset of a restarted run are skipped and not charged again; if the dataset of a restarted run cannot be read, the run fails with DATASET_READ_FAILED before any new row or charge.

Integrity hash semantics

record_sha256 and manifest_sha256 cover the scrubbed, key-sorted canonical JSON records; digit strings that look like phone numbers are replaced during scrubbing. content_sha256 is the only hash of the raw bytes.

Operational limits

• Accepts up to 180 documents per execution (MAX_DOCUMENTS = 180).
• Maximum document size is 25,000,000 bytes, with a total run limit of 50,000,000 bytes.
• Up to 100,000 records per document; maximum 2,000,000 bytes per record. Each document becomes one dataset item, so its JSON output must also fit in 5,000,000 bytes. A document above that, which can happen well below 100,000 records, returns INVALID_INPUT with the fixed reason code output_too_large, is neither pushed nor charged, and later documents still run.
• Container nesting depth within any record cannot exceed 2 levels below the record root (dataset item depth limit is 6, so 3 or more container levels like {"a":{"b":{"c":[1]}}} fail with max_depth). A document with deeper nesting returns INVALID_INPUT with the fixed reason code max_depth, is neither pushed nor charged, and later documents still run.
• Socket timeout is 20 seconds, with at most 5 redirects followed.
• Total container execution budget is 3,600 seconds (RUN_TIMEOUT_SECONDS). If remaining runtime falls below 20 seconds, the run halts gracefully, delivering and charging all completed documents up to that point.

Input and output examples

Input example

{
  "inline_documents": [
    {
      "name": "sample.csv",
      "format": "csv",
      "content": "name\nAda\n"
    }
  ]
}

Output example

{
  "content_sha256": "b97998758a2ee1f0237f7924feb7558fff4749725019a5974f26b109d1bf36c3",
  "format": "csv",
  "integrity": {
    "manifest_sha256": "b5a53d2c7fd5a87bc0012ccc4db227f74529e0322380d50e41a5d7ac64bd7728",
    "record_hashes": [
      "88bab6d8f6dc68a877064d584cbb5b6c50e74f617ea50d81d3a53c2ee6ffbc4f"
    ]
  },
  "records": [
    {
      "index": 1,
      "record": {
        "name": "Ada"
      },
      "record_sha256": "88bab6d8f6dc68a877064d584cbb5b6c50e74f617ea50d81d3a53c2ee6ffbc4f"
    }
  ],
  "source_id": "94295f9d7e0be26e51be9df3",
  "source_kind": "inline_document",
  "status": "SUCCESS",
  "url": ""
}

Output fields

Field	Type	Description
content_sha256	String	SHA-256 hash of the complete raw document bytes
format	String	Parsed file format (csv or json)
integrity.manifest_sha256	String	SHA-256 of the compact, key-sorted JSON array of the record hashes, UTF-8 encoded
integrity.record_hashes	Array	Array of individual record SHA-256 hashes
records	Array	Normalized record items containing index, parsed record object, and hash
source_id	String	Unique deterministic identifier derived from the source
source_kind	String	Input source indicator (file_url or inline_document)
status	String	Processing status code (SUCCESS)
url	String	Source URL if fetched remotely, or empty string for inline input

Actor recommendations

For data extraction and conversion workflows, pair this Actor with:

• Sitemap URL Extractor: Extract sitemaps and export URL listings for CSV/JSON verification.
• Inline HTML and Markdown to Clean Markdown: Clean documentation or text content extracted alongside tabular data.
• PDF Text and Page Integrity Extractor: Extract page-by-page text from documents and validate resulting record hashes.

FAQ and support

Are invalid or malformed files charged?

No. Documents that fail CSV parsing, contain mismatched columns, exceed the output size limit, or fail network downloads produce no output row and are not charged.

Which character encodings are supported?

UTF-8 is required. UTF-8 byte-order marks (BOM) are recognized and stripped automatically. Other encodings are not supported.

Does the Actor fetch private or localhost addresses?

No. Private, loopback, link-local, and reserved IP ranges are blocked for safety.

Support and inquiries

For questions, bug reports, or feature requests, contact support via the Apify Console.