# CSV Migration Verifier (`lucaslucas/csv-migration-verifier`) Actor

Prototype comparing two inline CSV exports using explicit mappings, keys and decimal tolerances. Produces JSON evidence, HTML report and CSV exceptions. Synthetic testing only.

- **URL**: https://apify.com/lucaslucas/csv-migration-verifier.md
- **Developed by:** [LucasLucas](https://apify.com/lucaslucas) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## CSV migration verifier — prototype

A small offline evidence generator for comparing two CSV exports across renamed columns. The offline CLI uses Python 3.10+ with **no runtime dependencies, network requests, accounts, telemetry, database, or installation required**. An optional, separately invoked Apify SDK adapter is also included; it is the only network-capable path. All included records are synthetic.

This is a prototype for testing whether migration handover reports solve a useful customer problem. It is **not evidence of unique demand or a production certification tool**. The Actor has been deployed on Apify and passed a synthetic smoke test. Existing free diff tools are substitutes; validate the workflow before building more.

### Run locally

From this directory:

```sh
python -m csv_migration_verifier examples/source.csv examples/target.csv \
  --config examples/config.json --out my-first-report
```

Open `my-first-report/report.html` in a browser. Keep the three output files together. The directory must not already exist; each run uses a fresh directory to prevent accidental overwrites.

Exit codes: `0` = equal under configured rules; `1` = differences found; `2` = invalid input/configuration or execution/output error. A result of `1` is a successful comparison, not a software crash.

The example returns `1`: three records on each side, two matched keys, one changed record, one missing, one added and one cell accepted by tolerance. Exact balance totals are `42.50` and `45.004`, delta `2.504`.

```sh
python -m unittest discover -s tests -v
```

### Configuration

```json
{
  "mapping": {"account": "account_id", "region": "territory", "name": "display_name", "balance": "amount"},
  "keys": ["account", "region"],
  "tolerances": {"balance": "0.01"},
  "totals": ["balance"],
  "null_token": null
}
```

- `mapping` is required. Each source header maps to exactly one target header. Keys, tolerances and totals refer to **source** column names. Target mappings must be unique. Extra unmapped columns are deliberately ignored for comparisons, but duplicate or empty headers anywhere invalidate the input.
- `keys` is a required, nonempty ordered list. Composite keys are tuples, not concatenated strings. Empty/null key components and duplicate keys invalidate the entire comparison, even if duplicate rows are otherwise identical. Keys are always exact strings, never tolerant numbers.
- `tolerances` is optional. Each value must be a **JSON string** containing a nonnegative finite decimal. A changed numeric field is accepted when `abs(target - source) <= tolerance`. There is no relative tolerance or rounding. Set `"0"` to accept numerically equal formatting variants such as `1.0` and `1.00`.
- `totals` is optional. Exact decimal totals across **all** records on each side, including added/missing records. Delta is target minus source. Totals are informational, not an additional equality test. A column listed only in totals is still compared as an exact string.
- `null_token` is optional and defaults to JSON `null` (disabled). If provided, it must be a nonempty string; cells exactly matching it become semantic null. `"NULL"` and empty string are distinct when the token is enabled. There is no escape to represent the literal token separately, so choose one that does not occur as real text.
- Unknown options, duplicate JSON object keys, missing mapped headers, and invalid configuration values fail closed.

### Exact semantics

#### CSV and strings

UTF-8 with optional leading BOM; comma delimiter; double-quoted fields; doubled quotes inside quoted fields; LF, CRLF or CR line endings. Quoted embedded newlines are supported. Quotes in bare fields and characters after a closing quote are rejected. Wrong field counts, blank records, malformed quoting, invalid UTF-8, NUL characters and missing/empty/duplicate headers invalidate the run. A trailing line ending is fine.

There is no trimming, case folding, type inference, date parsing, Unicode normalization, header auto-matching or delimiter guessing. IDs such as `001` remain distinct from `1`. Whitespace is meaningful. Unquoted empty and quoted empty both mean the empty string: CSV does not retain a null-vs-empty distinction by itself. Missing fields are structural errors, not empty values.

Header-only inputs are accepted as empty datasets. Two empty datasets can be equal; **this does not establish that exports are complete or a migration succeeded**. The HTML report explicitly flags any empty dataset. Verify the expected export scope/record count independently.

#### Numbers

Only columns explicitly selected for tolerance or totals are numeric. **Every row** in each such column must contain a valid finite decimal, including unmatched rows and textually identical values. Null/empty numeric cells invalidate the whole run; no null skipping or zero substitution.

Accepted grammar: optional sign, ASCII digits with optional decimal point (or leading dot), optional scientific exponent. Whitespace, thousands separators, underscores, NaN and Infinity are rejected. Numeric strings are limited to 1000 characters and Decimal's stored exponent to ±1000. Arithmetic uses a sufficiently large Decimal context for these bounds and record limits, including cancellation and exact sums. Negative zero is numerically equal to zero only with a tolerance rule; default string comparison distinguishes them.

#### Output and invalid runs

- `report.html`: self-contained, script-free handover view with counts, changed fields, missing/added records, errors, exact totals, hashes and rules. Displays at most 100 items **per exception category**, plus 100 validation errors, with exact category totals. Links to complete sibling files; no CDN or remote resources.
- `report.json`: authoritative machine-readable evidence containing all exceptions/errors, complete normalized config, version, hashes, input headers and counts. JSON null is distinct from `""`. The config hash is over normalized compact UTF-8 JSON with sorted object keys, not original config-file bytes.
- `exceptions.csv`: review copy with UTF-8 BOM and CSV quoting. Every data cell other than the fixed category is **apostrophe-prefixed JSON**, preventing formula execution and preserving string-type IDs during ordinary spreadsheet import. This intentionally changes presentation; use JSON for lossless programmatic work. Do not remove these prefixes before opening untrusted values in a spreadsheet; import/export applications can have different behavior.
- `missing` means present in source but absent from target. `added` means present in target but absent from source. `changed` contains one entry per different mapped cell. Keys and columns are sorted deterministically; row and column order do not affect reconciliation. Hashes intentionally change when the raw files change. `unchanged_records` includes records accepted by tolerance; `tolerated_cells` counts textually different cells accepted by their numeric rule.
- **Any validation error invalidates the entire run.** `counts` and `totals` are JSON null, and exceptions are empty. Never interpret empty exceptions as equality without checking `status`. Input `parsed_records` counts structurally parsed data records observed; `valid_records` counts valid-width/key/numeric rows including duplicates; `unique_keys` counts retained valid keys. `complete` says only whether parsing reached EOF, not whether validation passed. Early syntax/header/read failures may have zero/absent counts. Counts may be partial on resource limits. No diff is computed from a partially accepted file.

All HTML fields are escaped. A restrictive Content Security Policy disallows scripts and external resources. Hashes fingerprint bytes; they do not authenticate provenance, sign results, or prove snapshot completeness. Inputs are read independently, so obtain consistent source/target export snapshots yourself. Reports contain the actual compared data: keep them private and handle them like the inputs. No encryption, access control, redaction or secure deletion is provided.

### Bounded prototype limits

- At most **10 MiB per CSV**, **100,000 data records per input**, **1 MiB configuration JSON**
- Python CSV parser's default field-size limit (normally 131,072 characters); oversized fields fail closed
- Files and rows are held in memory; no streaming/external-sort mode
- No automatic schema/type discovery, database connections, remote URLs, archives, Excel files, report signing, joins, fuzzy matches, migration writes, or correction actions
- Outputs are fresh local files; an output-write error can leave a partial directory. Treat a non-completed command as failed and use a new directory after fixing the cause
- Test coverage is synthetic. Runtime/install checks were performed with the container's Python; an Apify synthetic smoke test passed; no Windows/Excel acceptance testing or production security audit has been performed

The limits prevent accidental unbounded local workloads; this is not a hardened multi-tenant service. Do not expose the Python API to untrusted callers or increase limits casually.

### Local Apify-compatible file convention

The adapter reads and writes **local files only**, using Apify's documented default key-value store layout. It does not require the Apify SDK, CLI, token or account; it does not invoke any API and is not a deployable cloud integration.

```sh
mkdir -p demo-storage/key_value_stores/default
python - <<'PY'
import json
from pathlib import Path
payload = {
    "sourceCsv": Path("examples/source.csv").read_text(encoding="utf-8"),
    "targetCsv": Path("examples/target.csv").read_text(encoding="utf-8"),
    "config": json.loads(Path("examples/config.json").read_text())
}
Path("demo-storage/key_value_stores/default/INPUT.json").write_text(json.dumps(payload), encoding="utf-8")
PY
python -m csv_migration_verifier.apify_local --storage demo-storage
```

Alternatively set `APIFY_LOCAL_STORAGE_DIR`; otherwise the root is `storage`. Only `key_value_stores/default` is supported. INPUT must contain exactly `sourceCsv`, `targetCsv`, `config`, with both CSVs inline strings. The input JSON limit is 22 MiB (escaping can make a CSV too large for this envelope). Core per-CSV limits still apply. The adapter creates `OUTPUT.json`, `REPORT.html`, `EXCEPTIONS.csv` beside INPUT and refuses to replace any existing output. Temporary CSV files are removed when processing finishes; local INPUT and output files remain. Malformed wrapper inputs fail before generating a comparison report. It refuses cloud mode when `APIFY_IS_AT_HOME` is `1` or `true`.

This local adapter remains separate from the optional SDK adapter below. **Uploading/running on Apify sends the supplied CSV data to Apify**, with cloud retention, access and billing considerations. A deployment and synthetic smoke test have been completed on Apify; no paid service has been purchased.

Storage-convention reference: [official Apify key-value store documentation](https://docs.apify.com/storage/key-value-store), checked 2026-09-30.

### Optional Apify SDK / Docker scaffold

`csv_migration_verifier.apify_actor` uses the real SDK interface: `async with Actor`, `Actor.get_input()`, and `Actor.set_value()`. It accepts the same inline payload and applies the same core rules. It writes three default key-value store records: `OUTPUT` (JSON object), `REPORT.html` (HTML) and `EXCEPTIONS.csv` (CSV bytes). After those records are saved, it writes one default-dataset summary row containing only `status` and `counts`, with no raw input values. Invalid comparisons retain `status: "invalid"` and null counts. It never fetches remote CSV URLs, touches databases, uses proxies, or calls custom billing/charging events.

`.actor/actor.json`, input/output schemas, `Dockerfile`, and `requirements-apify.txt` provide the platform packaging scaffold. Docker uses Python 3.12 and the optional Apify SDK pinned to 3.4.1. The offline CLI never imports or requires it. Building the image downloads an official Python base image and SDK dependencies; unlike running the offline CLI, this is not an offline operation.

**Verification boundary:** 38 distinct standard-library tests pass, including a stub SDK checking lifecycle, output keys/types, safe HTML, invalid-run persistence and persistence-error propagation. JSON files and referenced paths are checked. An independent local execution of the actual SDK adapter also passed with Apify 3.4.1, apify-client 2.5.1, Crawlee 1.10.3 and Python 3.12.14: synthetic fixtures, fresh local storage, tokens unset and `APIFY_IS_AT_HOME=false`; the SDK wrote OUTPUT, REPORT.html and EXCEPTIONS.csv and exited 0. The SDK dependencies were installed in an isolated temporary location for that check. Docker was unavailable for the local checks. An Apify platform build and a synthetic cloud smoke test have since passed. This remains a prototype; comprehensive platform acceptance and billing tests have not been performed. The direct SDK dependency is pinned; lock and review transitive dependencies before production.

For an approved later platform test, use synthetic input first. Successful Actor execution means the report was generated; read `OUTPUT.status` (`equal`, `different`, `invalid`) to determine the comparison result. Differences and CSV validation errors are persisted as evidence, not raised as infrastructure failures. Malformed wrapper inputs or output persistence failures fail execution. Partial records may remain if a cloud write fails; a failed run is not a completed report. The SDK receives already-parsed input objects, so duplicate JSON-key detection applies to local raw JSON files, not bytes consumed by the platform before this adapter runs.

The SDK may use network services when configured for cloud storage. Running it in the cloud transmits INPUT and output records to Apify; outputs contain actual values. No application-level redaction or additional access controls are provided. No custom charging code is included, but ordinary platform compute/storage costs can still apply. Account connection, uploads, deployment, publishing and billing configuration require separate authorization.

Official references: [Actor definition](https://docs.apify.com/actors/development/actor-definition/actor-json), [input schema](https://docs.apify.com/actors/development/actor-definition/input-schema/specification/v1), [output schema](https://docs.apify.com/actors/development/actor-definition/output-schema). Checked 2026-09-30.

### Files

- `csv_migration_verifier/core.py`: validation, comparison and provenance
- `csv_migration_verifier/reports.py`: safe HTML/JSON/CSV reports
- `csv_migration_verifier/__main__.py`: CLI
- `csv_migration_verifier/apify_local.py`: optional local storage adapter
- `csv_migration_verifier/apify_actor.py`: optional SDK persistence adapter
- `.actor/`, `Dockerfile`, `requirements-apify.txt`: cloud packaging used in the tested deployment
- `examples/`: synthetic CSVs and config
- `example-output/`: generated synthetic report
- `tests/`: standard-library unit tests

Optional packaging metadata is provided, but normal use does not install or download anything. Published use is governed by the [Apify Standard Actor Contract](https://docs.apify.com/legal/standard-actor-contract).

# Actor input Schema

## `sourceCsv` (type: `string`):

UTF-8 CSV content including headers, up to 10 MiB and 100,000 data rows. This is content, not a URL.

## `targetCsv` (type: `string`):

Target CSV content. Leading-zero IDs remain strings. Cloud storage contains this input and report values.

## `config` (type: `object`):

Required mapping and keys; optional tolerances (decimal strings), totals and null\_token. All column references use source names. Unknown fields fail closed.

## Actor input object example

```json
{
  "sourceCsv": "id,name,amount\n001,Ada,10.00\n",
  "targetCsv": "ID,Name,Amount\n001,Ada,10.01\n",
  "config": {
    "mapping": {
      "id": "ID",
      "name": "Name",
      "amount": "Amount"
    },
    "keys": [
      "id"
    ],
    "tolerances": {
      "amount": "0.01"
    },
    "totals": [
      "amount"
    ]
  }
}
```

# Actor output Schema

## `report` (type: `string`):

Reconciliation summary and capped exception previews with input hashes. Inspect OUTPUT status before treating a run as equal.

## `evidence` (type: `string`):

Authoritative status, counts, full exceptions, errors, rules and provenance. Invalid runs have null diff counts.

## `exceptions` (type: `string`):

Apostrophe-prefixed JSON cells. Use JSON evidence for lossless programmatic consumption.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "sourceCsv": `id,name,amount
001,Ada,10.00`,
    "targetCsv": `ID,Name,Amount
001,Ada,10.01`,
    "config": {
        "mapping": {
            "id": "ID",
            "name": "Name",
            "amount": "Amount"
        },
        "keys": [
            "id"
        ],
        "tolerances": {
            "amount": "0.01"
        },
        "totals": [
            "amount"
        ]
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("lucaslucas/csv-migration-verifier").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "sourceCsv": """id,name,amount
001,Ada,10.00
""",
    "targetCsv": """ID,Name,Amount
001,Ada,10.01
""",
    "config": {
        "mapping": {
            "id": "ID",
            "name": "Name",
            "amount": "Amount",
        },
        "keys": ["id"],
        "tolerances": { "amount": "0.01" },
        "totals": ["amount"],
    },
}

# Run the Actor and wait for it to finish
run = client.actor("lucaslucas/csv-migration-verifier").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "sourceCsv": "id,name,amount\\n001,Ada,10.00\\n",
  "targetCsv": "ID,Name,Amount\\n001,Ada,10.01\\n",
  "config": {
    "mapping": {
      "id": "ID",
      "name": "Name",
      "amount": "Amount"
    },
    "keys": [
      "id"
    ],
    "tolerances": {
      "amount": "0.01"
    },
    "totals": [
      "amount"
    ]
  }
}' |
apify call lucaslucas/csv-migration-verifier --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,lucaslucas/csv-migration-verifier"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/6FgetP2SQ9F6A4DlU/builds/ltfSJKhCSQBNE160I/openapi.json