Supplier Catalog Reconciliation & CSV/Excel Dedupe

Match a supplier product catalogue to your inventory export, then inspect confirmed matches, uncertain pairs and products without a confirmed match. Bring CSV, TSV, XLSX or Apify datasets. Download a merged catalogue and explainable comparison files. This Actor processes your supplied tables; it does not scrape a shop or automatically update your store.

Try a catalogue demo first

Open the tested catalogue example to copy a prepared configuration, or follow these steps:

1. Open Input and select Link two tables (A to B).
2. Leave both table sources empty to use the built-in synthetic product catalogues. Set Match threshold to 95 and Review threshold to 75 for a cautious first look.
3. Start the Actor, then open REPORT.html, matches.csv, REVIEW.csv and merged.xlsx from the run's key-value store. File links are listed in the run's OUTPUT record.
4. Compare the match explanations and review competing or borderline pairs before importing anything.

The built-in catalogues are invented demonstrations, not customer data or prior client work. The demo does not charge record-processed events. Apify's applicable run-start/platform rules still apply; check the displayed run options and set a maximum cost. For your own files, both input tables count toward event pricing.

What a buyer gets

Output	Use it for
matches.csv	Confirmed pairs with source row numbers, scores and reasons
REVIEW.csv	Borderline or competing pairs that need a decision
clusters.csv	A row-level audit trail; single-record clusters include unmatched records
merged.csv / merged.xlsx	A combined catalogue using your chosen survivorship rule
REPORT.html / OUTPUT	Counts, settings, coverage and output links

Price changes are not applied to a live shop. Choose which source to prefer and review uncertain matches. A similarity score is not a calibrated probability or a guarantee that two products are identical.

Small reproducible reconciliation example

This second, transparent fixture has 5 inventory rows and 6 supplier rows. Save the two CSV blocks as UTF-8 files and upload them as Table A and Table B. The expected results below were checked with the matching engine using the explicit settings shown here. They are separate from the larger built-in demo above.

Table A — our catalogue:

SKU,Product Name,Brand,Price
BOLT-005,Hex bolt 5 mm stainless,Acme,0.55
000123,Steel washer 10 mm,Acme,0.05
CABLE-002,USB-C cable 2 m black,Linea,4.20
,Workshop storage bin 10 L,Forge,8.50
DRILL-018,Cordless drill 18 V,Forge,79.00

Table B — supplier catalogue:

SKU,Product Name,Brand,Price
bolt 005,Hex bolt 5mm stainless,ACME,0.56
000123,Steel washer 10 mm,Acme,0.06
CABLE002,USB C cable 200 cm black,Linea,4.10
,Workshop storage crate 10 L,Forge,8.90
BOLT-006,Hex bolt 6 mm stainless,Acme,0.60
HOSE-025,Garden hose 25 m,Verde,22.00

Apply these settings after attaching the files in the Form editor. In JSON mode retain the actual fileUploadA / fileUploadB values generated by your uploads; don't replace them with local paths or leave the sources empty, which would run the built-in demo instead.

{
  "mode": "link",
  "autoConfigure": false,
  "fields": [
    {
      "columnA": "SKU",
      "columnB": "SKU",
      "type": "id_code",
      "weight": 4,
      "minScore": 100
    },
    {
      "columnA": "Product Name",
      "columnB": "Product Name",
      "type": "text",
      "weight": 3
    },
    {
      "columnA": "Brand",
      "columnB": "Brand",
      "type": "exact",
      "weight": 1
    }
  ],
  "matchThreshold": 95,
  "reviewThreshold": 75,
  "cardinality": "one_to_one",
  "survivorship": "prefer_a",
  "outputColumns": "all",
  "maxRows": 1000
}

Expected result	Why it matters
A1–B1, A2–B2 and A3–B3 confirmed (100.00)	SKU case/punctuation and compatible unit formatting
A4–B4 review (92.71)	No SKU; bin versus crate needs human confirmation
A5 has no confirmed match	Inventory drill is retained
B5 and B6 have no confirmed match	A different-size bolt and a new hose stay separate
Counts: A = 3 matched + 1 review + 1 unmatched; B = 3 + 1 + 2	All 11 source rows are accounted for

The demo uses prefer_a: existing inventory values are retained. Original identifiers are exported, but id_code matching normalizes punctuation and leading zeros. If those distinguish products in your catalogue, use exact for that field and check the mapping. Import CSV identifiers as text in Excel to preserve leading zeros. These fixture results demonstrate specific cases, not accuracy on arbitrary catalogues. At the default threshold of90 the bin/crate pair is accepted; this example deliberately uses95 to send it for review.

Price and safe first use

The current record event price is $0.001 per input record ($1 per1,000). Two files of1,000 rows each count as2,000 records: $2 in record events, plus any other charges shown in Apify's Pricing tab. The built-in demo has no record-event charge. Check the live Pricing tab and run limit before starting.

Begin with a representative subset and check its ambiguous pairs before a full catalogue. Keep a copy of the original export. No handwriting/PDF inputs, personal-contact enrichment, guaranteed match accuracy or automatic storefront changes are offered by this Actor.

Need help setting up a supplier comparison?

Open an issue with column headings, approximate row counts and a few synthetic, non-confidential examples. Say which source should win and how you want unmatched or uncertain products handled. Don't post confidential catalogues, credentials or personal data in public issues. We can assess a bounded paid setup separately; custom assistance is not included in the per-record price.

Detailed configuration, limits and data handling

The reference below documents the existing engine. Its default match threshold remains90; the cautious examples above override it to95.

What it does

You have	Mode	You get
One table with duplicates	dedupe	clusters of duplicate records, one golden record per cluster
Two tables describing the same things	link	every record of A paired with its counterpart in B, plus the unmatched rest of both

Matching is configurable per column and explainable: every pair carries a score per field and a sentence such as

gtin 85: codes differ by one digit/transposition ('5012345678900' vs '5012345678909'); check digit invalid in B; sku 100: same code HB600; name 96: 'Hex Bolt 5 mm Stainless' vs 'Hex bolt 5mm stainless steel' | = 93.4 -> match

Field types

type	Use it for	How it compares
id_code	EAN, UPC, GTIN, ISBN, SKU, part numbers, item codes	Ignores case, spaces, dashes, slashes and leading zeros (AB-0012 = ab 12), repairs spreadsheet damage (4006381333931.0, 4.006381333931E+12, UPC with a lost leading zero). Validates the GS1 check digit. Equal codes score 100. A code that differs by one digit or one transposition from a valid GTIN (a typo) scores 85 and is flagged in the explanation. Other differences score 0: two part numbers one digit apart are different parts.
text	product names, descriptions, company names, addresses	Fuzzy: token-set ratio blended with token-sort and plain ratio on normalized text: accents and case, punctuation, &/and, units (1/2 in = 0.5 inch = 1/2", 500 mL = 0.5 L, 2 m = 200 cm, lb/oz/kg, 12-pack = 12 pcs, 12 V = 12v), company legal forms (Inc, LLC, Ltd, GmbH, Corp, SA, BV, ...) and your own stopwords. Names that differ in a number (5 mm vs 6 mm) lose points (numberPenalty, default 25). A short name that is merely a subset of a long one (Bolt vs Hex bolt zinc 50 pack) is not a perfect match.
exact	country, postal code, domain, category	Equal after trimming, case and accent folding. "normalize": "domain" reduces https://www.Acme.com/about to acme.com.
number	price, weight, quantity	tolerance (absolute) or tolerancePct. Full score inside the tolerance, fading linearly to 0 at three times the tolerance.
geo	latitude/longitude	"columnA": ["lat", "lon"], meters radius. Haversine distance with the same fade-out.

A field can use several columns for text (they are joined) and exactly two for geo.

Scoring in one paragraph

The pair score (0-100) is the weighted mean of the field scores over the fields that have a value on both sides. Fields missing on either side are ignored, but a pair that has less than a third of the configured weight present can never be an automatic match (it goes to review). An id_code field marked decisive (auto-configured for EAN/UPC/GTIN columns) turns a valid-GTIN match into a match unless the other fields clearly disagree, in which case the pair goes to review ("same barcode, different product" is exactly what you want to look at). minScore on a field vetoes pairs where both values exist and that field scores below it. Scores >= matchThreshold (default 90) are matches, scores between reviewThreshold (default 75) and the match threshold go to REVIEW.csv.

autoConfigure

With no fields given, columns are recognised from their headers (autoConfigure, on by default): EAN/UPC/GTIN/ISBN/barcode, SKU/MPN/part number/item code/article number, product name / company name / title, brand/manufacturer, description, website/domain, address, city, ZIP/postal code, country and latitude/longitude. In link mode the matching columns of both tables are paired by role (EAN with Barcode, Product Name with Description). The guess is listed in REPORT.html and in the run's OUTPUT. Email, phone, person-name and date-of-birth columns are never used by autoConfigure.

Input

{
  "mode": "link",
  "urlA": "https://example.com/our-catalog.csv",
  "urlB": "https://example.com/supplier-price-list.xlsx",
  "sheetB": "Prices",
  "fields": [
    { "columnA": "EAN", "columnB": "Barcode", "type": "id_code", "weight": 4, "decisive": true },
    { "columnA": "SKU", "columnB": "Supplier Part No", "type": "id_code", "weight": 3 },
    { "columnA": "Product Name", "columnB": "Description", "type": "text", "weight": 3 },
    { "columnA": "Brand", "columnB": "Manufacturer", "type": "text", "weight": 1 },
    { "columnA": "Price", "columnB": "Cost", "type": "number", "tolerancePct": 15, "weight": 0.5 }
  ],
  "matchThreshold": 90,
  "reviewThreshold": 75,
  "cardinality": "one_to_one",
  "survivorship": "most_recent",
  "survivorshipDateColumn": "Updated"
}

Field	Meaning
mode	dedupe (table A only) or link (A to B). Default dedupe.
fileUploadA/B, urlA/B, datasetIdA/B	Where each table comes from (upload > URL > dataset). CSV, TSV and XLSX are detected from the content, not the file name. UTF-8 (with or without BOM), UTF-16 and Windows-1252 text and `, ; tab
sheetA/B	XLSX sheet name or number. Default: first visible sheet.
fields	Match fields (see above). Optional per field: name, weight, decisive, minScore, stopwords, unitNormalization, legalSuffixes, numberPenalty, tolerance, tolerancePct, meters, normalize. In link mode columnB defaults to the same header as columnA.
autoConfigure	Guess fields from headers when fields is empty. Default on.
matchThreshold, reviewThreshold	90 and 75 by default.
cardinality	one_to_one (default: best score first, competing pairs go to review) or many_to_many (link mode).
survivorship	Golden-record rule: most_complete (default), most_recent (needs survivorshipDateColumn), prefer_a, prefer_b. In dedupe, prefer_a keeps the first row.
fillGaps	Fill empty cells of the golden record from the other records of the cluster (default on; filled columns are listed).
mergedIncludeUnique	Keep records without a match in merged.xlsx (default on: the full combined list).
outputColumns	keys (matching columns only) or all columns in matches.csv / REVIEW.csv.
maxRows	Rows read per table (default 250,000, max 1,000,000). Rows beyond it are ignored and not charged.
maxBlockSize	Advanced: speed/recall trade-off for the candidate search (default 250).
datasetItems	Pairs and clusters pushed to the dataset (default 20,000). The CSV files always hold everything.

Leave all tables empty to run a small built-in demo (free).

Output

Dataset - one item per match or review pair (item_type: "pair": status, score, a_row, b_row, cluster_id, field_scores, explanation, record_a, record_b) and one per cluster of two or more records (item_type: "cluster").

Key-value store (default store of the run):

File	Content
matches.csv	Accepted pairs: score, cluster id, one column per field score, the explanation, and the values of both records.
REVIEW.csv	Borderline pairs (and pairs that lost a one-to-one competition) with the reason, best first (up to 100,000 rows).
clusters.csv	One row per input record: cluster id, cluster size, table and row, whether it is the golden source, best pair score, needs-review flag, weakest accepted pair in its cluster.
merged.xlsx / merged.csv	One golden record per cluster with the chosen survivorship rule, cluster_size, source_rows, weakest_match_score, filled_columns. In link mode the columns of B are mapped onto those of A (mapped fields and same-named columns merge; others are appended). Identifier columns stay text; numeric columns become numbers; cells starting with = are stored as text.
REPORT.html	Counts, a score histogram (inline SVG) with the thresholds, the top ambiguous pairs, the configuration used, field coverage and the blocking statistics.
OUTPUT	The run summary as JSON (counts, thresholds, rows covered, warnings, file links).

Row numbers are data rows: 1 is the first row under the header. Everything is deterministic: the same input and settings always give the same pairs, cluster ids and files (ties are broken by row number). CSV cells that could be executed as formulas in a spreadsheet (=..., @..., -abc) are prefixed with an apostrophe in the CSV files.

Pricing

Pay per event: record-processed, suggested $0.001 per input record - $1 per 1,000 records. Every row of every input table is charged exactly once, whatever the number of matches, clusters or files. Nothing else is charged (no dataset-item fee, no per-pair fee, the report and files are included). Rows you did not process (above maxRows, empty rows, blank lines) are not charged, and the built-in demo is free.

Example	Records	Cost
Dedupe a 20,000-row company list	20,000	$20
Merge two 50,000-row catalogs	100,000	$100
Link 200,000 x 200,000 rows	400,000	$400

Maximum charge per run. Set a maximum charge in the run options and the Actor stays inside it: rows are charged in whole blocks (in input order, interleaved across A and B in link mode; a block is 1,000 rows or 0.5% of the input, whichever is larger) and only after matching succeeded. When the next block no longer fits, the Actor matches the blocks it can pay for, writes complete results for those rows, and says in the status message, REPORT.html and OUTPUT how many rows were covered (rows_covered, rows_covered_a, rows_covered_b). The results are exact for the covered rows; the remaining rows are neither matched nor charged. Raise the limit and run again to cover them.

Performance and limits

Measured on a 2-core container with synthetic product catalogs (SKU + EAN + name + brand + price):

Task	Time	Peak memory
Dedupe 50,000 rows	about 9 s	about 0.25 GB
Link 25,000 x 25,000 rows	about 5 s	about 0.2 GB
Link 200,000 x 200,000 rows (all outputs, 400,000 charged rows)	about 2 min	about 1.2 GB

• The default run memory is 4 GB, the minimum 1 GB. Wide tables need more: rows are held in memory, so plan roughly 4 GB for 200,000 x 200,000 rows of 30-50 columns. XLSX files read much slower than CSV (roughly 50,000 cells per second, so 200,000 rows x 5 columns take about 20 s to read; CSV reads about 700,000 cells per second).
• Speed comes from blocking: only records that share a key are compared - the canonical id and its prefix/suffix (catches single-digit typos), the rarest words of a name, the first word, a text prefix, geo grid cells, and two sorted-neighbourhood windows. Groups larger than maxBlockSize are compared inside a window instead of being skipped. The number of candidate pairs is budgeted (40 million) and the block size is tightened automatically if it would be exceeded.
• Text fields are compared on their first 1,500 characters. Download limit 200 MB per file. Legacy .xls, PDFs and HTML pages are rejected with a clear message - save as XLSX or CSV.
• Blocking can miss a pair that shares no rare word, no id, no prefix and is not a neighbour in sorted order - for example two names with a typo in every word. Lower matchThreshold and raise maxBlockSize if recall matters more than speed.
• Text matching compares words and characters; it does not know that two different words are synonyms ("sofa" vs "couch") or translate between languages.
• many_to_many link mode and large duplicate groups can chain unrelated records through intermediate ones. The weakest accepted pair of each cluster is reported (weakest_match_score); clusters of more than 50 records are flagged in the report.
• Dates in survivorshipDateColumn are read as ISO (2024-05-31), YYYY/MM/DD, DD.MM.YYYY or D/M/YYYY (tick dayFirst when 03/04/2024 means 3 April).

FAQ

How do I choose the weights? Give identifiers the highest weight (EAN 4, SKU 3), the name 2-3 and descriptive fields 1. Start with autoConfigure, open REPORT.html, look at the top ambiguous pairs and REVIEW.csv, then adjust.

Why is a pair with the same name not a match? Look at its explanation: usually a size (5 mm vs 6 mm), a different valid barcode or a different SKU lowered the score on purpose. Change numberPenalty, the weights or minScore if you disagree.

Will two different products with sequential EANs be merged? No. Two valid GTINs that differ are scored 0; only a one-digit difference to an invalid check digit is treated as a typo.

Can I match on a column that is not in the list of types? Yes: use exact for anything that must be identical (category, country, supplier id) or text for anything fuzzy. Several columns can feed one text field.

What happens to rows I do not want to match? They stay in clusters.csv as single-record clusters and, unless you switch it off, in merged.xlsx.

Can I run it on a schedule or call it from code?

import os
from datetime import timedelta
from decimal import Decimal
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("jyinnovation/record-linker-dedupe-for-csv-excel-and-catalogs").call(
    run_input={"mode": "link", "matchThreshold": 95, "reviewThreshold": 75, "outputColumns": "all"},
    max_total_charge_usd=Decimal("0.01"),
    run_timeout=timedelta(seconds=60),
)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    if item.get("item_type") == "pair":
        print(item["status"], item["score"], item["explanation"])

This runs the built-in synthetic catalogue example with no file URLs. Set your own token locally as the APIFY_TOKEN environment variable; never paste it in an issue. For real files, provide accessible table sources and review/raise the cost limit after checking pricing. The dataset contains both pairs and clusters, so the example filters for pair items before reading pair-only fields.

Use the Apify API, Make, Zapier, Google Sheets or webhooks the same way.

Is my data sent anywhere? No. Matching runs entirely inside the Actor with open-source libraries; there are no third-party APIs and no AI/LLM calls. Files you upload, the results and the report stay in your run's storages under your Apify retention settings. Links to private networks and localhost are refused.

Acceptable use

This Actor is meant for business records: products, parts, suppliers, companies, branches and locations. Do not use it to find, track, profile or enrich individuals, or to build lists of personal contact data. autoConfigure deliberately ignores e-mail, phone, person-name and birth-date columns. You are responsible for having the right to process the data you provide.

Support

Something matched that should not have, or the other way round? Open an issue on the Actor page with the (non-confidential) pair, your explanation text and the settings. JYInnovation reads every report.