# BOM & Quote Normalizer (`leonyu/bom-quote-normalizer`) Actor

Normalize KiCad, Altium, EAGLE, JLCPCB and IPC-2581 BOMs plus supplier quotes (CSV, XLSX, PDF) into one clean schema: expanded designators, DNP, normalized values, price breaks and lead times, with source provenance.

- **URL**: https://apify.com/leonyu/bom-quote-normalizer.md
- **Developed by:** [Leon Yu](https://apify.com/leonyu) (community)
- **Categories:** Developer tools, Automation
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $10.00 / 1,000 document processeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## BOM & Quote Normalizer

Turn messy BOMs and supplier quotes into **one clean, comparable schema**, with a trace back to the source cell for every value.

**Supported inputs:** KiCad / Altium / EAGLE-Fusion / JLCPCB CSV & XLSX, Octopart-style BOM exports, IPC-2581 XML, supplier quotes in CSV/XLSX, and PDF BOMs and quotes that contain tables.

### How to provide files

- **Public link:** paste a direct URL to the CSV/XLSX/PDF/XML file into *Files (URLs)*.
- **Your own file:** in Apify Console open **Storage → Key-value stores**, create a store (or open any existing one), click **Add record / Upload file**, upload the BOM or quote, then copy the record's URL (`https://api.apify.com/v2/key-value-stores/<storeId>/records/<key>`) and paste it into *Files (URLs)*. Never put your API token in a URL. If the store is private, make that one record or store publicly readable (or use a signed/shareable link). Query strings and credentials are removed from every URL the Actor writes to logs and outputs.
- Only public `http`/`https` URLs are fetched; private-network and cloud-metadata addresses are refused.
- Sites that block automated downloads: download the file yourself and upload it as above.

### What it does

- Detects encoding (UTF-8, GB18030, Shift-JIS, …), delimiter and the real header row (it skips title and metadata rows).
- Maps columns with a multilingual alias table (English, 中文, Deutsch, 日本語), e.g. `位号`, `Menge`, `LCSC Part #`, `Price @100`.
- Expands designators (`R1-R4, R7` becomes R1 R2 R3 R4 R7) and flags rows where the designator count doesn't match the quantity.
- Detects DNP/DNI, normalizes values (`4k7`, `0.1u`, `470uF`, `2R2`), parses price breaks, MOQ/SPQ and lead times (`4 weeks`, `10-12 wks`, `现货`, `3 Wochen`).
- Reads quote header fields: supplier company, quote number, date, validity, currency, Incoterms and payment terms.
- **Privacy:** contact names, emails and phone numbers are removed during parsing and never written to the output.
- PDF: tables are extracted with pdfplumber. Pages without a table can optionally use an AI fallback (Gemini Flash-Lite). **LLM values must appear verbatim in their quoted source line** (`source_text`, stored on the line as `provenance.raw`). A value that is not in that quote is dropped, and a quote longer than 3 lines, 400 characters, or half the page is rejected.

#### PDF pages without tables

Pages where pdfplumber finds no table (free-text quotes, forms) can only be read with the AI fallback (`use_llm`). **The AI fallback is currently not active on this Actor (no AI key configured)**, so such pages are skipped with a warning, and a file with no extracted lines is not charged. Scanned or image-only PDFs need OCR, which isn't supported yet.

#### Limits

- `maxPdfPages` (default 200, range 1–2000): only the first N pages of each PDF are processed. A JSON `null` uses the default.
- `maxLlmPages` (default 50, range 0–1000): upper limit on AI-read pages per run. Each AI call times out after 30 seconds. Before every AI page the run checks that one more `llm-page` event still fits in **Max total charge**, and skips the rest when the count is 0.
- The run stops charging, and stops processing further files, once your **Max total charge** for the run is reached.
- The run stops starting new files and new AI pages once about 90% of the run timeout has elapsed (`ACTOR_STARTED_AT` … `ACTOR_TIMEOUT_AT`).
- Set the run timeout to about **900 seconds** in the run options (`timeoutSecs`). Actor specification 1 has no default-timeout field in `actor.json`. Default memory is 1024 MB.

### Example input

```json
{
  "files": [
    {"url": "https://example.com/board-bom.csv"}
  ],
  "use_llm": true,
  "maxLlmPages": 50,
  "maxPdfPages": 200
}
```

### Output

Dataset records with `record_type` = `document` (one per file) or `line_item`. Each line item has `confidence` (1.0 for exact column matches, 0.7–0.9 for fuzzy matches, ≤0.8 for AI values) and `provenance` (sheet/row/page, raw source text, method). For AI lines, `provenance.raw` is the quoted source line, and every kept value appears verbatim in that quote.
The key-value store also gets `OUTPUT.csv` and `OUTPUT.xlsx` with all line items.

Example records for a one-line CSV (`Designator,Qty,Value,MPN,Manufacturer` / `R1,1,10k,RC0402FR-0710KL,Yageo`):

```json
{
  "record_type": "document",
  "document_id": "6fb7a755167d5a60",
  "source_file": "board-bom.csv",
  "doc_type": "bom",
  "detected_format": "csv/generic",
  "supplier_name": null,
  "quote_number": null,
  "quote_date": null,
  "valid_until": null,
  "currency_default": null,
  "incoterms": null,
  "payment_terms": null,
  "line_count": 1,
  "total_amount": null,
  "parse_method": "deterministic",
  "llm_pages": 0,
  "column_mapping": {
    "Designator": {"field": "reference_designators", "confidence": 1.0},
    "Qty": {"field": "quantity", "confidence": 1.0},
    "Value": {"field": "value", "confidence": 1.0},
    "MPN": {"field": "mpn", "confidence": 1.0},
    "Manufacturer": {"field": "manufacturer", "confidence": 1.0}
  },
  "warnings": ["encoding=utf-8 delimiter=','"],
  "pii_redacted": false
}
```

```json
{
  "record_type": "line_item",
  "mpn": "RC0402FR-0710KL",
  "manufacturer": "Yageo",
  "alternates": [],
  "supplier_part_number": null,
  "supplier": null,
  "description": null,
  "value": "10k",
  "value_normalized": {"number": 10000.0, "unit": "ohm"},
  "footprint": null,
  "reference_designators": ["R1"],
  "designator_count_mismatch": false,
  "quantity": 1,
  "unit": null,
  "dnp": false,
  "unit_price": null,
  "currency": null,
  "price_breaks": [],
  "extended_price": null,
  "moq": null,
  "spq": null,
  "lead_time_days": null,
  "lead_time_raw": null,
  "packaging": null,
  "confidence": {
    "overall": 1.0,
    "fields": {
      "reference_designators": 1.0,
      "quantity": 1.0,
      "value": 1.0,
      "mpn": 1.0,
      "manufacturer": 1.0
    }
  },
  "provenance": {
    "sheet": null,
    "row": 2,
    "page": null,
    "bbox": null,
    "columns": {
      "reference_designators": "Designator",
      "quantity": "Qty",
      "value": "Value",
      "mpn": "MPN",
      "manufacturer": "Manufacturer"
    },
    "raw": "R1,1,10k,RC0402FR-0710KL,Yageo",
    "method": "deterministic"
  },
  "document_id": "6fb7a755167d5a60",
  "line_no": 1
}
```

### Pricing (pay per event)

- `document-processed`: $0.01 per file that yields line items
- `line-item`: $0.002 per line
- `llm-page`: $0.01 per PDF page read by the AI fallback
- `apify-actor-start`: $0.00005 (Apify platform default)
- **Capped at $5 per document.** Charging stops at the cap, and you still get every line.
- Files that fail or yield no line items are not charged. Identical lines repeated across sheets of one workbook are charged once.

### Not included (yet)

No price or stock lookups on Octopart, Digi-Key, Mouser or LCSC; no OCR for scanned PDFs; no ERP integration.

### Development

`pip install -r requirements.txt pytest && pytest -q`. Test fixtures: `tests/fixtures` (public samples, see ATTRIBUTION.md) and `tests/fixtures/synthetic` (fictional data).

# Actor input Schema

## `files` (type: `array`):

Direct links to BOM / quote files: CSV, XLSX, PDF or IPC-2581 XML. You can also upload a file to an Apify key-value store and paste its record URL.

## `use_llm` (type: `boolean`):

Only used for PDF pages where no table could be extracted. Every AI value is checked against the page text; anything not found is dropped. Charged per AI page.

## `maxLlmPages` (type: `integer`):

Upper limit on PDF pages read by the AI fallback in one run (each is charged as llm-page).

## `maxPdfPages` (type: `integer`):

Only the first N pages of each PDF are processed.

## Actor input object example

```json
{
  "files": [
    {
      "url": "https://raw.githubusercontent.com/tinyvision-ai-inc/pico-ice/main/Board/Rev0/Output%20Files/Assembly/pico-ice_bom_0.3.csv"
    }
  ],
  "use_llm": true,
  "maxLlmPages": 50,
  "maxPdfPages": 200
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset with document and line_item records

## `csv` (type: `string`):

All line items as CSV

## `xlsx` (type: `string`):

All line items as an Excel workbook

## `files` (type: `string`):

Key-value store with the exports

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "files": [
        {
            "url": "https://raw.githubusercontent.com/tinyvision-ai-inc/pico-ice/main/Board/Rev0/Output%20Files/Assembly/pico-ice_bom_0.3.csv"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("leonyu/bom-quote-normalizer").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "files": [{ "url": "https://raw.githubusercontent.com/tinyvision-ai-inc/pico-ice/main/Board/Rev0/Output%20Files/Assembly/pico-ice_bom_0.3.csv" }] }

# Run the Actor and wait for it to finish
run = client.actor("leonyu/bom-quote-normalizer").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "files": [
    {
      "url": "https://raw.githubusercontent.com/tinyvision-ai-inc/pico-ice/main/Board/Rev0/Output%20Files/Assembly/pico-ice_bom_0.3.csv"
    }
  ]
}' |
apify call leonyu/bom-quote-normalizer --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,leonyu/bom-quote-normalizer"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/helQrpLRKiCpE8VUZ/builds/Bj0ExaHlTPN7ZDIDv/openapi.json