# AI Token Saver — Deterministic Tools for AI Agents (`michiganpublictech/ai-token-saver`) Actor

Cut LLM token spend: 80+ deterministic text, web and data operations (HTML to Markdown, chunking, PII redaction, JSON repair, extraction, dedupe, math, dates) for AI agents, with per-call token-savings estimates.

- **URL**: https://apify.com/michiganpublictech/ai-token-saver.md
- **Developed by:** [Justin Hagerty](https://apify.com/michiganpublictech) (community)
- **Categories:** AI, Developer tools
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 operations

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## AI Token Saver — Deterministic Tools for AI Agents

**Stop paying LLM prices for work that ordinary software does perfectly.** AI Token Saver gives your agent
80+ fast, deterministic operations behind one tool: HTML → Markdown, main-content extraction, RAG chunking,
token counting, PII redaction with reversible placeholders, JSON repair and schema validation, CSV/JSON/YAML/XML
conversion, SQL-like table queries, fuzzy matching, safe math, timezone and cron maths, validation of emails,
IBANs and card numbers — and more. Every result reports an **estimate of the LLM tokens and dollars it saved**,
so you can see the ROI per call.

- **Built for agents:** one tool with a precise schema that works through the [Apify MCP server](#use-it-from-an-ai-agent-mcp),
  the Apify API, LangChain/LlamaIndex/CrewAI, n8n, Make and Zapier. Lean output: no envelopes, nulls dropped.
- **Deterministic and exact:** the same input always gives the same output. No hallucinated links, dropped rows or
  arithmetic slips.
- **Low latency:** supports [Actor Standby](#standby-mode-low-latency-http) — a warm HTTP endpoint instead of a
  container start per call.
- **Safe on hostile input:** size caps, ReDoS-guarded regexes, no XML DTDs, SSRF-guarded fetching, robots.txt
  respected, prompt-injection warnings on fetched content.
- **Cheap:** $0.001 per operation. 1,000 operations = $1. Failed operations and the catalog calls are free.

### The problem: agents burn tokens on mechanical work

Agents routinely ask a frontier model to clean HTML, count or chunk text, pull emails and dates out of a page,
fix broken JSON or add up a column. Each of those calls pays for the model to **read the whole input** and then
**write the whole result** at output-token prices — and sometimes gets it subtly wrong.

#### Cost comparison (estimates)

Assumptions: an LLM priced at **$3 / $15 per million input / output tokens** (a typical frontier model; set your own with
`llmPricing`), ~150 tokens of instructions per prompt, and the LLM writing the same result we return. "This Actor"
is our $0.001 price plus your agent reading our (compact) result back into its context. Sizes are typical, not
measured on your data. Formula: see [Savings methodology](#savings-methodology).

| Task | Operation | Typical size (tokens) | LLM tokens | LLM cost | This Actor | Cheaper by |
|---|---|---|---|---|---|---|
| Convert a 60 KB web page to Markdown | `html_to_markdown` | ~15,000 in / ~2,500 out | ~17,650 | $0.0829 | $0.0085 | 9.8× |
| Extract the main article from a news page | `extract_main_content` | ~20,000 in / ~1,500 out | ~21,750 | $0.0844 | $0.0055 | 15.4× |
| Chunk a 20-page document for RAG (500-token chunks) | `chunk_text` | ~10,000 in / ~10,500 out | ~20,750 | $0.1895 | $0.0325 | 5.8× |
| Redact PII in a 2,000-word support ticket | `redact_pii` | ~2,700 in / ~2,900 out | ~5,750 | $0.0520 | $0.0097 | 5.4× |
| Repair malformed JSON from an LLM reply | `repair_json` | ~800 in / ~700 out | ~1,650 | $0.0134 | $0.0031 | 4.3× |
| Validate a JSON object against a schema | `validate_json_schema` | ~1,200 in / ~60 out | ~1,710 | $0.0095 | $0.0012 | 8.0× |
| Group-by + aggregate 500 CSV rows | `table_query` | ~12,000 in / ~300 out | ~12,750 | $0.0455 | $0.0019 | 23.9× |
| Next 10 runs of a cron expression | `cron_next_runs` | ~20 in / ~400 out | ~1,170 | $0.0155 | $0.0022 | 7.1× |
| Convert a 200-row CSV to JSON | `convert_format` | ~4,000 in / ~7,000 out | ~11,150 | $0.1174 | $0.0220 | 5.3× |
| Count tokens / check a context budget | `count_tokens` | ~8,000 in / ~30 out | ~8,280 | $0.0264 | $0.0011 | 24.2× |
| Compress scraped context before prompting | `compress_context` | ~6,000 in / ~3,500 out | ~9,750 | $0.0725 | $0.0115 | 6.3× |
| Extract all dates as ISO 8601 from an email thread | `extract_dates` | ~3,000 in / ~250 out | ~3,550 | $0.0155 | $0.0018 | 8.8× |

These are **estimates** with stated assumptions, not quotes: with a cheaper model the gap shrinks; with a
reasoning model (which also bills thinking tokens) it grows. Tasks with tiny inputs can cost *more* here than in an
LLM — the per-item `usd_saved_est` then goes negative, and we report it honestly. The bigger, unmeasured win is
correctness: exact counts, valid JSON, real links, verified checksums.

### Who it's for

- **Agent builders** (Claude, GPT, Gemini agents, MCP clients) who want their model to orchestrate, not to type.
- **RAG pipelines** that need clean Markdown and well-formed chunks with offsets for citations.
- **Automation teams** (n8n, Make, Zapier) that need text and data plumbing without writing code.
- **Privacy-conscious teams** that must strip personal data before text reaches an LLM provider, and restore it after.

### Use it from an AI agent (MCP)

The [Apify MCP server](https://docs.apify.com/platform/integrations/mcp) exposes this Actor as a tool; its input
schema becomes the tool definition, and the dataset items come back as the tool result.

**Remote (Claude, Cursor, VS Code and other clients that support remote MCP):**

```json
{
  "mcpServers": {
    "ai-token-saver": {
      "url": "https://mcp.apify.com?tools=michiganpublictech/ai-token-saver",
      "headers": { "Authorization": "Bearer YOUR_APIFY_TOKEN" }
    }
  }
}
```

(OAuth is also supported by mcp.apify.com, in which case the `headers` block can be omitted.)

**Local stdio (Claude Desktop `claude_desktop_config.json`, Cursor `~/.cursor/mcp.json`):**

```json
{
  "mcpServers": {
    "ai-token-saver": {
      "command": "npx",
      "args": ["-y", "@apify/actors-mcp-server", "--tools", "michiganpublictech/ai-token-saver"],
      "env": { "APIFY_TOKEN": "YOUR_APIFY_TOKEN" }
    }
  }
}
```

Tip for the system prompt: *"For mechanical text/data work (format conversion, counting, chunking, extraction,
redaction, maths, dates, validation) call the ai-token-saver tool instead of doing it yourself. Call it with
`{"operation": "list_operations"}` (free) if unsure which operation fits."*

#### OpenAI / Anthropic tool use via the Apify API

Define one tool whose parameters mirror the input (`operation`, `input`, `url`, `data`, `options`, `pipeline`)
and execute it with the synchronous run endpoint, which returns the dataset items directly:

```python
import requests

def ai_token_saver(args: dict) -> list[dict]:
    r = requests.post(
        "https://api.apify.com/v2/acts/michiganpublictech~ai-token-saver/run-sync-get-dataset-items",
        params={"token": APIFY_TOKEN},
        json=args,          # e.g. {"operation": "repair_json", "input": "{'a': 1,}"}
        timeout=120,
    )
    r.raise_for_status()
    return r.json()         # [{"operation": "repair_json", "ok": true, "result": {...}, "stats": {...}}]
```

#### LangChain / LlamaIndex / CrewAI (apify-client)

```python
from apify_client import ApifyClient

client = ApifyClient(APIFY_TOKEN)

def run_tool(operation: str, **request) -> list[dict]:
    run = client.actor("michiganpublictech/ai-token-saver").call(run_input={"operation": operation, **request})
    return list(client.dataset(run["defaultDatasetId"]).iterate_items())

chunks = run_tool("chunk_text", input=long_text, options={"maxTokens": 400, "overlapTokens": 40})
```

Wrap `run_tool` as a `StructuredTool` (LangChain), a `FunctionTool` (LlamaIndex) or a CrewAI `@tool`.
LangChain's `ApifyActorsTool("michiganpublictech/ai-token-saver")` also works out of the box.

#### n8n, Make, Zapier

Use the Apify app/node → **Run Actor (synchronously)** → `michiganpublictech/ai-token-saver`, pass the JSON input
(for example `{"operation": "extract_emails", "input": "{{ $json.body }}"}`) and read the dataset items.
Batch mode (`inputs`) processes many rows in one run.

### Standby mode (low-latency HTTP)

The Actor supports [Actor Standby](https://docs.apify.com/platform/actors/running/standby): Apify keeps a warm
instance behind an HTTPS endpoint, so calls skip container start-up. Charging is identical to normal runs.

```bash
BASE=https://michiganpublictech--ai-token-saver.apify.actor

## Catalog and per-operation help (free)
curl -s "$BASE/operations?family=pii" -H "Authorization: Bearer $APIFY_TOKEN"
curl -s "$BASE/operations/chunk_text" -H "Authorization: Bearer $APIFY_TOKEN"

## Run: same JSON shape as the Actor input
curl -s -X POST "$BASE/run" -H "Authorization: Bearer $APIFY_TOKEN" -H "Content-Type: application/json" \
  -d '{"operation": "redact_pii", "input": "Mail jane@acme.test or call +1 415 555 0132"}'
## → {"items":[{"operation":"redact_pii","ok":true,"result":{"text":"Mail [EMAIL_1] or call [PHONE_1]", ...}}]}
```

Endpoints: `GET /` (health/help), `GET /operations`, `GET /operations/<name>`, `POST /run` (body: `operation` +
`input`/`url`/`data` + `options`, or `inputs`, or `pipeline`, plus optional `includeStats`, `compactOutput`,
`llmPricing`, `respectRobotsTxt`, `pipelineOutput`). Request bodies are limited to 8 MB. In Standby, results are
returned in the HTTP response only — nothing is written to a dataset.

### Operation catalog

**86 operations in 9 families.** Full option reference with worked examples: [OPERATIONS.md](OPERATIONS.md). Agents can also call `list_operations` / `explain_operation` (free).

##### Web & HTML

| Operation | Input | What it does |
|---|---|---|
| [`extract_headings`](OPERATIONS.md#extract-headings) | html | Build the heading outline (h1–h6) of an HTML page as a nested tree or a flat list — useful to understand a document's structure cheaply. |
| [`extract_images`](OPERATIONS.md#extract-images) | html | List images in HTML with absolute src, alt text, title and declared size; skips tracking pixels and data: URIs by default. |
| [`extract_links`](OPERATIONS.md#extract-links) | html | List the hyperlinks in HTML as absolute URLs with anchor text, flagged internal/external, deduplicated and with tracking parameters (utm\_\*, fbclid, gclid...) removed. |
| [`extract_main_content`](OPERATIONS.md#extract-main-content) | html | Readability-style extraction of a page's main article: returns title and clean Markdown of the body, without menus, sidebars, ads, comments or footers. |
| [`extract_metadata`](OPERATIONS.md#extract-metadata) | html | Extract page metadata from HTML: title, description, canonical URL, language, OpenGraph, Twitter card, parsed JSON-LD (schema.org), favicon, robots and author/date meta tags. |
| [`extract_tables`](OPERATIONS.md#extract-tables) | html | Extract HTML tables as structured data: header row plus rows as objects (column → value), optionally CSV; colspans expanded, nested tables handled separately. |
| [`fetch_url`](OPERATIONS.md#fetch-url) | url | Fetch a public web page (SSRF-guarded, robots.txt-aware) and return status, final URL, content type and body text — the usual first step of a pipeline. |
| [`html_to_markdown`](OPERATIONS.md#html-to-markdown) | html | Convert HTML to clean GitHub-flavoured Markdown (headings, lists, links, emphasis, code, tables), dropping scripts, styles, nav, footers, forms and cookie banners. |
| [`html_to_text`](OPERATIONS.md#html-to-text) | html | Convert HTML to readable plain text with paragraph breaks (no markup), dropping scripts, styles and page boilerplate. |

##### Text & tokens

| Operation | Input | What it does |
|---|---|---|
| [`change_case`](OPERATIONS.md#change-case) | text | Convert identifiers or phrases between snake_case, camelCase, PascalCase, kebab-case, CONSTANT_CASE, dot.case, Title Case and Sentence case. |
| [`chunk_text`](OPERATIONS.md#chunk-text) | text | Split text into RAG-ready chunks by tokens (with overlap), sentences, paragraphs, Markdown headings (keeps heading path) or fixed characters; returns offsets and token estimates. |
| [`classify_by_rules`](OPERATIONS.md#classify-by-rules) | text | Classify text into your labels using keyword / regex rules (labels: {label: \[keywords or 're:<regex>']}); returns per-label scores, best label and matched terms. |
| [`clean_text`](OPERATIONS.md#clean-text) | text | Clean messy text: Unicode NFKC, fix mojibake (’ → ’), remove zero-width/control characters, re-join hyphenated line breaks, collapse whitespace, optional smart-quote → ASCII. |
| [`compress_context`](OPERATIONS.md#compress-context) | text | Shrink text/Markdown/JSON before putting it in an LLM prompt: collapse whitespace, minify JSON, drop duplicate lines/paragraphs and cookie/nav boilerplate, strip tracking params, optionally abbreviate repeated URLs; reports tokens before/after. |
| [`count_tokens`](OPERATIONS.md#count-tokens) | text | Estimate how many LLM tokens a text uses (gpt/claude/llama/generic heuristics, ±10–15%) and whether it fits a token budget — without calling a model. |
| [`deduplicate`](OPERATIONS.md#deduplicate) | text | Remove duplicate lines, paragraphs, sentences or list items (exact or normalized: case, whitespace, punctuation), keeping first occurrences in order. |
| [`detect_language`](OPERATIONS.md#detect-language) | text | Detect the language of text (35+ languages: script detection + stopword and character-trigram profiles); returns ISO 639-1 code, name, confidence and top candidates. |
| [`diff_text`](OPERATIONS.md#diff-text) | text | Compare two texts: unified diff, similarity ratio (0–1) and counts of added/removed lines; optional word-level diff. |
| [`extractive_summary`](OPERATIONS.md#extractive-summary) | text | Summarize text by selecting its most central sentences (TextRank); returns them in original order. Extractive only — never invents text. |
| [`keywords`](OPERATIONS.md#keywords) | text | Extract ranked key phrases from text (RAKE combined with TF-IDF over sentences) — no LLM needed for tags, topics or SEO keywords. |
| [`near_duplicates`](OPERATIONS.md#near-duplicates) | text | Cluster near-duplicate texts (MinHash/LSH Jaccard or SimHash) in a list or in the paragraphs of a document — find reposts, boilerplate and template variants. |
| [`normalize_whitespace`](OPERATIONS.md#normalize-whitespace) | text | Normalize whitespace: LF line endings, tabs → spaces, collapse spaces, trim lines, limit consecutive blank lines. |
| [`readability`](OPERATIONS.md#readability) | text | Readability scores for English text: Flesch reading ease, Flesch-Kincaid grade, Gunning fog, SMOG, Coleman-Liau, ARI, with a plain-English verdict. |
| [`regex_extract`](OPERATIONS.md#regex-extract) | text | Run a (ReDoS-guarded) regular expression over text and return matches, capture groups and named groups — reliable where LLMs miscount or hallucinate matches. |
| [`sentiment`](OPERATIONS.md#sentiment) | text | English sentiment (VADER-style lexicon with negation, intensifiers, caps, '!' and 'but' clauses): compound score −1…1, label and pos/neg/neu shares. |
| [`slugify`](OPERATIONS.md#slugify) | text | Turn a title into a URL slug (ASCII-folded, lowercase, separator, max length on word boundary). |
| [`split_sentences`](OPERATIONS.md#split-sentences) | text | Split text into sentences (abbreviation-, decimal- and initials-aware; handles CJK full stops). |
| [`strip_markdown`](OPERATIONS.md#strip-markdown) | text | Convert Markdown to plain text: remove headings marks, emphasis, links (keep text), images, code fences, tables rules, footnotes and HTML tags. |
| [`text_stats`](OPERATIONS.md#text-stats) | text | Count characters, words, unique words, sentences, paragraphs and lines; average lengths, reading/speaking time and token estimates per model family. |
| [`truncate_to_tokens`](OPERATIONS.md#truncate-to-tokens) | text | Cut text to fit a token budget at sentence or word boundaries, keeping the head, the tail, or both ends (middle-out) with an omission marker. |

##### Extraction

| Operation | Input | What it does |
|---|---|---|
| [`extract_code_blocks`](OPERATIONS.md#extract-code-blocks) | text | Extract fenced code blocks (`` or ~~~) from Markdown/LLM output with their language tags. |
| [`extract_dates`](OPERATIONS.md#extract-dates) | text | Find dates and times in text (ISO, '5 March 2026', '03/04/2026', 'next Friday', '3 days ago') and return ISO 8601 values with spans. |
| [`extract_emails`](OPERATIONS.md#extract-emails) | text | Find every email address in text (deduplicated); optionally decode 'name [at] site [dot] com'. |
| [`extract_entities`](OPERATIONS.md#extract-entities) | text | Find technical entities: IPs, domains, UUIDs, #hashtags, @mentions, file paths, semver, ISBN, IBAN, MACs, crypto-like addresses. |
| [`extract_json`](OPERATIONS.md#extract-json) | text | Pull JSON objects/arrays out of LLM output or free text (``json fences, inline), repairing broken JSON. |
| [`extract_key_values`](OPERATIONS.md#extract-key-values) | text | Turn 'Key: Value' / 'Key = Value' lines into a JSON object (duplicate keys become lists). |
| [`extract_money`](OPERATIONS.md#extract-money) | text | Find monetary amounts ($1,234.50, 1.234,56 €, USD 5bn, 20 euros) as amount + ISO currency. |
| [`extract_numbers`](OPERATIONS.md#extract-numbers) | text | Find numbers in text (integers, decimals, 1,234 separators, negatives, %, 1e-3, units like km/GB). |
| [`extract_phone_numbers`](OPERATIONS.md#extract-phone-numbers) | text | Find phone numbers in text and normalize them to E.164 (+14155550123) using a default region. |
| [`extract_urls`](OPERATIONS.md#extract-urls) | text | Find every http(s)/ftp/www URL in text, trimming trailing punctuation; optionally strip tracking params. |

##### PII & safety

| Operation | Input | What it does |
|---|---|---|
| [`detect_secrets`](OPERATIONS.md#detect-secrets) | text | Find leaked credentials (API keys, tokens, private keys, passwords in URLs); reports redacted previews only. |
| [`redact_pii`](OPERATIONS.md#redact-pii) | text | Redact PII (emails, phones, cards, SSNs, IBANs, IPs, addresses, birth dates, secrets) before sending text to an LLM; placeholder mode returns a mapping for restore_pii. |
| [`restore_pii`](OPERATIONS.md#restore-pii) | text | Put original values back into text redacted by redact_pii (placeholder mode) using its mapping. |
| [`scan_prompt_injection`](OPERATIONS.md#scan-prompt-injection) | text | Scan untrusted text/HTML (web pages, emails, tool output) for prompt-injection attempts before an LLM reads it; returns risk none/low/medium/high with markers. |

##### JSON & data

| Operation | Input | What it does |
|---|---|---|
| [`coerce_to_schema`](OPERATIONS.md#coerce-to-schema) | data | Cast JSON data to match a JSON Schema: numeric/boolean strings → numbers/booleans, dates → ISO, null-likes → null, fill defaults, optionally drop unknown keys; reports every change and remaining errors. |
| [`convert_format`](OPERATIONS.md#convert-format) | text | Convert data between JSON, CSV, TSV, YAML, JSONL, XML and Markdown tables (auto-detects the source format; XML/YAML parsed safely) — no LLM re-typing of tables. |
| [`dedupe_records`](OPERATIONS.md#dedupe-records) | data | Remove duplicate records — exact (normalized case/whitespace) or fuzzy (rapidfuzz similarity ≥ threshold) on chosen key fields; returns kept records and duplicate clusters. |
| [`diff_json`](OPERATIONS.md#diff-json) | data | Structural diff of two JSON documents: added, removed and changed paths with old/new values. |
| [`flatten_json`](OPERATIONS.md#flatten-json) | data | Flatten nested JSON into a single-level object with path keys (a.b\[0].c) — ideal for CSV export or comparing records. |
| [`fuzzy_match`](OPERATIONS.md#fuzzy-match) | text | Find the closest matches for a query string among choices (typo-tolerant, rapidfuzz scorers) — e.g. map a messy company/product name to a canonical list. |
| [`infer_json_schema`](OPERATIONS.md#infer-json-schema) | data | Infer a JSON Schema (Draft 2020-12) from example data: merges types across array items, marks keys present everywhere as required and detects formats (email, date-time, date, uri, uuid, ipv4). |
| [`json_query`](OPERATIONS.md#json-query) | data | Select values from JSON with a safe JSONPath subset ($.a.b\[\*], $..price, slices, filters like \[?(@.price > 10 && @.tags contains 'sale')]) — instead of an LLM reading the whole document. |
| [`json_to_pydantic`](OPERATIONS.md#json-to-pydantic) | data | Generate Pydantic v2 models from example JSON (nested models, Optional fields, aliases for non-Python keys, datetime/date/UUID detection). |
| [`json_to_typescript`](OPERATIONS.md#json-to-typescript) | data | Generate TypeScript interfaces from example JSON (nested interfaces, optional fields, unions) — saves writing boilerplate types by hand or by LLM. |
| [`merge_json`](OPERATIONS.md#merge-json) | data | Deep-merge JSON objects (later wins) with a list strategy: replace, concat, union (dedupe) or by_index. |
| [`normalize_records`](OPERATIONS.md#normalize-records) | data | Clean a list of records: trim strings, collapse whitespace, turn null-likes ('', 'N/A', '-') into null, and cast fields per spec (number, integer, boolean, date, lower, title, digits, list...). |
| [`pivot`](OPERATIONS.md#pivot) | data | Pivot records into a cross-tab: one row per `rows` value, one column per `columns` value, cells aggregated (sum/count/avg/min/max) from `values`. |
| [`repair_json`](OPERATIONS.md#repair-json) | text | Fix broken JSON from LLMs or logs (code fences, trailing commas, single quotes, comments, unquoted keys, Python literals, truncation) and return the parsed value plus what was fixed. |
| [`sample_records`](OPERATIONS.md#sample-records) | data | Deterministic sample of a list (seeded random, systematic, first/last, or stratified by a field) — show an LLM 20 representative rows instead of 20,000. |
| [`statistics`](OPERATIONS.md#statistics) | data | Descriptive statistics for a numeric list or a record field: count, sum, mean, median, mode, stdev, variance, min/max/range, percentiles and a histogram — exact, no LLM arithmetic. |
| [`table_query`](OPERATIONS.md#table-query) | data | SQL-like query over a list of records without an LLM: where filters (and/or/not), computed fields, group by with count/sum/avg/min/max/median/p90/distinct, having, order by, limit/offset, select. |
| [`unflatten_json`](OPERATIONS.md#unflatten-json) | data | Rebuild nested JSON from a flat object whose keys are paths (a.b\[0].c or a.b.0.c). |
| [`validate_json_schema`](OPERATIONS.md#validate-json-schema) | data | Validate JSON data against a JSON Schema (Draft 2020-12, formats checked) and list every error with its JSONPath — no LLM guesswork about whether an object is valid. |

##### Math & units

| Operation | Input | What it does |
|---|---|---|
| [`compound_interest`](OPERATIONS.md#compound-interest) | none | Future value of an investment with compound interest and optional periodic contributions (yearly schedule optional). |
| [`convert_units`](OPERATIONS.md#convert-units) | text | Convert between units (length, mass, volume, temperature, area, speed, data size SI/IEC, time, energy, pressure, power), e.g. '5 km to mi'. |
| [`loan_payment`](OPERATIONS.md#loan-payment) | none | Monthly payment, total paid and total interest for an amortizing loan; optional amortization schedule. |
| [`math_eval`](OPERATIONS.md#math-eval) | text | Evaluate an arithmetic expression exactly and safely (+ - \* / // % ^, sqrt, log, sin, round, min/max…; variables; Decimal mode for money). |
| [`percent_change`](OPERATIONS.md#percent-change) | none | Percent change, absolute change and ratio between two numbers (from → to). |
| [`round_numbers`](OPERATIONS.md#round-numbers) | text | Round every number in text or JSON data to N decimals or significant figures (half-up, half-even, floor, ceil, truncate). |

##### Date & time

| Operation | Input | What it does |
|---|---|---|
| [`convert_timezone`](OPERATIONS.md#convert-timezone) | text | Convert a date/time to another time zone (IANA names, DST-aware); reports offset, abbreviation and DST. |
| [`cron_explain`](OPERATIONS.md#cron-explain) | text | Explain a 5- or 6-field cron expression (or @daily-style macro) in plain English, with normalized field values. |
| [`cron_next_runs`](OPERATIONS.md#cron-next-runs) | text | List the next N fire times of a cron expression in a time zone (DST-aware, DOM/DOW OR rule). |
| [`date_add`](OPERATIONS.md#date-add) | text | Add/subtract calendar units (years, months, days, hours…) or business days to a date/time. |
| [`date_diff`](OPERATIONS.md#date-diff) | text | Difference between two dates/times: calendar breakdown (years/months/days), totals, and business days. |
| [`duration_parse_format`](OPERATIONS.md#duration-parse-format) | text | Parse a duration ('1h30m', '90 minutes', 'PT1H30M', '01:30:00', seconds) and return seconds, ISO 8601, human and clock forms. |
| [`format_datetime`](OPERATIONS.md#format-datetime) | text | Format a date/time as ISO, RFC 2822, unix, human-readable, or any strftime pattern, optionally in a time zone. |
| [`parse_datetime`](OPERATIONS.md#parse-datetime) | text | Parse almost any date/time string (ISO, RFC 2822, '5 March 2026 3pm', unix, 'next Friday') into ISO 8601. |
| [`unix_time`](OPERATIONS.md#unix-time) | text | Convert unix timestamps (s/ms/µs auto-detected) to ISO dates, or dates to unix time. |

##### Validation

| Operation | Input | What it does |
|---|---|---|
| [`semver_compare`](OPERATIONS.md#semver-compare) | text | Compare two semantic versions (incl. pre-release precedence) or sort a list of versions. |
| [`validate`](OPERATIONS.md#validate) | text | Validate a value offline: email, url, domain, ipv4/ipv6/cidr, credit_card (Luhn+brand), iban (mod-97), isbn, uuid, semver, phone (E.164), json, date, us_zip, vat_eu_format. |

##### Encoding & utilities

| Operation | Input | What it does |
|---|---|---|
| [`encode_decode`](OPERATIONS.md#encode-decode) | text | Encode or decode text: base64, base64url, URL/percent-encoding, HTML entities, hex, unicode escapes, ROT13, punycode (IDN). |
| [`explain_operation`](OPERATIONS.md#explain-operation) | text | FREE. Detailed help for one operation: description, input kind, options with defaults, and a worked example with its real output. |
| [`generate_uuid`](OPERATIONS.md#generate-uuid) | none | Generate random UUIDs (v4) or time-ordered UUIDs (v7). Non-deterministic by design. |
| [`hash_text`](OPERATIONS.md#hash-text) | text | Hash text with md5/sha1/sha256/sha512/sha3_256/blake2b/blake2s/crc32 (hex or base64), or an HMAC with a key — LLMs cannot compute hashes. |
| [`list_operations`](OPERATIONS.md#list-operations) | none | FREE. List every available operation with a one-line summary (optionally filtered by family or with option specs). Call this first to pick the right tool. |
| [`random_sample`](OPERATIONS.md#random-sample) | data | Pick a random sample (or shuffle) from a JSON list or lines of text; reproducible with a seed (the seed used is always returned). |

### Pipelines and batch

**Single:** `operation` + one of `input` (text), `url`, `data` (JSON) + `options`.
**Batch:** `operation` + `inputs: [...]` — the same operation and options for every item (each item charged; max 10,000).
**Pipeline:** `pipeline: [{operation, options?, input?}, ...]` — each step receives the previous step's result.
References inside a step's `input` or `options`: `$input` (the run's input/url/data), `$prev` (previous result),
`$steps[N]` and paths such as `$steps[0].markdown`, `$prev.chunks[0].text`, `$steps[1]['some key']`
(write `$$` for a literal leading `$`). One `operation` charge per executed step; with
`pipelineOutput: "last"` (default) you get one item with the final result and the whole pipeline's savings.
Result objects are unwrapped sensibly between steps: a text step receiving `{"markdown": ...}` uses the Markdown.

#### Recipe 1 — web page → RAG chunks

```json
{
  "url": "https://example.com/blog/post",
  "pipeline": [
    {"operation": "fetch_url"},
    {"operation": "extract_main_content"},
    {"operation": "chunk_text", "options": {"strategy": "markdown", "maxTokens": 400, "overlapTokens": 40}}
  ]
}
```

Chunks carry `start`/`end` offsets (exact slices of the extracted Markdown) and their heading path.

#### Recipe 2 — scrape → PII-safe input for summarisation

```json
{
  "url": "https://example.com/support/thread/123",
  "pipeline": [
    {"operation": "fetch_url", "options": {"convert": "main"}},
    {"operation": "redact_pii", "options": {"mode": "placeholder"}},
    {"operation": "compress_context"}
  ]
}
```

Send the redacted text to your LLM, then put the real values back with
`{"operation": "restore_pii", "input": "<LLM answer>", "options": {"mapping": <mapping from redact_pii>}}`
(use `pipelineOutput: "all"` to get the mapping item). Fetched pages containing prompt-injection markers come back
with an `ATS-WEB-002` warning.

#### Recipe 3 — messy LLM output → validated object

````json
{
  "input": "Sure! Here is the data:\n```json\n{'name': 'Ada', 'age': '36', tags: ['x',],}\n```",
  "pipeline": [
    {"operation": "extract_json"},
    {"operation": "repair_json", "input": "$prev.values[0].value"},
    {"operation": "coerce_to_schema", "input": "$prev.data", "options": {"schema": {
      "type": "object", "required": ["name", "age"],
      "properties": {"name": {"type": "string"}, "age": {"type": "integer"}, "tags": {"type": "array"}}}}}
  ]
}
````

#### Recipe 4 — CSV analytics without an LLM

```json
{
  "input": "region,product,revenue\nEU,A,120\nEU,B,80\nUS,A,200\nUS,B,50\nUS,A,30",
  "pipeline": [
    {"operation": "convert_format", "options": {"from": "csv", "to": "json"}},
    {"operation": "table_query", "options": {
      "where": "revenue >= 50",
      "groupBy": ["region"],
      "aggregates": {"orders": "count", "revenue": "sum(revenue)", "p90": "p90(revenue)"},
      "orderBy": ["-revenue"]}}
  ]
}
```

#### Recipe 5 — prompt context compression

```json
{
  "operation": "compress_context",
  "input": "<several scraped pages, logs or JSON blobs>",
  "options": {"abbreviateUrls": true, "dropImages": true}
}
```

Returns the compacted text with `tokens_before`, `tokens_after`, `saved_pct` and the list of transformations
applied (all meaning-preserving unless you opt into `lossy`).

### Input

| Field | Type | Description |
|---|---|---|
| `operation` | string | Operation name (see catalog). Leave empty for pipelines. |
| `input` | string | Text / HTML / Markdown / JSON text to process (max 5 MB per operation). |
| `url` | string | URL for `fetch_url` or the first pipeline step. |
| `data` | object | JSON input for data operations. Wrap arrays as `{"items": [...]}` (or send JSON text in `input`). |
| `options` | object | Operation options; unknown options are rejected with a "did you mean" hint. |
| `inputs` | array | Batch mode: one item per input. |
| `pipeline` | array | Pipeline mode: steps `{operation, options, input}`. |
| `pipelineOutput` | `last` | `all` | One item per pipeline (default) or one per step. |
| `includeStats` | boolean | Add token-savings stats to each item (default true). |
| `compactOutput` | boolean | Drop null fields (default true). |
| `llmPricing` | object | `{"inputPerMTok": 3, "outputPerMTok": 15}` — the model you'd otherwise use, for `usd_saved_est`. |
| `respectRobotsTxt` | boolean | `fetch_url` obeys robots.txt (default true). |
| `failOnSeverity`, `disabledRules`, `licenseKey` | | Shared "Licensing & CI" fields. `disabledRules` suppresses ATS warnings (Pro plan and above). |

Exactly one mode is allowed per run: single, batch or pipeline. Structural mistakes (unknown operation, mixed
modes) fail the run with a clear message; problems with one item's options or content produce an `ok: false`
item instead and are not charged.

### Output

One lean dataset item per operation (or per pipeline):

```json
{
  "operation": "html_to_markdown",
  "ok": true,
  "result": {"title": "Pricing | Acme", "markdown": "# Acme Pricing\n\nPlans start at **$9/month**…"},
  "warnings": ["ATS-PII-001 Personal data present in the result (types: email). Add a redact_pii step…"],
  "stats": {"input_tokens_est": 164, "output_tokens_est": 71, "llm_tokens_est": 385,
            "tokens_saved_est": 314, "usd_saved_est": 0.000794, "ms": 1},
  "index": 0
}
```

- `ok: false` items carry `error` (and for pipelines `step`, `last_ok_step`, `last_ok_result`) and are never charged.
- `warnings` (only when present) are short ATS rule hints — see [RULES.md](RULES.md).
- `index` appears in batch mode, `step`/`pipeline` in pipeline mode.
- Key-value store: `OUTPUT` (run summary, billing, notes) and `SAVINGS` (totals: operations, tokens and USD saved,
  per-operation breakdown, the pricing used).

### Savings methodology

For each operation we estimate what an LLM would spend to produce the same result:

```
llm_tokens  = overhead + input_tokens × input_factor            (prompt: instructions + reading the input)
            + output_tokens × output_factor + reasoning          (completion: writing the result, thinking)
llm_usd     = prompt × inputPerMTok / 1e6 + completion × outputPerMTok / 1e6
ours_usd    = output_tokens × inputPerMTok / 1e6 + our price    (your agent still reads our result)
tokens_saved_est = llm_tokens − output_tokens
usd_saved_est    = llm_usd − ours_usd                             (can be negative for tiny jobs)
```

`overhead` is 150 tokens; `reasoning` is 0 for pure transformations, ~50–150 for lookups/extraction and 300–600 for
computations (maths, dates, cron, checksums, aggregation) where a model must work step by step to be reliable.
Each operation's profile is fixed in code and listed by `explain_operation`. For pipelines the LLM side is summed
over steps. Token counts come from the offline estimator below. **All of this is an estimate** meant to show order of
magnitude, not an invoice.

**Token estimation.** We don't download tokenizer vocabularies at runtime. `count_tokens` and all stats use a
documented heuristic (word length, digit groups, punctuation, whitespace, CJK and other non-ASCII scripts) with
per-family scaling for `gpt`, `claude`, `llama` and `generic`. Expect ±10–15 % on English prose and more on unusual
text; use the provider's tokenizer when you need billing-grade counts.

### Pricing

Pay per event — you only pay for successful work:

| Event | Price | When |
|---|---|---|
| `operation` | $0.001 ($1 per 1,000) | Each successful operation or executed pipeline step. |
| `page-fetch` | $0.001 ($1 per 1,000) | Each page fetched successfully by `fetch_url` (status < 400), on top of the operation. |

Free: `list_operations`, `explain_operation`, and every failed operation. Example: 1,000 HTML→Markdown conversions =
$1; fetching and converting 1,000 pages in one step (`fetch_url` with `convert`) = $2. Set a
**maximum cost per run** in Apify (spending limit): the Actor checks the remaining budget *before* each operation and
stops gracefully, so you are never charged beyond it. Standard Apify platform usage applies to runs.

### Limits and safety

- **Sizes:** 5 MB input per operation by default (lower for heavy operations, e.g. 1 MB for regex, diffs and
  summaries); results above ~8 MB are refused with a hint to chunk. Inputs over a cap are truncated *with an
  `ATS-INPUT-001` warning*, never silently. Batch ≤ 10,000 items, pipelines ≤ 25 steps.
- **Regex:** user patterns are limited to 500 characters and screened for catastrophic backtracking (nested or
  overlapping quantifiers, backreferences) before running — `ATS-REGEX-001`.
- **XML/YAML:** DOCTYPE/ENTITY declarations are rejected (billion laughs, XXE) — `ATS-XML-001`; YAML alias bombs
  are refused.
- **Expressions:** `math_eval` and `table_query` computed fields use an AST whitelist evaluator — no `eval`, no
  attribute access, capped exponents, digits and nesting.
- **Network:** only `fetch_url` touches the network, through an SSRF-guarded client (no private, loopback,
  link-local or cloud-metadata addresses; redirects re-validated; size and time caps; honest User-Agent
  `ai-token-saver/<version>`). robots.txt is respected by default (RFC 9309) — `ATS-WEB-001`.
- **Prompt injection:** fetched pages are scanned for instruction-like text aimed at agents; hits are reported as
  `ATS-WEB-002` so your agent treats the content as data. `scan_prompt_injection` gives a full report for any text.
- **Secrets:** detected credentials are never echoed (only redacted previews), and are never stored in
  `redact_pii` mappings.

### Privacy

Your inputs are processed in memory inside your own Apify run. Results go to that run's default dataset and
key-value store (in Standby: only to the HTTP response), which you control and can delete; we don't store them
anywhere else and we don't send them to any LLM or third-party service. Anonymous usage counters (operation counts,
no content) may be reported for licensing. If personal data appears in a non-redaction result, the item carries an
`ATS-PII-001` hint so you can add a `redact_pii` step before the text reaches an LLM provider.

### Licensing and plans

Store users need nothing beyond an Apify account: pay-per-event pricing applies. Enterprise and prepaid licenses
(waived platform charges, custom rules, volume terms) are described in [docs/licensing.md](../../docs/licensing.md).

### Limitations

- Token counts and savings are **estimates** (see methodology).
- `sentiment` is English-only and lexicon-based (VADER-style): it misses sarcasm and domain jargon.
- `extractive_summary` selects existing sentences (TextRank); it does not paraphrase. `keywords` is statistical.
- `detect_language` is reliable for a sentence or more; very short strings are flagged `reliable: false`.
- `extract_main_content` is heuristic: unusual layouts may need `html_to_markdown` with `stripBoilerplate`.
- `fetch_url` does not run JavaScript; for client-rendered pages use a browser-based Apify scraper and pipe its
  HTML into this Actor.
- Phone/address detection is best-effort; validation checks formats and checksums, not existence (no DNS/SMTP).

### FAQ

**Why not just ask the model?** You can — but you pay for every input and output token, wait for generation, and
accept occasional mistakes. Deterministic code is cheaper, faster and exact for these tasks.

**How does my agent know which operation to use?** The input schema lists all operations; `list_operations`
(free) returns one-line summaries and `explain_operation` (free) returns options and a worked example.

**Are results reproducible?** Yes, except the explicitly random operations (`generate_uuid`, and `random_sample`
without a `seed` — the seed used is returned so you can replay it).

**Is my data used to train anything?** No. Nothing is sent to an LLM.

**What happens when my spending limit is reached?** The Actor stops before the next paid operation; items already
produced are kept and you are never charged past the limit.

**Can I self-host?** Enterprise licenses can run the same image on your infrastructure — see the licensing doc.

### Roadmap

- Accurate tokenizer counts for selected open models (bundled vocabularies) as an opt-in.
- PDF and DOCX text extraction; sitemap and RSS helpers.
- More languages for sentiment and keywords; locale-aware number and date formatting.
- Per-agent savings dashboards from the `SAVINGS` records.

Have a mechanical task your agent keeps paying an LLM for? Open an issue on the Actor page — operations are cheap
for us to add.

# Changelog

This Actor's version history is a separate document: https://apify.com/michiganpublictech/ai-token-saver/changelog.md

# Actor input Schema

## `operation` (type: `string`):

Operation to run, e.g. html_to_markdown, chunk_text, redact_pii, repair_json, table_query, math_eval, cron_next_runs. Call list_operations (free) for one-line summaries or explain_operation for options. Leave empty when using pipeline.

## `input` (type: `string`):

Text, HTML, Markdown or JSON text the operation works on (single mode; also $input of a pipeline). Max 5 MB.

## `url` (type: `string`):

Public http(s) URL for fetch_url (or the first step of a pipeline). Fetched with SSRF protection; robots.txt respected by default.

## `data` (type: `object`):

JSON input for data operations (validate_json_schema, table_query, convert_format...). To pass an array, wrap it: {"items": \[...]} — or put JSON text in Input text.

## `options` (type: `object`):

Operation options as JSON, e.g. {"maxTokens": 500} for chunk_text. Unknown options are rejected with a hint; see explain_operation.

## `inputs` (type: `array`):

Batch mode: run `operation` with the same options on each item (strings, or JSON values for data operations). Each successful item is charged. Max 10,000.

## `pipeline` (type: `array`):

Pipeline mode: steps \[{"operation": ..., "options": {...}, "input": "$prev"}]. Each step gets the previous result by default; references: $input, $prev, $steps\[0].field. One charge per executed step. Leave Operation empty.

## `pipelineOutput` (type: `string`):

last: one item with the final step's result (default, leanest). all: one item per step.

## `includeStats` (type: `boolean`):

Add stats (input/output token estimates, LLM-equivalent tokens, estimated tokens and USD saved, ms) to each item.

## `compactOutput` (type: `boolean`):

Drop null fields from results to save your agent tokens.

## `llmPricing` (type: `object`):

USD per million tokens of the model your agent would otherwise use: {"inputPerMTok": 3, "outputPerMTok": 15}. Only affects usd_saved_est.

## `respectRobotsTxt` (type: `boolean`):

fetch_url obeys robots.txt (RFC 9309). Disable only for sites you are entitled to fetch.

## `failOnSeverity` (type: `string`):

CI gate shared by our Actors. This Actor emits no scored findings, so it has no effect here.

## `disabledRules` (type: `array`):

ATS rule IDs whose warnings to suppress (see RULES.md). Requires the custom_rules feature (Pro plan and above).

## `licenseKey` (type: `string`):

Optional. Enterprise / prepaid license key. Store users don't need one.

## Actor input object example

```json
{
  "operation": "html_to_markdown",
  "input": "<article><h1>Release notes</h1><p>Version <b>2.0</b> ships <a href='https://example.com/docs?utm_source=mail'>new docs</a>.</p><ul><li>Faster parsing</li><li>Smaller output</li></ul></article>",
  "options": {
    "baseUrl": "https://example.com/"
  },
  "pipelineOutput": "last",
  "includeStats": true,
  "compactOutput": true,
  "llmPricing": {
    "inputPerMTok": 3,
    "outputPerMTok": 15
  },
  "respectRobotsTxt": true,
  "failOnSeverity": "none"
}
```

# Actor output Schema

## `results` (type: `string`):

One item per operation: operation, ok, result or error, warnings and token-savings stats.

## `summary` (type: `string`):

Plan, item count, billing and notes for the run (OUTPUT record).

## `savings` (type: `string`):

Estimated LLM tokens and USD saved by this run, per operation (SAVINGS record).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "operation": "html_to_markdown",
    "input": "<article><h1>Release notes</h1><p>Version <b>2.0</b> ships <a href='https://example.com/docs?utm_source=mail'>new docs</a>.</p><ul><li>Faster parsing</li><li>Smaller output</li></ul></article>",
    "options": {
        "baseUrl": "https://example.com/"
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("michiganpublictech/ai-token-saver").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "operation": "html_to_markdown",
    "input": "<article><h1>Release notes</h1><p>Version <b>2.0</b> ships <a href='https://example.com/docs?utm_source=mail'>new docs</a>.</p><ul><li>Faster parsing</li><li>Smaller output</li></ul></article>",
    "options": { "baseUrl": "https://example.com/" },
}

# Run the Actor and wait for it to finish
run = client.actor("michiganpublictech/ai-token-saver").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "operation": "html_to_markdown",
  "input": "<article><h1>Release notes</h1><p>Version <b>2.0</b> ships <a href='\''https://example.com/docs?utm_source=mail'\''>new docs</a>.</p><ul><li>Faster parsing</li><li>Smaller output</li></ul></article>",
  "options": {
    "baseUrl": "https://example.com/"
  }
}' |
apify call michiganpublictech/ai-token-saver --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,michiganpublictech/ai-token-saver"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/8b0b3qT9f6dFUC6oG/builds/xzjRrqAD3ApcYvMib/openapi.json