# Changelog of PDF, DOCX & XLSX to Text (`humble-echidna/document-to-text`) Actor

- **URL**: https://apify.com/humble-echidna/document-to-text/changelog.md
- **Full Actor documentation**: https://apify.com/humble-echidna/document-to-text.md

## Changelog

Versions follow MAJOR.MINOR.PATCH (`src/version.py`); Apify shows MAJOR.MINOR from `.actor/actor.json`.
Every run logs its version and records it in the `RUN_STATS` key-value record.

### 1.3.1 (2026-09-25)

- Input field descriptions rewritten for AI agents (Apify's MCP server shows agents only the description, not the
  form): each now states its default, its allowed range and how it combines with other fields. No change to field
  names, types, defaults or behaviour.

### 1.3.0 (2026-09-25)

- **PDF text is read with PDFium by default**, 3-20x less compute per PDF than 1.2's pdfplumber (measured locally,
  parser process included: IRS W-9, 6 pages, 0.26 CPU-s against 0.88; IRS Pub 17, 100 pages, 0.84 against 14.0; 1040
  instructions, 100 pages, 0.96 against 20.0). It also reads two-column pages column by column, where pdfplumber
  interleaved the columns line by line, rejoins words hyphenated at line ends, and reads sideways text the right way
  round (pdfplumber spelled the W-9's margin note backwards).
- **PDF tables are now opt-in: `extractTables` defaults to off** (was on). With it on, PDFs are read with pdfplumber as
  in 1.2: ruled tables as rows and as Markdown tables in place. With it off, a PDF table's cells come back as text in
  the PDF's own drawing order, which for dense tables can be cell by cell rather than row by row. Word and Excel
  tables are always returned now, whatever `extractTables` says (they cost nothing extra to read).
- **New input `pages`**: read only some pages of each PDF, e.g. `1-10` or `1-3, 7, 20-`. Max pages per PDF still
  applies within it; a PDF with none of the pages is reported and not charged. A malformed range fails the run
  before anything is downloaded. `pagesExtracted` is now the number of pages read (with no range, the same as
  before).
- **New input `pdfPassword`** (stored encrypted by Apify): opens password-protected PDFs with the password you give.
  PDFium saves a decrypted copy inside the run for parsing and OCR, deleted with the download. A wrong password is
  reported as password-protected and not charged. The password is never logged or written to `RUN_STATS` (only
  whether one was given).
- PDF text from PDFium: the "fi" ligature that some fonts leave unmapped (U+001F, e.g. "\x1fling" in IRS
  publications) is read as "fi", other control characters are dropped, and trailing spaces at line ends are removed.
- **Price: $5.00 per 1,000 documents** (was $20.00), **plus a new event, `table-page` ("Table page"), $0.30 per
  1,000 PDF pages read with `extractTables` on.** OCR pages stay at $10.00 per 1,000, the run start at $0.00005.
  Table pages are charged only once the document's result is in the dataset, only for pages read with table
  detection (not on the text-only fallback, never for Word or Excel), and never for a document that fails. Before a
  document is downloaded with tables on, the most table pages it can need (Max pages per PDF, or a shorter bounded
  `pages` range) are reserved against the maximum cost per run, after the documents in flight; the parser reads no
  more pages than were granted (`truncatedReasons` says when the cost cap cut it), and the unused rest goes back.
  If not even one table page is covered, the run stops there, as it does when a document isn't covered. Measured cost
  and contribution per workload: COSTS.md, "document-to-text 1.3".
- `RUN_STATS`: new `tablePages` and `tablePagesCharged`, next to the OCR counts.
- Limits: a download gets 120 s in total, retries included (was 300 s); a PDF read without tables gets 60 s to parse
  (180 s as before with tables, and for Word, Excel and images; OCR pages keep their own 180 s each). Both bound what
  a stalled server or a hostile file costs while it's never charged.
- Internal: the per-page budget (`Spend`) moved from `ocr_pages.py` to `spend.py` and prices OCR and table pages
  from one reservation pool.

### 1.2.0 (2026-09-24)

- **Price: $20.00 per 1,000 documents** (was $5.00 before release), so a document of up to 100 pages is covered: text
  extraction costs by the page. OCR pages stay at $10.00 per 1,000.
- Memory: at most 4 GB (was 16 GB). Parsing and OCR use one core at a time below 4 GB anyway.

### 1.1.1 (2026-09-24)

- **Fix: PDFs that pdfminer cuts short are read in full.** On some incrementally updated PDFs (cross-reference
  streams), pdfminer saw only the first pages and the document failed with "the PDF has no text on the pages read"
  (the FY2025 US budget's 295-page Analytical Perspectives: 2 empty pages). When pdfminer finds fewer pages than
  the file declares, the text is now read with PDFium instead; tables aren't detected on that path, and
  `truncatedReasons` says so.

### 1.1.0 (2026-09-24)

- **OCR for scanned PDF pages and images.** PDF pages that are only a picture are read with Tesseract OCR (the same
  code as image-ocr, now the shared `mms_ocr` package), and their text goes where the page is, in both `text` and
  `markdown` (with its `<!-- page N -->` marker). Image files (PNG, JPEG, WebP, TIFF, GIF, BMP) are read the same
  way. New input `ocr` (default on) and `languages` (32 languages, up to 4, default English). New output fields
  `ocrPages` and `ocrConfidence`; `imageOnlyPages` still lists the scanned pages.
- Each OCR page runs in its own process with image-ocr's limits: 3 minutes and a memory cap per page, pages rendered
  at 300 DPI, a 50-megapixel cap. A page that fails is named in `truncatedReasons` and the rest of the document still
  comes back. Pages that already have a text layer are never OCR'd.
- **Charged as a separate event, `ocr-page` ("OCR page"), $10.00 per 1,000** (dense scanned pages measured
  $3.40-4.20 per 1,000 in compute),
  only for pages where OCR found text, and only once the document's result is in the dataset. Blank scanned pages and
  failed pages aren't charged. Before a document's scanned pages are read they are reserved against the maximum cost
  per run, after setting aside every document in flight; pages it doesn't cover aren't read (`truncatedReasons`).
- With `ocr` off, 1.0's behaviour: scanned PDFs are refused (`scannedNoText`), and images are reported as
  unsupported, both uncharged.
- The Docker image installs tesseract and the same 32 language models as image-ocr. Default memory 2 GB (was 1 GB),
  minimum 1 GB, maximum 16 GB: see the measurement below.
- The PDF's own text now has NUL characters removed (they come from broken font maps and are never real text).
- Internal: the run-limit `Slots` now comes from mms\_common (this actor kept its own copy before).

Measured 2026-09-24, locally in the Docker image (arm64, Tesseract 5.5.0), on a 20-page scanned PDF (IRS Form W-9
pages rendered at 300 DPI grayscale, ~1,050 words per page, 25 MB), with the run's memory and CPU capped as on Apify:

- 1 GB / 0.25 CPU: 756 s, peak memory 194 MB. 2 GB / 0.5 CPU: 321 s, peak 193 MB. 4 GB / 1 CPU: 155 s, peak 186 MB.
  All 20 pages read, mean confidence 95.1. Memory stays under 200 MB at every size; OCR is CPU-bound, so the default
  went to 2 GB for speed (a dense page takes ~16 s instead of ~38 s at 1 GB, well inside the 3-minute page limit even
  with several languages), not for memory.
- Compute per dense page: ~0.009-0.011 CU at every size, about $3.40-4.20 per 1,000 pages at $0.40/CU. The $5.00 OCR
  page price comes from image-ocr's measured cost (its per-image price); on pages this dense the margin is thin (a
  sparser page, ~3 s of OCR, costs about half). Re-check with `tools/confirm_run.py` on Apify before publishing.

Before building (ACTOR-CHECKLIST.md, section 0), 2026-09-24:

- **Source and terms:** unchanged: only the files the user supplies. Tesseract and its tessdata models are
  Apache-2.0; Pillow MIT-CMU; pypdfium2 BSD-3/Apache-2.0 on PDFium BSD-3 (licences read for image-ocr 1.0.0).
- **Demand (RESEARCH-candidates.md, batch 10):** PDF-to-text group 516 users/30d over 183 actors; its leader
  memo23/pdf-text-extractor (84) advertises "Reads scanned PDFs via OCR (23 languages)" at $15 per 1,000 OCR pages.

### 1.0.0 (2026-09-24)

First release.

- PDF, Word (.docx) and Excel (.xlsx) documents from user-supplied URLs to plain text, Markdown (headings, lists,
  tables as Markdown tables, PDF page markers), tables as arrays of rows, and metadata (title, author, created and
  modified dates, page count, sheet names, file name and size). The format is read from the file's bytes.
- PDF with pdfplumber (text and ruled tables) and pypdf (metadata, encryption); Word with python-docx, including
  content controls; Excel with openpyxl in read-only mode (saved formula results, typed cell values).
- Password-protected PDFs, scanned (image-only) PDFs, old .doc/.xls, PowerPoint, OpenDocument, RTF, web pages and
  missing files are reported per URL with the reason, and never charged. Mixed PDFs list their image-only pages.
- Limits: 50 MB per file, 1 GB per run, "Max pages per PDF" (default 50, at most 100), 800,000 characters of text
  and of Markdown per document, Excel row/column/cell caps, 300 MB unpacked, ZIP-bomb check. What's left out is
  listed in `truncatedReasons`.
- Each document is parsed in its own process with a 3-minute deadline and a memory cap, so one bad file fails alone.
- Charged per document, through Apify's standard `apify-default-dataset-item` event. "Max documents per run" and the
  maximum cost per run are honoured before anything is downloaded; a failed document gives its place back.
- robots.txt (and AI-crawler opt-outs), Crawl-delay and the private-network guard from the shared fetching code.
