# Changelog of Arcaeon Verified Snapshot (`dan8433/verified-snapshot`) Actor - **URL**: https://apify.com/dan8433/verified-snapshot/changelog.md - **Full Actor documentation**: https://apify.com/dan8433/verified-snapshot.md ## CHANGELOG — Arcaeon Verified Snapshot (Apify actor, fast-dollar product A) CONVENTION (2026-09-05, board item 135): headings from here forward are `## - - `, so `.actor/actor.json`'s `version` field and this file's highest heading agree, the way `prepublish_gate.py`'s version-agreement check expects. Every entry below this line stays exactly as written -- this repo's CHANGELOG was date-keyed from the start and the old entries are not being retrofitted. `0.1.4` is not a new release: it is the build tag the 2026-08-12 entry below already records as the one actually built and deployed to Apify ("Build succeeds (0.1.4)"); carrying it forward as the declared version names what is real rather than implying work that has not happened. `actor.json` had stayed at the bare `0.1` it started with. CORRECTION (2026-09-07, the Monday publish window): the convention above conflated two numbers. Apify's `.actor/actor.json` `version` field MUST be `MAJOR.MINOR` only (`apify push` refuses anything else, verbatim: "Version number must be MAJOR.MINOR"), and the third component in this file's headings is Apify's own BUILD NUMBER, assigned by the platform when a push succeeds (`taggedBuilds.latest`). So `actor.json` stays at `0.1`, headings carry `0.1.`, and a heading's build number is TRUE only once the build with that number exists on Apify. Measured today before writing this: Apify's newest build was 0.1.9 from 2026-08-13, which means the 0.1.9 entry below labelled work that had never been pushed; its fixes ship for the first time in the 0.1.10 build this window produces. `prepublish_gate.py`'s version-agreement check for the docker-actor shape compares `actor.json`'s MAJOR.MINOR against the heading's prefix, and the build number is confirmed against the platform after the push, not before. CORRECTION (2026-09-05, board item 154(a)): 135(a) picked `0.1.4` off the first same-day build note in the 2026-08-12 entry below and missed that the SAME entry's follow-up ("2026-08-12 (later)") already records a clean `0.1.7` build working end-to-end. Neither was actually the newest. Checked against the Apify API directly (`GET /v2/acts/qi1AAY5TcgZZjQgFD/builds`) instead of trusting either changelog note: the newest build with `status: SUCCEEDED` is `0.1.9` (build id `CTGuZoNZXMLKito36`, started 2026-08-13T15:41:04Z) -- ahead of `0.1.8` (`WipHAIq1OWipuqmz1`, same day, earlier) and `0.1.7` (`pceA3FYoiVmno78nH`, 2026-08-12). Note the board item that queued this fix named a different build id and date (`27hty7USnSK9RXDd7`, 2026-08-15) than what the API actually returns for the newest SUCCEEDED build; the version number it asked for (`0.1.9`) checks out, the id/date attached to it in the board text does not match the API and is not repeated here as fact. The version tag below is corrected to `0.1.9` on the same logic as the CONVENTION note above: naming the last real build, not implying the unpublished work in this entry has shipped. ### 0.1.11 - 2026-09-07 - failed captures leave the dataset, because the dataset is what bills MEASURED ON THE LIVE PLATFORM, and it overturns this file's own 0.1.9 claim that the billing gate was closed in code. The Store listing is `PAY_PER_EVENT`, which is why 0.1.9 said failures are pushed but not charged. The priced event is Apify's built-in `apify-default-dataset-item`, and that event fires per item pushed to the DEFAULT DATASET, not when this actor calls `_charge()`. So a refused capture billed the customer anyway. Two runs today, same build, same account: ``` SUCCESS (arcaeon.io captured and signed) chargedEventCounts {"apify-default-dataset-item": 1} FAILURE (http://127.0.0.1:8080 refused by the URL guard, row says billable:false) chargedEventCounts {"apify-default-dataset-item": 1} ``` Identical. `billable: false` is a claim to the CALLER. Not writing the row is the only thing the PLATFORM reads. The 0.1.9 entry's own honest limit ("enforceable in code only under pay-per-event pricing... under pay-per-result the platform bills per dataset item") turned out to describe the live configuration rather than a hypothetical. A caution about the measurement, because it nearly fooled me twice. Reading the run object immediately after `waitForFinish` showed `chargedEventCounts: 0`; the charge settles after the response. And the RUNS LIST endpoint omits `chargedEventCounts` entirely, so a sweep of ten runs reported zero charges everywhere. Both readings said "no leak" and both were the instrument, not the world. Only fetching each run individually, after settlement, is true. `accountedChargedEventCounts` stays 0 on every run because we own the actor; owners are not billed for their own runs, so that field says nothing about what a customer pays. WHAT CHANGED. `process()` gains an injectable `fail` sink beside `push` and `charge`. Failed captures no longer reach the dataset at all: they go to the run's key-value store as a `FAILURES` record, and every run now writes an `OUTPUT` record with submitted / verified / failed / charged and a pointer to that record. The caller still sees every URL attempted, which is the thing 0.1.9 refused to give up, and now sees it without paying for it. The tests moved with the contract rather than being made to pass: seven billing tests now assert `push.count == 0` on every failure path and read the row out of the failure sink, and the token-scrubbing test follows the row to the key-value record, which is just as durable and just as customer-visible. 135 passed. ### 0.1.10 - 2026-09-07 - the trusted-key list is no longer everyone's key - FIX, board item 144 (velouria repo): `snapshot-signing-keys.json` gained a second entry the same day, `purpose: "mcp_vet_receipt"`, publishing mcp_vet's receipt-signing key alongside the snapshot key so a stranger can verify a receipt's did:key against something other than mcp_vet's own word. `load_trusted_keys()` had no filter on WHAT an entry in `keys` was for: it read every dict as a snapshot key regardless of a `purpose`/`use` field, so publishing that second entry would have silently made mcp_vet's key trusted for Arcaeon Verified Snapshot records too -- not a crash, a wrong "trusted". - NEW `SNAPSHOT_KEY_PURPOSE` + `_entry_purpose()` in `src/snapshot.py`: an entry naming no purpose/use at all (every entry published before this fix, including the live `arcaeon-snapshot-2026-08` entry, which spells it "use") counts as a snapshot key by default; an entry naming a different purpose is excluded from `load_trusted_keys()`'s result. - Tests: a new `test_entry_for_a_different_purpose_is_not_trusted_as_a_snapshot_key` proves a same-document, different-purpose key is excluded while the real snapshot key still verifies alongside it; `test_published_key_document_shape_matches_what_the_loader_expects` (reads the live `velouria/projects/arcaeon_site/.well-known/...` file) updated to apply the snapshot-shape checks (`key_id`, `valid_from`) only to snapshot-purpose entries, and to assert a differently-purposed entry never ends up in the trusted set. All 19 tests in `test_snapshot.py` pass. - Verified end to end against the live two-entry file: a real snapshot record, signed with the actual `arcaeon_snapshot_signing.key`, checked with `src/verify_cli.py verify --keys-file --record ` reports `key_trusted: true`, `key_reason` naming `arcaeon-snapshot-2026-08`. - No commits, no deploys, no sends -- pre-staged for the Mon 9/7 6:05 window per board item 144. ### 0.1.9 - 2026-09-05 - the fetch is fenced, failures are not billed, the token leaves the URL - FIX, board item 25 (SSRF): the actor took a URL from whoever ran it, fetched it with `follow_redirects=True`, and signed the body under Arcaeon's key. Nothing constrained where that fetch could go. A caller could aim it at an address only the container can reach -- `169.254.169.254`, `127.0.0.1`, anything on the private network the actor sits in, and this repo's own CHANGELOG records an internal proxy at `10.0.34.85:8010` -- and walk away with that body carrying a valid Arcaeon signature saying "this is what the web said". The proof machinery would have worked exactly as designed while certifying something that was never a web page. - NEW `src/url_guard.py`, the fence. Scheme allowlist: `http`/`https` only, with `file:`, `ftp:`, `data:` and `gopher:` refused by name. Address allowlist: the host is RESOLVED before the socket opens and every answer is classified, refusing loopback, link-local (which is what covers the metadata address), RFC1918 and IPv6 unique-local, carrier-grade NAT, multicast, reserved and unspecified ranges. Refusing by hostname would not be a fence at all, since a public name resolves wherever its owner points it; the refusal is made against the resolved address. - Every resolved answer is checked, not just the first: a name returning one public address and one loopback address is refused, because which address the connection actually uses is not ours to choose. - Redirects are now followed BY HAND, one hop at a time, so the fence applies to every hop. `follow_redirects` is off. httpx would have walked the whole chain inside one call with no way to inspect the intermediate targets, which is the interesting case: a public first hop redirecting to the metadata service. Hop cap of 5, and exceeding it is a refusal rather than a silent stop, because returning the last redirect's empty body as if it were the page would be worse than an error. - BUG FOUND BY THE NEW TESTS, in the stdlib rather than in our code: on Python 3.14 `ipaddress.IPv4Address("100.64.0.1").is_private` is **False**. RFC 6598 shared address space (carrier-grade NAT) moved out of `is_private` after 3.12. A fence written against that one predicate would have quietly opened a /10 of ISP-internal space on a routine interpreter upgrade with no test failing anywhere. Fixed by naming the ranges explicitly (`EXTRA_PRIVATE_NETWORKS`) and adding an `is_global` backstop as the last test, so the fence does not depend on one stdlib predicate keeping its meaning. - HONEST LIMIT, now in the README: DNS rebinding is NOT closed. The host is resolved and checked, then the URL is handed to httpx, which resolves it again. A name whose answer changes between those two moments can pass the check and be fetched elsewhere. Closing it needs the connection pinned to the checked address, which needs a custom transport, and it is not done. What is closed is the ordinary case. - ALSO STATED: the fence is on the capture fetch only. The actor's own Apify API calls deliberately bypass it, because Apify's in-container API base is itself an RFC1918 proxy and a guard there would refuse the platform. Those calls go to a fixed configured host, never to a caller-supplied URL. - `snapshot()` gains `max_redirects`, `resolver` and `client` keyword arguments (all optional, all defaulted to the previous behaviour plus the fence) so the guard is testable without DNS or a socket. New `snapshot_with_content()` returns `(record, raw_bytes)`; `snapshot()` is now the thin wrapper over it. No record schema change, no signature change. - FIX, board item 26 (billing): `main.py:76-78` caught a failed capture and pushed `{"url", "error"}` into the pay-per-result dataset, which billed the customer for a row containing no proof of anything. Failures are still pushed -- a silent gap is worse than an error row -- but now carry `status: "failed"`, `billable: false` and the reason, and the charge call is not made for them. - The billing gate is the SIGNATURE, verified against the bytes that were actually fetched, not "no exception was raised". New `is_billable(record, content)` refuses an unsigned row, a record whose signature does not check out, a record signed over different bytes than were delivered, an empty body, and a legacy v1 artifact (whose `http_status`/`final_url` were never bound, so it is a weaker thing than what this actor now sells). - Charging is an explicit `_charge()` call against Apify's pay-per-event endpoint, event name `ARCAEON_CHARGE_EVENT` (default `verified-snapshot`). Push happens BEFORE charge: if the push fails the customer never received the result, so nothing is billed for it. Of the two ways to be wrong, giving a snapshot away free is the one that does not take somebody's money. - HONEST LIMIT, in the README: this is enforceable in code only under pay-per-event pricing. Under Apify's pay-per-result model the platform bills per dataset item, so a failure row still counts no matter what the code decides. The actor no longer ASKS to be paid for a failure; the pricing on the actor has to be set to pay-per-event for that to be the whole story. Not claimed as fixed beyond what the code controls. - FIX, board item 27 (credentials): the Apify token travelled in the URL query string on both API calls (`?token={TOKEN}` at the old `main.py:39` and `:49`). Query strings are written down -- by proxies, by access logs, by the error message attached to a failed request. The token now goes in an `Authorization: Bearer` header. - All three platform calls (`_get_input`, `_push`, the new `_charge`) are built through one `build_request()`, so there is exactly one place a credential could be put in the wrong half of the message. `build_request()` REFUSES a URL that already carries one, checked by parameter name (`token`, `apiToken`, `api_token`, `access_token`, `auth`) and, when a token is configured, by value as well, so a credential smuggled under another parameter name is caught too. - `preflight()` refuses to run at all when the only credential available is a query param on the configured API base, and `main()` calls it first. - The token is never printed. Everything logged goes through `_scrub()`, including the exception text copied into a pushed failure row -- a dataset row is a durable customer-visible artifact and a leak there has a long half-life. - SIBLING, NOT FIXED HERE: `arcaeon-hiring-alerts/src/main.py:42` and `:52` carry the identical `?token={TOKEN}` pattern. Named in the 9/5 report for the sibling task; untouched by this change. - Tests: 18 -> 134, all passing, in three new files. `test_url_guard.py` (77) parametrises every refused address class three ways -- classifier, hostname resolved by a stub, and IP literal -- plus scheme refusals, split DNS answers, redirect-to-private refused mid-chain with the client asserting no second request left, a relative-Location hop, redirect-to-`file:`, the hop cap, a chain exactly AT the cap still succeeding (the off-by-one that would refuse real captures), and the guard proven to be wired into `snapshot()` rather than sitting beside it. `test_actor_billing.py` (18) mocks the charging call: success charges once, three successes charge three times, a failed capture charges zero and is still pushed, a tampered record charges zero, a mismatched-bytes record charges zero, an undeliverable result charges zero, and a mixed batch of three charges exactly twice. `test_apify_auth.py` (21) replaces `urlopen` with a recorder and asserts the request that would have gone out: header present, query string empty, token absent from the URL, refusals on credential-bearing URLs, and four separate proofs that the token reaches no log line and no dataset row. - NO REAL DNS and NO SOCKETS in any new test. Hostnames go through a stub resolver that raises on anything a test did not name, so a must-hit is a statement about the guard and not about what somebody's nameserver answered that afternoon. - RED-THEN-GREEN, proven externally against a scratch copy, six sabotages each restoring the pre-fix behaviour: only the first hop checked -> 3 red; hop cap removed -> 2 red; address fence removed -> 40 red; failed capture charged again -> 3 red; signature no longer gating the charge -> 3 red; token back in the query string -> 12 red. Restored: 133 passed. - Not published, not deployed, not committed. ### 2026-09-04 (later) - key identity: published signing keys, and a verifier that checks them - FIX (the limit the v2 work left standing): `verify_snapshot()` checked the signature against `record["public_key"]`, the key carried inside the record itself. That establishes internal consistency and nothing else. Anyone can generate a keypair, sign any content they like, ship the matching public key in their own record, and pass verification perfectly. The product's whole promise is "prove what this page said", and we could not actually prove the record was ours. - The signing key was already persistent, not per-run: one dedicated Arcaeon Ed25519 keypair created 2026-08-12, private half in the ACL-locked secrets directory, never in the repo. Confirmed on 2026-09-04 that the private half still derives exactly the public key recorded on 2026-08-12 (`4d0a4156...`), so every record signed since then shares one verifiable identity. Nothing was generated, moved, or rotated. - NEW: the public half is published at a stable location, `https://arcaeon.io/.well-known/arcaeon/snapshot-signing-keys.json`, listing `{key_id, public_key (base64), algorithm, valid_from, status}` plus a stated rotation policy. `active` and `retired` keys are trusted (a scheduled rotation must not invalidate records signed before it); `revoked` is not (an exposed private half invalidates everything it signed). - NEW: `verify_snapshot(record, content, trusted_keys=None)` returns two more fields, `key_trusted` and `key_reason`. `key_trusted` is `True`, `False`, or `"unchecked"`. The third state is deliberate: no key list supplied, or one that could not be fetched, is an unanswered question and never reports `False`. Treating a network failure as evidence of forgery would be a lie in the direction that looks rigorous. - `valid` and `key_trusted` are reported as independent axes rather than folded together, because a forged record is `valid: True`. Integrity and identity are different facts and a caller needs to see which one failed. - Matching is on key BYTES, not labels. Records stay free of any `key_id` field on purpose: a self-asserted identifier proves nothing, since a forger copies it. `_coerce_public_key()` accepts hex (record side) or base64 (document side) unambiguously, since 32 bytes is 64 hex characters and 64 base64 characters would decode to 48. - NEW: `src/verify_cli.py`, a command-line verifier. `py src/verify_cli.py verify --record r.json [--content page.html] [--keys-url URL | --keys-file PATH | --no-key-check] [--require-trusted-key]`. Fetching is stdlib only (`urllib`), with a timeout, and **refuses a plaintext `http` key URL**: a key list substituted in transit would make the identity check theater. Exit codes separate the two failure modes, `1` altered, `2` signed by a stranger, so a script can tell them apart. - `--content` verifies against an independently held copy of the bytes; the default recovers them from the record's own `content` field, and the output states which happened rather than letting a self-check pass for an independent one. - No record schema change. Existing v2 records verify unchanged and gain identity checking for free; only `how_to_verify`'s wording is extended to name the second step. - BUG FOUND BY THE NEW TESTS: `load_trusted_keys()` was not idempotent, so the CLI (which normalizes once, to fail early on a bad key file) had its key set silently emptied by the second pass inside `verify_snapshot()` and reported `unchecked` where it should have reported `False`. A silent downgrade to the permissive state is the worst shape this bug could have taken. Fixed by passing an already-normalized mapping straight through. - Tests: 7 → 18, all passing. Must-hit: a record signed by a listed key is trusted. Must-miss: an unlisted key is `False`. Third state: no key set, and a failed fetch, both give `unchecked` carrying the reason. Mutation proof for the `False` arm: a forged record built with the forger's own keypair, same URL and same `captured_at`, verifies flawlessly against itself (`valid: True`) and is caught only by the published list. Plus: revoked-vs-retired status handling, hex/base64 interop, plaintext-http refusal, valid and key_trusted moving independently, CLI exit codes, and a shape test that fails here if the published site document drifts from the loader that reads it. - Verified end-to-end with the real key and the real published document: a record signed by the production key reports `key_trusted: True, key_id 'arcaeon-snapshot-2026-08'`. Against the live https URL it currently reports `unchecked (HTTP 404)`, which is correct, and is the proof the file is not deployed yet. - Site: `.well-known/arcaeon/snapshot-signing-keys.json` added to `projects/arcaeon_site`, plus a `vercel.json` Content-Type header entry matching the existing `.well-known/offers.json` precedent. Vercel uploads the directory whole, so no file list needed updating. - Not published, not deployed, not committed. The README sentence "verify against Arcaeon's published public key" becomes literally true at the moment the site deploys, so the site deploy leads and any README republish follows it. ### 2026-09-04 - schema v2: raw bytes hashed, status and final_url bound, tests added - FIX (audit finding, arcaeon_audit_51_53_40_2026-09-04.md Task 3): stored `content` was `content.decode("utf-8", errors="replace")`. For any non-UTF-8 page (Latin-1, binary, a PDF) the stored artifact re-encoded to different bytes, so `sha256(stored content)` did not match the signed `content_sha256` and the record's own `how_to_verify` recipe failed on its own output. The dead `content_base64` fallback never ran because `errors="replace"` never raises. - FIX: `content_sha256` is now always computed over the RAW bytes as fetched, before any text decoding. `content` is stored as UTF-8 text only when the bytes round-trip losslessly (`bytes == text.encode("utf-8")`); otherwise `content` is base64 and a new `content_encoding` field says which. Added `decode_content(record)` to recover the raw bytes correctly either way, and it refuses (does not silently corrupt) a record with no `content_encoding` field. - FIX: the signed statement previously bound only `url + captured_at + content_sha256`. `http_status` and `final_url` were recorded but unsigned, `follow_redirects=True`, so a 404 or redirected body could be presented under a different status or URL with a signature that still checked out. The statement now also binds `http_status` and `final_url` (schema `arcaeon-verified-snapshot/2`). `sign_snapshot()` requires both as keyword arguments. - BACKWARD COMPAT: `verify_snapshot(record, content)` now returns `{"valid": bool, "legacy": bool, "reason": str}` instead of a bare bool. A v1 record (`signature_version` `arcaeon-verified-snapshot/1`) is read without crashing and comes back `legacy: True` with a reason explaining that its `http_status`/`final_url` were never bound to the signature and must not be trusted even when `valid: True`. - `how_to_verify` rewritten to match the new shape (recover raw bytes per `content_encoding`, hash those, rebuild the v2 statement including `http_status`/`final_url`). - Tests added (first test file for this repo): `test_snapshot.py`, 7 tests, all passing. Must-hit: non-UTF-8 body round-trips and verifies; a 404 body cannot be relabeled 200 and still verify; a changed `final_url` breaks verification; a legacy v1 record verifies but is flagged unbound rather than crashing or being silently trusted. Must-miss (regression): a plain UTF-8 200 page verifies exactly as before; a tampered content byte still breaks verification. - README updated: v2 vs v1 schema section, corrected "how to verify" steps, field list includes `content_encoding`. - Not published, not deployed, not committed. ### 2026-08-12 — BUILT + DEPLOYED (core proven; input-wiring last mile) - His A-and-B pick (33679) → built the FAST-DOLLAR product (portfolio option A): an Apify pay-per-result actor with our tamper-evidence as the differentiator. Niche = **verified web snapshot / notarized web content** ("prove what a page said at a moment") — chosen because provenance is the WHOLE value, not a bolt-on. My call under the AI-business autonomy grant. - **CORE PROVEN** (`src/snapshot.py`, tested locally): fetch URL → capture content → Ed25519-sign a canonical statement binding url + captured_at + sha256(content). `verify_snapshot()` passes on real content, FAILS on a single tampered byte AND on a forged hash. Signing key = dedicated Arcaeon Ed25519 keypair (private in `~/.velouria-secrets/arcaeon_snapshot_signing.key`, public `4d0a4156...` for verifiers). - **DEPLOYED to Daniel's Apify** (acct Dan8433, FREE plan; token in `~/.velouria-secrets/apify_token.txt`, ACL-locked): actor `qi1AAY5TcgZZjQgFD`, https://console.apify.com/actors/qi1AAY5TcgZZjQgFD . Build succeeds (0.1.4), container runs, no crash. `ARCAEON_SIGNING_KEY` set as a secret env var (version-scoped endpoint POST /v2/acts/{id}/versions/0.1/env-vars). - **Rewrote SDK-free**: the `apify` SDK pulled a crawlee/pydantic version conflict ("cannot specify both default and default_factory") that broke the build-run. Dropped the SDK → talk to the platform via env + REST directly (requirements = just httpx + cryptography). Robust, no dep conflict. - **LAST MILE (open):** the SDK-free input read returns HTTP 404 on the default-KVS INPUT record under Apify's LIMITED_PERMISSIONS runtime → actor runs but gets no URLs → 0 items. Tried ACTOR\_*/APIFY\_* env-var fallback + ACTOR_INPUT_KEY; still 404, so the input isn't at the KVS path I'm reading. NEXT: diagnose where Apify actually exposes run input to an SDK-free container (log all env vars in a diagnostic run; likely a local mounted path or a different record/store). THEN: publish to public store + set pay-per-result monetization (needs his payout details). Reported honestly to him (33690) — NOT earning yet, not claimed as such. ### 2026-08-12 (later) — WORKING END-TO-END on platform (last mile fixed) - ROOT CAUSE of the input 404: `ACTOR_API_BASE_URL` is Apify's internal proxy `http://10.0.34.85:8010/` WITH a trailing slash, so `f"{API}/v2/..."` made a double-slash `8010//v2/` → 404. Fix: `.rstrip("/")` on API base. (Found via a diagnostic build 0.1.5 that logged the runtime env.) - **Build 0.1.7 (clean, diagnostic removed): WORKS.** Test-ran live via run-sync-get-dataset-items on 2 URLs (news.ycombinator.com + example.com) → 2 dataset items, each a signed verified snapshot, and BOTH signatures independently verify against the public key. The actor produces genuine tamper-evident notarized snapshots on the platform. "Refine current, no half-step" = DONE. - BLOCKED ON HIS APIFY ACCOUNT (console, one-time): (1) make-public requires a PUBLIC PROFILE (username) on his account ("Actor owner needs a public profile"); (2) pay-per-result monetization requires payout details. Once both: flip isPublic + set pricing → live + sellable. Product itself is complete.