Arcaeon Verified Snapshot

Capture exactly what a web page says right now, with a cryptographic, timestamped proof that the capture wasn't altered.

Most scrapers give you data. This gives you data you can prove: notarized web content. Give it a URL; get back the page content plus an Ed25519 signature binding the URL, the capture time, the HTTP status, the final URL after redirects, and a SHA-256 of the raw content bytes. Anyone can verify it against Arcaeon's published public key. Change one byte, or the status, or the final URL, and the proof breaks.

Who it's for: anyone who might have to prove what a page said at a moment, legal and compliance teams, journalists, dispute resolution, and AI agents that need a provable record of a source.

Output (per URL): url, captured_at (UTC), content_sha256 (of the raw bytes), content_length, content, content_encoding (utf-8 or base64), signature, signature_version, public_key, how_to_verify, plus http_status / final_url / content_type.

Schema: v2 (current) vs v1 (legacy)

signature_version on every record tells you which shape you have.

v2 (arcaeon-verified-snapshot/2), current since 2026-09-04: the signed statement binds v + url + captured_at + content_sha256 + http_status + final_url. content_sha256 is always the SHA-256 of the raw bytes as fetched, never of a re-encoded copy. The content field is stored as UTF-8 text only when the bytes round-trip losslessly through UTF-8 (bytes == text.encode("utf-8")); otherwise content is base64 and content_encoding says so. Because http_status and final_url are bound into the signature, a 404 or redirected body cannot be presented as a verified 200 of the original URL.

v1 (arcaeon-verified-snapshot/1), legacy, records from before 2026-09-04: the signed statement bound only v + url + captured_at + content_sha256. http_status and final_url were recorded on the record but were not part of what was signed, so a caller could not tell a genuine 200 from a substituted status or redirect target by the signature alone. content was stored via content.decode("utf-8", errors="replace"), which silently corrupts non-UTF-8 bodies (binary, Latin-1, PDFs): for those pages, sha256 of the stored content does not match the signed content_sha256, so the old how_to_verify recipe failed on its own output. UTF-8 pages were unaffected. If you hold a v1 record, treat its http_status and final_url as unproven even if the signature checks out.

How to verify

Verification has two halves, and they answer different questions. Doing only the first is the common mistake.

1. Integrity: has this record been altered since it was signed?

1. Recover the raw bytes of content: if content_encoding is "utf-8", content.encode("utf-8"); if "base64", base64.b64decode(content).
2. Recompute SHA-256 over those raw bytes; it must equal content_sha256.
3. Rebuild the canonical statement as compact sorted-key JSON:
   • v2: {v, url, captured_at, content_sha256, http_status, final_url}
   • v1 (legacy): {v, url, captured_at, content_sha256} only, status/final_url are not part of this statement
4. Check the Ed25519 signature against public_key.

2. Identity: is the key that signed it actually Arcaeon's?

Step 1 checks the signature against the public key the record carries inside itself. That proves the record is internally consistent, and nothing more. Anyone can generate a keypair, sign any content they like, and ship their own public key in their own record, and it will pass step 1 perfectly. A record cannot vouch for its own author.

So compare public_key against Arcaeon's published signing keys:

https://arcaeon.io/.well-known/arcaeon/snapshot-signing-keys.json

That document lists each key as {key_id, public_key (base64), algorithm, valid_from, status}. Compare the key bytes (records use hex, the document uses base64, they are the same 32 bytes) rather than any label. There is deliberately no key_id field inside a record: a self-asserted identifier proves nothing, because a forger would simply copy it. The key material is the identifier.

A status of active or retired is trusted; retired means rotated out on schedule, so records it signed while it was current still verify. A revoked key means the private half is believed exposed, and nothing it signed can be relied on.

Doing both, in code or from the shell

verify_snapshot(record, content, trusted_keys=None) in src/snapshot.py returns:

{"valid": bool, "legacy": bool, "reason": str,
 "key_trusted": True | False | "unchecked", "key_reason": str}

valid and key_trusted are independent, and both must be True before treating a record as a genuine Arcaeon snapshot. key_trusted has a third state on purpose: "unchecked" means no key list was supplied or it could not be fetched. An unreachable key list is an unanswered question, never evidence of forgery, so it never reports False.

From the command line:

py src/verify_cli.py verify --record snapshot.json                       # fetches the published key list
py src/verify_cli.py verify --record snapshot.json --content page.html   # check against your own copy of the bytes
py src/verify_cli.py verify --record snapshot.json --keys-file keys.json # use a local key list
py src/verify_cli.py verify --record snapshot.json --require-trusted-key # fail unless identity is confirmed

The key list is fetched with the standard library only, over https only (a plaintext http key URL is refused, since anyone on the path could substitute the list and make the whole check theater), with a timeout. Exit codes: 0 verified, 1 signature does not verify, 2 signed by a key that is not Arcaeon's, 3 usage error.

Without --content, the tool recovers the bytes from the record's own content field, which checks the record against itself. Passing --content with a copy you hold independently is the stronger check, and the output says which one happened.

Always check legacy before trusting a record's http_status/final_url.

Where it will and will not fetch

The actor signs whatever it captures under Arcaeon's key, so where the capture is allowed to go is part of what the signature means. Since 2026-09-05 the fetch is fenced:

• Schemes: http and https only. file:, ftp:, data: and gopher: are refused by name, and anything else off the allowlist is refused too.
• Addresses: the host is resolved before the socket opens and the answer is checked. Loopback, link-local (which covers the 169.254.169.254 cloud metadata address), RFC1918 and IPv6 unique-local, carrier-grade NAT (100.64.0.0/10), multicast, reserved and unspecified addresses are all refused. Refusing by hostname would not be a fence at all: a public name resolves wherever its owner points it.
• Every answer, not just the first. A name returning one public address and one loopback address is refused.
• Every redirect hop. Redirects are followed one at a time and each hop is checked before it is fetched, with a cap of five hops. A chain that runs past the cap is refused rather than captured at whatever it had reached.

A refused URL is not a capture. It produces no signed record; it appears in the dataset as a status: "failed" row with the reason, and it is not billed.

Honest limits

Things this tool does not do, stated here rather than discovered later.

• DNS rebinding is not closed. The host is resolved, the addresses are checked, and then the URL is handed to the HTTP client, which resolves it again on its own. A name whose DNS answer changes between those two moments can pass the check and be fetched at a different address. Closing that requires pinning the connection to the address that was checked, which needs a custom transport and is not implemented. What is closed is the ordinary case: an internal URL typed in, or a redirect chain that ends somewhere internal.
• The fence is on the capture fetch only. The actor's own calls to the Apify API deliberately bypass it: Apify's in-container API base is an RFC1918 proxy (10.0.34.85:8010), so a guard applied there would refuse the platform itself. Those calls go to a fixed, configured host and never to a caller-supplied URL.
• A valid signature says nothing about whether the page was telling the truth. It binds what was served, at that status, at that final URL, at that moment. Content accuracy is not a claim this makes.
• key_trusted: "unchecked" is not a pass. It means the key list could not be read. See "How to verify" above; both halves matter.
• A zero-byte capture is not billed. A 200 with an empty body is a real capture and produces a real record, but it is treated as unproven for billing and is pushed as a failure row. The bias is deliberate and costs us, not the caller.
• Verification against the record's own content field checks the record against itself. Pass --content with a copy you hold independently for the stronger check. The output says which one happened.

Billing

One charge per delivered, verified snapshot. The charge is made only after the record's signature has been checked against the bytes that were actually fetched, and only after the record has been pushed, so a capture that failed, a record that does not verify, or a result that could not be delivered is never charged. Failures are still pushed as status: "failed" rows carrying the reason, because a silent gap is worse than an error.

The billable event is named by ARCAEON_CHARGE_EVENT (default verified-snapshot) and is raised with an explicit call. This is enforceable only under pay-per-event pricing. Under Apify's pay-per-result model the platform bills per dataset item, so a failure row would still count no matter what the code decides; the actor's own charge call never fires for one, but the pricing on the actor has to be configured as pay-per-event for that to be the whole story. Stated plainly because the alternative is a promise the code cannot keep on its own.

Built by Arcaeon.