IIIF Presentation Harvester

Point at ANY IIIF manifest or collection URL and get flat, structured metadata rows — one actor for the whole GLAM world.

IIIF (International Image Interoperability Framework) Presentation API is the metadata backbone of galleries, libraries, archives and museums. Wellcome Collection, the Bodleian, Harvard, Yale, the Library of Congress, the Bibliothèque nationale de France, the Getty and thousands more expose IIIF manifests describing digitised books, manuscripts, photographs, maps and artworks. Every one of them serves the same Manifest / Collection JSON document shape.

Every other "museum" or "digital collection" scraper wraps one institution. This actor is the generic runner: give it any IIIF manifest or collection URL and it turns the metadata into dataset rows — cleanly flattened columns plus a lossless copy of each original manifest.

What it does

• Manifest → one flat row: id, label, description/summary, the metadata label/value pairs, canvas (page/image) count, thumbnail, rights/license, attribution, provider, homepage, navDate, plus a lossless raw copy of the whole manifest.
• Collection → walked breadth-first over its manifest references (and nested sub-collections), harvesting each manifest up to your maxRecords cap.
• Per-canvas image info (optional) → with includeImageInfo on, one extra row per canvas: canvas id, label, width, height, the IIIF Image API service base (used to build tile / region / full-size image URLs like {service}/full/max/0/default.jpg), the direct image URL, and format.

Handles both major versions of the Presentation API transparently:

• IIIF Presentation API 2.x — @id / @type: sc:Manifest, label as a plain string, canvases under sequences[].canvases[], images under images[].resource, license + attribution.
• IIIF Presentation API 3.x — id / type: Manifest, every human string as a language map ({"en": ["…"]}), canvases under items[], images under items[].items[].body, rights + requiredStatement.

The version is detected from @context and every language map is collapsed to a plain string (preferring English), so the output columns are identical regardless of source version.

Input

Field	Type	Description
manifestUrl	string	The IIIF manifest or collection URL, e.g. https://iiif.wellcomecollection.org/presentation/v2/b18035723.
collectionUrl	string	Optional alias when pointing at a collection. manifestUrl wins if both are set.
includeImageInfo	boolean	Also emit one row per canvas (image/page) with dimensions and the IIIF Image service base. Default off.
maxRecords	integer	Global cap on the number of manifests harvested from a collection. Default 100.
bearer	string (secret)	Optional bearer token for gated deployments. Never logged.
extraHeaders	object	Optional extra HTTP headers, e.g. {"x-api-key": "…"}. Never logged.

Output

One dataset item per manifest (and, with includeImageInfo, per canvas). A manifest row from the Wellcome Collection, for example:

{
  "_type": "manifest",
  "iiifVersion": 2,
  "_sourceUrl": "https://iiif.wellcomecollection.org/presentation/v2/b18035723",
  "manifestId": "https://iiif.wellcomecollection.org/presentation/v2/b18035723",
  "label": "Wunder der Vererbung / von Fritz Bolle.",
  "metadata": [
    {"label": "Publication/creation", "value": "Murnau ; München : Sebastian Lux, [1951]"},
    {"label": "Physical description", "value": "31 pages : illustrations ; 15 cm."}
  ],
  "canvasCount": 36,
  "thumbnail": "https://iiif.wellcomecollection.org/thumbs/b18035723_0004.JP2/full/72,100/0/default.jpg",
  "rights": "http://creativecommons.org/licenses/by-nc/4.0/",
  "raw": { "…": "the complete manifest" }
}

Missing fields are null; raw always preserves the complete original manifest so nothing is lost.

Finding manifests

Any IIIF-compliant institution exposes manifest URLs on its object/viewer pages (look for a IIIF logo or a "IIIF manifest" link). The IIIF community maintains lists of adopters, and most GLAM APIs document their manifest URL pattern.

Pricing

Pay-per-event: $0.0005 per manifest (one harvested manifest row). Per-canvas image rows, when enabled, are bundled with their manifest at no extra charge. For a collection, leave maxRecords at the default 100 or raise it to harvest a larger set.

Notes & limits

• Scope is the IIIF Presentation API (manifests and collections). The IIIF Image API is not fetched — but each canvas's Image service base is emitted so you can build image URLs yourself.
• A malformed URL, a non-JSON response, or an HTTP 4xx returns a clear error; an empty collection simply yields 0 records.
• Binary image tiles are never downloaded — this actor harvests metadata, keeping runs fast and cheap.