Darwin Core Archive Extractor

Point this Actor at any GBIF IPT (Integrated Publishing Toolkit) host and pull biodiversity data out of the Darwin Core Archive (DwC-A) standard — the format used by natural-history museums, herbaria, universities and biodiversity networks worldwide to publish occurrence, taxon and sampling-event datasets.

This is a generic, per-host runner: it works against any institutional IPT installation, not one hard-coded site — and it is not a wrapper around the central GBIF API. You bring the IPT host; it does the rest.

What it does

Two modes, auto-selected by the mode input:

resources — the dataset catalog (cheap)

Reads the host's /rss.do feed and returns one row per published dataset:

• title, description, publisher
• dwcaUrl — the Darwin Core Archive ZIP download link
• emlUrl — the EML metadata link
• resourceKey, guid, recordCount (when the host publishes it), lastPublished

Perfect for discovering what a given institution publishes, or for feeding a second run in archive mode.

archive — records from one dataset

Downloads a chosen resource's Darwin Core Archive, parses meta.xml as the source of truth (which file is the core, the exact column order, delimiter, quote char and encoding), then streams the core file into one flat row per record:

• Common Darwin Core terms hoisted to the top level for easy table viewing: scientificName, eventDate, decimalLatitude, decimalLongitude, basisOfRecord, country, countryCode, kingdom, family, recordedBy, catalogNumber, occurrenceID, taxonID, eventID
• The full record in a nested record object (every column from the archive)
• Optional termUris mapping each field to its full Darwin Core term URI
• Works for Occurrence, Taxon and Event core types

The archive is streamed and capped by maxRecords, so large datasets don't blow up memory.

Input

Field	Type	Description
iptHost	string (required)	Any IPT base URL. Bare host, host+path, or a full rss.do/archive.do URL — all normalized.
mode	select	resources (catalog) or archive (records). Default resources.
resource	string	Resource key (?r= value) to download in archive mode. Empty = auto-select.
resourceFilter	string	Case-insensitive substring on title/key. Filters the catalog or picks the archive to download.
maxRecords	integer	Cap for archive mode. Default 1000.
publishedOnly	boolean	In resources mode, skip registered-but-unpublished items. Default true.
keepRawTerms	boolean	In archive mode, attach full Darwin Core term URIs. Default false.

Example

List every dataset an IPT publishes:

{ "iptHost": "https://ipt.gbif.org", "mode": "resources" }

Stream occurrence records from one dataset:

{
  "iptHost": "https://ipt.gbif.org",
  "mode": "archive",
  "resource": "quercus_parc_test",
  "maxRecords": 5000
}

Example IPT hosts

Any GBIF IPT works. A few public ones:

• https://ipt.gbif.org
• https://ipt.gbif.es
• https://data.canadensys.net/ipt
• https://ipt.inbo.be
• https://ipt.idigbio.org

Output

Records are pushed to the dataset with a clean Overview table view. Export as JSON, CSV, Excel or feed straight into the API.

Pricing

Pay-per-result: you are charged per record produced (per catalog resource in resources mode, per occurrence/taxon/event row in archive mode). No subscription.

Notes

• meta.xml is authoritative — column positions are never assumed.
• Delimiter, quote char and encoding are read from meta.xml per archive.
• Registered-but-unpublished catalog items (no archive URL) are skipped gracefully.
• An IPT with zero published resources is a valid empty answer; a run that produces zero records on a mode that should yield rows exits non-zero rather than pretend success.

Standards

• Darwin Core: https://dwc.tdwg.org/
• Darwin Core text (archive) guide: https://dwc.tdwg.org/text/
• IPT: https://www.gbif.org/ipt