# Darwin Core Archive Extractor (`datamule/darwin-core-archive-extractor`) Actor

Point at ANY GBIF IPT host (ipt.gbif.org, VertNet, iDigBio, Canadensys, INBO...) and extract biodiversity data. Lists the dataset catalog, or streams a Darwin Core Archive into flat occurrence/taxon/event rows. Parses meta.xml authoritatively. Per-IPT runner, not the GBIF central API.

- **URL**: https://apify.com/datamule/darwin-core-archive-extractor.md
- **Developed by:** [Datamule](https://apify.com/datamule) (community)
- **Categories:** Developer tools, Other
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.35 / 1,000 record extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Darwin Core Archive Extractor

Point this Actor at **any GBIF IPT** (Integrated Publishing Toolkit) host and pull
biodiversity data out of the **Darwin Core Archive (DwC-A)** standard — the format used
by natural-history museums, herbaria, universities and biodiversity networks worldwide to
publish occurrence, taxon and sampling-event datasets.

This is a **generic, per-host runner**: it works against *any* institutional IPT
installation, not one hard-coded site — and it is **not** a wrapper around the central
GBIF API. You bring the IPT host; it does the rest.

### What it does

Two modes, auto-selected by the `mode` input:

#### `resources` — the dataset catalog (cheap)

Reads the host's `/rss.do` feed and returns **one row per published dataset**:

- `title`, `description`, `publisher`
- `dwcaUrl` — the Darwin Core Archive ZIP download link
- `emlUrl` — the EML metadata link
- `resourceKey`, `guid`, `recordCount` (when the host publishes it), `lastPublished`

Perfect for discovering what a given institution publishes, or for feeding a second run
in `archive` mode.

#### `archive` — records from one dataset

Downloads a chosen resource's Darwin Core Archive, **parses `meta.xml` as the source of
truth** (which file is the core, the exact column order, delimiter, quote char and
encoding), then **streams** the core file into **one flat row per record**:

- Common Darwin Core terms hoisted to the top level for easy table viewing:
  `scientificName`, `eventDate`, `decimalLatitude`, `decimalLongitude`, `basisOfRecord`,
  `country`, `countryCode`, `kingdom`, `family`, `recordedBy`, `catalogNumber`,
  `occurrenceID`, `taxonID`, `eventID`
- The full record in a nested `record` object (every column from the archive)
- Optional `termUris` mapping each field to its full Darwin Core term URI
- Works for **Occurrence**, **Taxon** and **Event** core types

The archive is streamed and capped by `maxRecords`, so large datasets don't blow up memory.

### Input

| Field | Type | Description |
|---|---|---|
| `iptHost` | string (required) | Any IPT base URL. Bare host, host+path, or a full `rss.do`/`archive.do` URL — all normalized. |
| `mode` | select | `resources` (catalog) or `archive` (records). Default `resources`. |
| `resource` | string | Resource key (`?r=` value) to download in `archive` mode. Empty = auto-select. |
| `resourceFilter` | string | Case-insensitive substring on title/key. Filters the catalog or picks the archive to download. |
| `maxRecords` | integer | Cap for `archive` mode. Default 1000. |
| `publishedOnly` | boolean | In `resources` mode, skip registered-but-unpublished items. Default true. |
| `keepRawTerms` | boolean | In `archive` mode, attach full Darwin Core term URIs. Default false. |

### Example

List every dataset an IPT publishes:

```json
{ "iptHost": "https://ipt.gbif.org", "mode": "resources" }
```

Stream occurrence records from one dataset:

```json
{
  "iptHost": "https://ipt.gbif.org",
  "mode": "archive",
  "resource": "quercus_parc_test",
  "maxRecords": 5000
}
```

### Example IPT hosts

Any GBIF IPT works. A few public ones:

- `https://ipt.gbif.org`
- `https://ipt.gbif.es`
- `https://data.canadensys.net/ipt`
- `https://ipt.inbo.be`
- `https://ipt.idigbio.org`

### Output

Records are pushed to the dataset with a clean **Overview** table view. Export as JSON,
CSV, Excel or feed straight into the API.

### Pricing

Pay-per-result: you are charged per record produced (per catalog resource in `resources`
mode, per occurrence/taxon/event row in `archive` mode). No subscription.

### Notes

- `meta.xml` is authoritative — column positions are never assumed.
- Delimiter, quote char and encoding are read from `meta.xml` per archive.
- Registered-but-unpublished catalog items (no archive URL) are skipped gracefully.
- An IPT with zero published resources is a valid empty answer; a run that produces zero
  records on a mode that should yield rows exits non-zero rather than pretend success.

### Standards

- Darwin Core: https://dwc.tdwg.org/
- Darwin Core text (archive) guide: https://dwc.tdwg.org/text/
- IPT: https://www.gbif.org/ipt

# Actor input Schema

## `iptHost` (type: `string`):

Base URL of any GBIF IPT (Integrated Publishing Toolkit) installation. You can paste the bare host, the host + path, or a full rss.do/archive.do URL — it is normalized automatically. Examples: https://ipt.gbif.org, https://ipt.gbif.es, https://data.canadensys.net/ipt, https://ipt.inbo.be, https://ipt.idigbio.org

## `mode` (type: `string`):

resources = list every published dataset in the IPT catalog (cheap; one row per resource with its DwC-A archive URL, EML URL, publisher, guid, last-published date). archive = download one resource's Darwin Core Archive and stream its core file into one flat row per occurrence / taxon / event record.

## `resource` (type: `string`):

The IPT resource identifier to download in archive mode (the ?r= value, e.g. 'quercus\_parc\_test'). Leave empty to auto-select the first published resource (optionally narrowed by the filter below).

## `resourceFilter` (type: `string`):

Case-insensitive substring matched against each resource's title and key. In resources mode it filters the catalog; in archive mode it picks the first matching resource to download.

## `maxRecords` (type: `integer`):

Maximum number of records to extract in archive mode. The archive is streamed, so large datasets are capped without loading everything into memory. Ignored in resources mode.

## `publishedOnly` (type: `boolean`):

In resources mode, skip catalog items that are registered but have no DwC-A archive yet (no download URL).

## `keepRawTerms` (type: `boolean`):

In archive mode, attach a 'termUris' object mapping each short field name to its full Darwin Core / Dublin Core term URI (e.g. scientificName -> http://rs.tdwg.org/dwc/terms/scientificName).

## Actor input object example

```json
{
  "iptHost": "https://ipt.gbif.org",
  "mode": "resources",
  "maxRecords": 1000,
  "publishedOnly": true,
  "keepRawTerms": false
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "iptHost": "https://ipt.gbif.org"
};

// Run the Actor and wait for it to finish
const run = await client.actor("datamule/darwin-core-archive-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "iptHost": "https://ipt.gbif.org" }

# Run the Actor and wait for it to finish
run = client.actor("datamule/darwin-core-archive-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "iptHost": "https://ipt.gbif.org"
}' |
apify call datamule/darwin-core-archive-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=datamule/darwin-core-archive-extractor",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/3P8b10GvOcRCUzl29/builds/lX26HlpdtLczN1zju/openapi.json