# Reference List Verifier: Crossref DOI Checker (`leadharbor/reference-list-verifier`) Actor

Paste a bibliography and find the closest Crossref or OpenAlex record for each reference: DOI, match status and score, retraction or correction flag from Crossref. Pay per reference.

- **URL**: https://apify.com/leadharbor/reference-list-verifier.md
- **Developed by:** [Mohammed Samaan](https://apify.com/leadharbor) (community)
- **Categories:** AI, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$1.00 / 1,000 reference checkeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Reference List Verifier: Crossref DOI Checker

Paste a bibliography (or a list of references) and get one row per reference: the closest record found in Crossref or OpenAlex, its DOI, a match status, a score, and a retraction or correction flag when Crossref itself records one.

It reads references the way they are written in papers and theses, in any citation style, with or without numbering. It does not need DOIs up front. If a reference already contains a DOI, the DOI is checked directly.

**What it tells you, and what it does not.** A row says that a close record *was* or *was not* found. It never says a reference is correct. A `not_found` reference may still be real (a book, a thesis, a web page, a very new paper); a `matched` reference should still be read by a human before you rely on it.

### Who uses it

- Students and supervisors checking a thesis bibliography for typos, wrong years and invented citations.
- Editors and reviewers screening a manuscript's reference list.
- Teams that use an AI writer and want to catch references the model made up.
- Anyone who needs the DOI for a pile of plain-text references.

### Quick start

1. Paste references into **References**, one per line, or paste the whole list into **Or paste a whole bibliography**.
2. Add your email in **Your contact email** (recommended, see "Speed and politeness").
3. Run. Results appear in the dataset, one row per reference, in input order.

#### Sample input

```json
{
  "references": [
    "LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.",
    "Hirsch JE (2005) An index to quantify an individual's scientific research output. Proc Natl Acad Sci USA 102:16569-16572 doi:10.1073/pnas.0507655102"
  ],
  "minScore": 0.6
}
```

Or a pasted list:

```json
{ "text": "[1] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521:436-44.\n[2] Wakefield AJ, et al. Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet 1998;351:637-41." }
```

#### Sample output (real run, 21 Oct 2026, shortened)

```json
{ "index": 1, "status": "matched", "doi": "10.1038/nature14539", "title": "Deep learning", "year": 2015,
  "score": 1, "titleSimilarity": 1, "authorMatch": true, "yearMatch": true,
  "isRetractedOrCorrected": null, "source": "crossref", "matchedBy": "text",
  "inputDoi": "10.1038/nature1453999", "warnings": ["input_doi_not_found"] }
{ "index": 2, "status": "matched", "doi": "10.1016/s0140-6736(97)11096-0",
  "title": "RETRACTED: Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children",
  "score": 1, "isRetractedOrCorrected": true, "source": "crossref", "matchedBy": "text" }
{ "index": 3, "status": "not_found", "doi": null, "score": 0.395,
  "closest": { "doi": "...", "title": "Border makers: clandestine migration from Morocco", "score": 0.395 } }
{ "index": 4, "status": "error", "error": "Not a recognisable reference: it needs at least 4 words (author, title, year, source) or a DOI." }
```

In row 1 the DOI typed in the reference had a typo; the actor reports that, then finds the right paper from the text.

### Input

| Field | Meaning |
|---|---|
| `references` | List of reference strings. Up to 500 per run, together with `text`. |
| `text` | A pasted bibliography. Split on blank lines, on `[1]`, `(1)`, `1.`, `1)` numbering, or one reference per line. Indented or lower-case-start lines are joined to the previous reference. |
| `mailto` | Optional contact email for Crossref and OpenAlex (their "polite pool"). Faster, and recommended. Used only for those requests. |
| `minScore` | Lowest score reported as `probable` (default 0.6, range 0.3 to 0.95). |
| `useOpenAlex` | Ask OpenAlex when Crossref finds nothing close (default on). |
| `openalexApiKey` | Optional free OpenAlex key (secret). |

Duplicate references (same text ignoring case and spacing) are looked up once and charged once; every input row still gets its answer, and repeats carry a `duplicate_of_row_N` warning. More than 500 references, a non-text entry, an invalid email or an out-of-range `minScore` stops the run with a message naming the field.

### Output fields

| Field | Meaning |
|---|---|
| `input` | The reference as checked (squashed to one line, cut at 1000 characters with a warning). |
| `status` | `matched`, `probable`, `not_found` or `error`. |
| `doi`, `url`, `title`, `authors`, `year`, `container`, `type` | The record found (`null` or empty for `not_found` and `error`). Abstracts are never returned. |
| `score` | 0 to 1, our own score of how well the record fits your reference. For `not_found` it is the score of the closest record. |
| `titleSimilarity` | Share of the record's title words that appear in your reference. |
| `authorMatch`, `yearMatch` | `true`/`false`, or `null` when there was nothing to compare (no authors on the record, no year in your reference). |
| `isRetractedOrCorrected` | `true` only when Crossref's own `updated-by` or `update-to` fields (or a retraction/correction relation) carry a notice of type retraction, partial retraction, withdrawal, removal, correction, corrigendum, erratum or expression of concern. If you matched the notice itself, the flag is `true` and `recordType` is `notice`. Otherwise `null`: "no notice found", never "confirmed clean". Always `null` for OpenAlex rows. |
| `updateNotices` | The Crossref notices behind the flag: `direction`, `type`, `doi`. |
| `source` | `crossref` or `openalex`. |
| `recordType` | `article`, or `notice` when the matched record is itself a correction, erratum or retraction notice (it has an `update-to` field). |
| `matchedBy` | `doi` (DOI in your text resolved to a fitting record) or `text` (search). |
| `inputDoi` | The DOI found in your text, normalised (`doi.org` URLs, `doi:` prefixes and trailing punctuation are handled). |
| `closest` | On `not_found`: the nearest record seen. Use it to see why nothing was accepted. |
| `warnings` | See below. `error` holds the message when `status` is `error`, and `errorKind` says why: `unreadable` (not a usable reference), `upstream` (Crossref unreachable), `charge`, `unexpected`. |

Warnings: `input_doi_not_found` (the DOI in your text does not exist in Crossref or OpenAlex), `input_doi_resolves_to_different_work` (it exists but belongs to another paper), `crossref_unavailable`, `openalex_unavailable`, `input_truncated`.

### How matching works

For each reference the actor looks up candidates (DOI lookup, else Crossref bibliographic search, else OpenAlex search) and scores each one itself:

- **Title similarity** (weight 0.60): the share of the record's distinctive title words found in your reference. Common words (the, of, and...) are ignored.
- **Author agreement** (0.25): the record's first author's family name appears in your reference (1.0), another of its first six authors does (0.6), none does (0). A record with no author list counts as neutral (0.5) and can never be `matched`.
- **Year agreement** (0.15): a four-digit year in your reference is within one year of the record's issued, print or online year. Left out of the score when your reference has no year.

| Status | Rule |
|---|---|
| `matched` | Title similarity at least 0.90, at least one author agrees, year does not disagree, score at least 0.85. Titles with fewer than 3 distinctive words also need the first author to agree. |
| `probable` | Title similarity at least 0.60, score at least `minScore`, the year does not disagree by more than one year, and corroboration: an author agrees, or the year agrees on a title of 4 or more distinctive words. |
| `not_found` | Anything else, including a year that disagrees by more than one year. |
| `error` | The reference could not be checked: unreadable input, or Crossref could not be reached after retries. Not charged. |

A reference that contains a DOI is first checked against that DOI's record with the same rules. If the DOI does not exist or belongs to a different paper, the actor says so in `warnings` and falls back to searching by the text without the DOI.

### Accuracy, measured

Live probe on 21 Oct 2026 (Crossref public pool): 20 real references in APA, Vancouver, IEEE, Springer and journal-specific styles; 5 invented references; 3 variants of real references with the year changed by 6, 10 and 10 years.

| Test | Result |
|---|---|
| 20 real references | 19 `matched` with the correct DOI. 1 returned the **wrong DOI** as `probable` (the FAIR Guiding Principles paper, cited with an article number: Crossref did not return the right record in its top 10 and we accepted a related Nature Genetics item, 10.1038/ng.3544). 0 `not_found`. |
| 5 invented references | 0 `matched`, 0 `probable`, 5 `not_found`. |
| 3 real references with the year wrong by more than one year | 0 accepted; all 3 `not_found` (the record is shown in `closest`). |

In this sample there were 0 false `matched` and 1 wrong-DOI `probable`. Twenty-eight references is a small sample, not a guarantee. An earlier build accepted a real title and authors with a wrong year as `probable`; a year that disagrees by more than one year now always gives `not_found`, so a genuine reference with a mistyped year also lands there, with the real record in `closest`. Expect misses on books, theses, conference papers without DOIs and references with missing or abbreviated titles.

### Speed and politeness

Without a contact email the actor sends no contact in its User-Agent, and the public pool is shared with everyone else using Crossref without one, so a contact email is recommended. Crossref has two access levels. With `mailto`, the actor sends your address in the User-Agent and request (Crossref's polite pool) and checks 3 references at a time. Without it, the actor uses the public pool: one request at a time, spaced out, so 500 references can take several minutes. Rate-limit and server errors (HTTP 429, 5xx) are retried with back-off up to 4 times, then reported as `error` (not charged). OpenAlex has no retraction data and, without an API key, shares a small daily budget per network address; if it is out of budget, the Crossref answer stands and `openalex_unavailable` is added to `warnings`.

### Pricing

Pay per event: **$0.001 per reference that got a definite answer** (`matched`, `probable` or `not_found`) - event name `reference-checked`. Rows with `status: "error"` are not charged, and duplicates of a reference in the same run are charged once. Example: 200 references with 3 errors costs 197 x $0.001 = $0.197 plus Apify platform usage, which for this actor is small (plain HTTP, 256 MB). If you set a maximum charge for the run and it is reached, the run stops, says so in its status message and in the `OUTPUT` summary (`stoppedBecause: charge_limit_reached`), and the remaining references are left unchecked. If a charge itself fails, that row becomes an error row (`errorKind: charge`, result withheld), the run stops and fails with the message; this is reported separately from the charge limit.

**Run safeguards.** The run stops and fails after 8 consecutive Crossref failures (`stoppedBecause: upstream_outage`) or after 50 minutes (`time_budget`), and says how many references were not checked. Unreadable lines do not count towards the outage limit.

### Use with AI agents and MCP

Send `{ "references": ["<one reference string>", "..."], "mailto": "<contact email>" }`. Read `status` first, then `doi` and `isRetractedOrCorrected`. Treat `matched` as "a close record exists", treat `probable` as "needs a human look", and never present `not_found` as "fake": say "no close record found in Crossref or OpenAlex". The run summary (counts per status) is in the key-value store record `OUTPUT`.

### FAQ

**Does `matched` mean my reference is right?** No. It means a record with nearly the same title, a matching author and a compatible year exists. Page numbers, volume and journal are not compared.

**Why is `isRetractedOrCorrected` null and not false?** Crossref's notices are incomplete, and OpenAlex does not provide them. `null` means "no notice found", not "verified clean".

**Can I check a reference without authors or year?** Yes, but the evidence is thinner, so it is more likely to be `probable` or `not_found`.

**Is my list stored?** The actor keeps nothing beyond the run's input, dataset and summary. References and the optional email go only to Crossref and OpenAlex.

### Limits (honest)

- Coverage is whatever Crossref and OpenAlex hold. Books, theses, reports, preprints and web pages are often missing or have no usable title.
- Matching uses title words, first-author family name and year. It does not read journal, volume or pages, so two papers with near-identical titles by the same authors in the same year can be confused.
- Very short titles (fewer than 3 distinctive words) need an author match to count.
- Up to 500 references per run; each is cut at 1000 characters.
- Chinese, Japanese and Korean references are supported: unspaced text is compared as character pairs. Coverage and ranking for those languages were tested only on a recorded example, not on a live sample.
- The retraction flag depends on Crossref's own update records (which include the Retraction Watch data Crossref publishes) and may lag behind a journal's website.
- No abstracts, no full texts, no citation graph.

### Open-source credits

| Component | Licence | Checked in |
|---|---|---|
| `apify` 3.7.2 (Apify SDK) | Apache-2.0 | `node_modules/apify/LICENSE.md` |
| Transitive dependencies (187 entries in the lockfile; 169 installed without optional packages and all read; 167 of them carry a plain licence field) | MIT, ISC, BSD-2/3-Clause, Apache-2.0 | each package's `package.json` |
| `pause-stream` | MIT and Apache-2.0 dual ("Dual Licensed MIT and Apache 2") | its `LICENSE` |
| `sax` | BlueOak-1.0.0 | its `package.json` |
| `tslib` | 0BSD | its `package.json` |
| `type-fest` | MIT or CC0-1.0 | `license-mit`, `license-cc0` |
| `caniuse-lite` | CC-BY-4.0 (browser-support data, attribution: Alexis Deveria; pulled in by the SDK's crawling library, not used by this actor) | its `package.json` |
| `map-stream` | no licence field; MIT text in its `LICENCE` file | its `LICENCE` |

Data sources: Crossref REST API (metadata of scholarly works, freely reusable facts) and OpenAlex (CC0). Matching code is original: token overlap, no fuzzy-matching library.

# Actor input Schema

## `references` (type: `array`):

One reference per entry, in any citation style (APA, Vancouver, IEEE, Harvard...). A DOI inside the text is checked directly. Up to 500 per run, combined with the pasted text below.

## `text` (type: `string`):

Paste the reference list as it appears in your document. It is split on blank lines, on numbering such as \[1], (1), 1. or 1), and on one reference per line; wrapped and indented continuation lines are joined. Up to 500 references per run.

## `mailto` (type: `string`):

Sent to Crossref and OpenAlex as their 'polite pool' contact, so they can reach you if your traffic causes trouble. With it the actor checks 3 references at a time; without it, Crossref's public pool allows one at a time, so large lists run slower. Never stored by this actor.

## `minScore` (type: `number`):

Candidates scoring below this are reported as not_found. 'matched' always needs a score of at least 0.85 plus title, author and year agreement. Raise it for fewer, safer 'probable' answers.

## `useOpenAlex` (type: `boolean`):

When Crossref finds no close record, ask OpenAlex (CC0 data). OpenAlex gives no retraction information, so the retraction flag stays null for those rows.

## `openalexApiKey` (type: `string`):

Free key from openalex.org. Without one, OpenAlex shares a small daily budget per network address, and the fallback may report 'openalex_unavailable' in warnings.

## Actor input object example

```json
{
  "references": [
    "LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.",
    "Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation. 1997;9(8):1735-1780.",
    "K. He, X. Zhang, S. Ren, and J. Sun, \"Deep residual learning for image recognition,\" in Proc. IEEE CVPR, 2016, pp. 770-778.",
    "Hirsch JE (2005) An index to quantify an individual's scientific research output. Proc Natl Acad Sci USA 102:16569-16572 doi:10.1073/pnas.0507655102",
    "Watson, J. D., & Crick, F. H. C. (1953). Molecular structure of nucleic acids: a structure for deoxyribose nucleic acid. Nature, 171, 737-738."
  ],
  "minScore": 0.6,
  "useOpenAlex": true
}
```

# Actor output Schema

## `results` (type: `string`):

One row per reference: matched, probable, not_found or error.

## `summary` (type: `string`):

Counts per status and how many references were charged.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "references": [
        "LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.",
        "Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation. 1997;9(8):1735-1780.",
        "K. He, X. Zhang, S. Ren, and J. Sun, \"Deep residual learning for image recognition,\" in Proc. IEEE CVPR, 2016, pp. 770-778.",
        "Hirsch JE (2005) An index to quantify an individual's scientific research output. Proc Natl Acad Sci USA 102:16569-16572 doi:10.1073/pnas.0507655102",
        "Watson, J. D., & Crick, F. H. C. (1953). Molecular structure of nucleic acids: a structure for deoxyribose nucleic acid. Nature, 171, 737-738."
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("leadharbor/reference-list-verifier").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "references": [
        "LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.",
        "Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation. 1997;9(8):1735-1780.",
        "K. He, X. Zhang, S. Ren, and J. Sun, \"Deep residual learning for image recognition,\" in Proc. IEEE CVPR, 2016, pp. 770-778.",
        "Hirsch JE (2005) An index to quantify an individual's scientific research output. Proc Natl Acad Sci USA 102:16569-16572 doi:10.1073/pnas.0507655102",
        "Watson, J. D., & Crick, F. H. C. (1953). Molecular structure of nucleic acids: a structure for deoxyribose nucleic acid. Nature, 171, 737-738.",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("leadharbor/reference-list-verifier").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "references": [
    "LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.",
    "Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation. 1997;9(8):1735-1780.",
    "K. He, X. Zhang, S. Ren, and J. Sun, \\"Deep residual learning for image recognition,\\" in Proc. IEEE CVPR, 2016, pp. 770-778.",
    "Hirsch JE (2005) An index to quantify an individual'\''s scientific research output. Proc Natl Acad Sci USA 102:16569-16572 doi:10.1073/pnas.0507655102",
    "Watson, J. D., & Crick, F. H. C. (1953). Molecular structure of nucleic acids: a structure for deoxyribose nucleic acid. Nature, 171, 737-738."
  ]
}' |
apify call leadharbor/reference-list-verifier --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,leadharbor/reference-list-verifier"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/6gw064wDq4rsgatfh/builds/O4yMKw96I9oMfRLlF/openapi.json