Citation Checker: Find Fake References and Check DOIs

What this Actor does

Citation Checker takes a list of references and looks each one up in Crossref, DataCite and OpenAlex. For every reference it tells you whether the cited work exists and whether the citation describes it correctly:

• verified: a record matches the title, authors, year and venue.
• mismatch: the work exists, but some details in the citation are wrong. mismatchedFields lists which: title, authors, year, venue or doi.
• not_found: Crossref, DataCite and OpenAlex were all searched and none has a matching record. The citation may be made up, or the work may not be indexed (see the limits below).
• unverified: Crossref and DataCite have no match and OpenAlex could not be searched (its free daily budget was used up, or a lookup failed), so the Actor cannot tell whether the work exists. These rows are never flagged as possibly made up and are not charged.
• ambiguous: two different records fit about equally well, or only part of the citation agrees with the closest record.

Each row also carries the matched record (DOI, title, authors, year, venue, type), a retraction flag from Crossref and Retraction Watch, the open-access status and best free link from OpenAlex, the citation count, and a corrected citation in APA, BibTeX and the style you pick.

It was built for checking text written with AI tools, which often cite papers that do not exist or attach a real DOI to the wrong paper. Editors, reviewers, students and AI agents can run it on a bibliography before it goes out.

Input

Give references in any of these fields. You can combine them in one run.

Field	Default	What it does
References (references)	none	One reference per item: a citation in any style (APA, MLA, Chicago, Harvard, IEEE, Vancouver, Nature), a DOI, a doi.org link or an arXiv ID. An item can also hold a BibTeX entry.
Reference list (paste) (text)	empty	A whole bibliography: one reference per line, or references separated by blank lines. Numbered lists ([1], 1.) whose items wrap over several lines are joined. BibTeX pasted here is read as BibTeX.
BibTeX (bibtex)	empty	BibTeX entries, for example an export from Zotero, Mendeley, JabRef or Overleaf. Entries need a title or a DOI.
PDF URLs (pdfUrls)	none	Direct links to PDFs. The Actor reads the text after the last "References" or "Bibliography" heading and checks each reference. Up to 20 PDFs and 500 references per PDF.
Style of corrected citations (citationStyle)	apa	apa, mla, chicago (author-date), harvard, ieee, vancouver or bibtex. APA and BibTeX are always included as well.
Contact email for Crossref (crossrefMailto)	empty	Optional. With an email, Crossref serves your searches from its polite pool (3 a second instead of 1). Falls back to the CROSSREF_MAILTO environment variable.
OpenAlex API key (openAlexApiKey)	empty	Optional free key from openalex.org/settings/api. Falls back to the OPENALEX_API_KEY environment variable. See the limits section for why it helps.
Search OpenAlex when Crossref and DataCite have no good match (useOpenAlexSearch)	true	Turn off to use OpenAlex only for open-access data and citation counts. With it off, a reference that Crossref and DataCite do not hold is reported as not found, even though OpenAlex might have it.
Maximum references (maxReferences)	1000	References beyond this number are not checked and not charged (1 to 5,000).

Example input with a pasted list and a PDF:

{
    "text": "[1] Granovetter MS. The strength of weak ties. Am J Sociol. 1973;78(6):1360-80.\n[2] Hardin G. The tragedy of the commons. Science. 1968;162(3859):1243-8.",
    "pdfUrls": ["https://arxiv.org/pdf/1810.04805"],
    "citationStyle": "vancouver",
    "crossrefMailto": "you@your-university.edu"
}

Output

One row per reference in the dataset. This is a real row from a test run on Apify. The input was the bare DOI of the 1998 Wakefield paper, which The Lancet retracted in 2010:

{
    "index": 3,
    "input": "10.1016/S0140-6736(97)11096-0",
    "inputType": "doi",
    "sourcePdfUrl": null,
    "status": "verified",
    "confidence": 1,
    "mismatchedFields": [],
    "possibleHallucination": false,
    "notes": [],
    "matchedDoi": "10.1016/s0140-6736(97)11096-0",
    "matchedTitle": "RETRACTED: Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children",
    "matchedAuthors": ["Wakefield, AJ", "Murch, SH", "Anthony, A", "Linnell, J", "Casson, DM", "Malik, M", "Berelowitz, M", "Dhillon, AP", "Thomson, MA", "Harvey, P", "Valentine, A", "Davies, SE", "Walker-Smith, JA"],
    "matchedYear": 1998,
    "matchedVenue": "The Lancet",
    "matchedType": "journal-article",
    "matchedPublisher": "Elsevier BV",
    "matchedVolume": "351",
    "matchedIssue": "9103",
    "matchedPages": "637-641",
    "matchedUrl": "https://doi.org/10.1016/s0140-6736(97)11096-0",
    "matchSource": "crossref",
    "retracted": true,
    "retractionDate": "2010-02-06",
    "retractionNoticeDoi": "10.1016/s0140-6736(10)60175-4",
    "expressionOfConcern": false,
    "corrected": true,
    "updateNotices": [
        { "type": "correction", "date": "2004-03-06", "doi": "10.1016/s0140-6736(04)15715-2", "source": "retraction-watch" },
        { "type": "retraction", "date": "2010-02-06", "doi": "10.1016/s0140-6736(10)60175-4", "source": "retraction-watch" }
    ],
    "isOpenAccess": false,
    "oaStatus": "closed",
    "bestOaUrl": null,
    "citationCount": 3011,
    "openAlexId": "W2117847125",
    "citationStyle": "apa",
    "formattedCitation": "Wakefield, A. J., Murch, S. H., Anthony, A., Linnell, J., Casson, D. M., Malik, M., Berelowitz, M., Dhillon, A. P., Thomson, M. A., Harvey, P., Valentine, A., Davies, S. E., & Walker-Smith, J. A. (1998). RETRACTED: Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. The Lancet, 351(9103), 637–641. https://doi.org/10.1016/s0140-6736(97)11096-0",
    "apaCitation": "Wakefield, A. J., Murch, S. H., Anthony, A., Linnell, J., Casson, D. M., Malik, M., Berelowitz, M., Dhillon, A. P., Thomson, M. A., Harvey, P., Valentine, A., Davies, S. E., & Walker-Smith, J. A. (1998). RETRACTED: Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. The Lancet, 351(9103), 637–641. https://doi.org/10.1016/s0140-6736(97)11096-0",
    "bibtex": "@article{wakefield1998retracted,\n  author = {Wakefield, AJ and Murch, SH and Anthony, A and Linnell, J and Casson, DM and Malik, M and Berelowitz, M and Dhillon, AP and Thomson, MA and Harvey, P and Valentine, A and Davies, SE and Walker-Smith, JA},\n  title = {{RETRACTED: Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children}},\n  journal = {The Lancet},\n  year = {1998},\n  volume = {351},\n  number = {9103},\n  pages = {637--641},\n  doi = {10.1016/s0140-6736(97)11096-0}\n}",
    "alternatives": [],
    "parsedTitle": null,
    "parsedFirstAuthor": null,
    "parsedYear": null,
    "parsedVenue": null,
    "checkedAt": "2026-10-26T21:21:10.334Z",
    "error": null
}

Other fields worth knowing:

• confidence is the confidence in the status, from 0 to 1. For verified and mismatch it is the match score. For not_found it is higher when even the closest record is far from the citation.
• possibleHallucination is true for not_found references that look like articles, for references whose DOI belongs to a different work, and for mismatches in three or more fields.
• alternatives lists up to three of the closest records (DOI, title, first author, year, venue, titleSimilarity, score) for not_found, ambiguous and mismatch rows.
• parsedTitle, parsedFirstAuthor, parsedYear and parsedVenue show what the Actor read from your citation, which helps when a result looks wrong.
• notes explains special cases: a preprint matched instead of the published version, a cited DOI that belongs to another paper, or OpenAlex search being unavailable.

Statuses that are not charged: unverified (no match in Crossref or DataCite, and OpenAlex could not be searched), duplicate (the same reference appeared earlier in the run), invalid (too short to be a reference), error (the indexes could not be reached), not_allowed (a PDF URL that is private or disallowed by robots.txt) and skipped (the run reached your cost limit).

The run also saves these records in its key-value store:

• OUTPUT: counts per status, the list of possibly made-up references with the reason, retracted references, mismatches, PDF results and the number of charged references.
• BIBLIOGRAPHY: the verified, mismatched and ambiguous references as corrected citations in your chosen style.
• BIBTEX: the same references as a .bib file.

How matching works

1. The Actor reads the citation and pulls out the DOI, arXiv ID, years, first author, title and venue. It handles APA, MLA, Chicago, Harvard, IEEE, Vancouver and Nature styles, and the author-year and numbered reference lists found in PDFs.
2. A DOI or arXiv ID in the citation is looked up directly. Then the citation is searched in Crossref (query.bibliographic). If Crossref has no convincing match, the title is searched in DataCite (arXiv, Zenodo and other repositories, exact phrase first and then with typo tolerance) and in OpenAlex.
3. Each candidate gets a score from title similarity (60%), author surnames (20%), year (10%) and venue (10%). Titles are compared after removing accents, punctuation, case and HTML tags, with an edit-distance ratio, so typos still match. Years within one of each other count as the same, which covers online-first and print dates. Journal abbreviations (N Engl J Med, Proc Natl Acad Sci USA) and conference short names (NeurIPS, CVPR, ACL) are matched against the full names.
4. A preprint and its published version are treated as one work. If you cite the journal, the published version is returned; if you cite arXiv, the preprint is, with a note about the published version.
5. Retraction data comes from the updated-by field of the Crossref record, which includes notices from publishers and from the Retraction Watch database. OpenAlex's is_retracted flag is used as a second source.

Accuracy

The numbers depend on whether OpenAlex search is available (see the limits section).

Blind test, 26 October 2026. 30 references the matcher was never tuned on: 16 real references sampled at random from the reference lists of three recent papers (forest-bird ecology in Scientific Reports, 2025; family economics in the Journal of Population Economics, 2026; protein language models in Nature Machine Intelligence, 2025), 6 more of their references with one detail changed (year, journal, first author, DOI or a title word), and 8 made-up references in the same fields, most under real authors' names. The set is in test/eval/blind-review-2026-10-26.json.

Condition	Real references verified	Made-up flagged (precision / recall)	Wrong detail found (precision / recall)	Real references flagged as possibly made up
Run on Apify, OpenAlex search available	10 of 16	0.73 / 1.00	0.63 / 0.83	1 of 16
Local run, OpenAlex search budget used up	9 of 16	0.50 / 0.13	0.57 / 0.67	0 of 16

With OpenAlex search available:

• One real reference was flagged as possibly made up: the 2023 UniProt database update, cited as "The UniProt Consortium UniProt: ..." without a full stop after the group name, so the group name was read as part of the title and older yearly updates ranked higher.
• The other two flags were the reference with a changed title word (reported not_found rather than as a title error) and the one with another paper's DOI, which is flagged on purpose.
• Three correct citations came back mismatch: a discussion paper matched to its later journal version (venue), a 1996 journal article matched to its 1993 working paper (year) and an ICLR 2019 paper matched to its 2017 arXiv preprint (year).
• A web page and a book cited without its DOI came back not_found, with a note and without the flag.

Without OpenAlex search, 13 of the 30 came back unverified, including 7 of the 8 made-up references; the one flagged had another paper's DOI.

Sets used while building the matcher (test/eval/references.json, 41 references, and test/eval/holdout.json, 25). They were used for tuning, so they are not blind. With OpenAlex search unavailable, the current version returns verified for all 38 real and retracted references, finds all 10 wrong details and flags all 6 retractions. 4 of the 18 made-up references are flagged (those with a fake or borrowed DOI); the other 14 come back unverified. One real reference cited with another paper's DOI is flagged on purpose.

In an earlier development run on real reference lists from three papers (BERT, "Attention Is All You Need" and a PLOS ONE stroke study; 135 references) in a run on Apify with OpenAlex search available, 126 came back verified, 4 mismatch, 2 ambiguous and 3 not_found. Two of the 135 were flagged as possibly made up, both real: a dataset ("Quora question pairs") and a magazine article that none of the indexes holds. The four mismatches were a title abbreviated in the citation, a shortened title, a symposium paper whose record names another venue, and a book chapter whose record carries a later date. That run took 4 minutes 47 seconds, including reading the three PDFs.

Pricing

You pay per event:

Event	Price
Actor start	$0.002 per run
Reference checked	$0.004 per reference

A reference is charged when it gets an answer: verified, mismatch, not_found or ambiguous. References that could not be verified, duplicates, invalid lines, failed lookups, refused PDFs and references skipped at your cost limit are free. Reading a PDF is free; its references are charged like any others. Checking a 50-reference bibliography costs $0.002 + 50 × $0.004 = $0.202.

If a run reaches the maximum cost you set, the remaining references are marked skipped and the run finishes normally.

Limits

• Made-up references are only flagged when OpenAlex can be searched. Many real works (conference papers without DOIs, older articles, small journals) are only in OpenAlex. Without an OpenAlex key, OpenAlex search uses a free daily budget shared by everyone on the same IP address (about 100 searches a day), which often runs out on shared servers. When it is unavailable, a reference that Crossref and DataCite do not hold comes back unverified instead of not_found: it is not flagged and not charged, and its notes say why. Only a cited DOI that is not registered, or that belongs to another work, is still flagged. In the blind test above, 7 of 8 made-up references came back unverified this way. A free OpenAlex key avoids this.
• Books, reports, theses, datasets and web pages are often not indexed. A not_found reference that looks like one of these gets a note and is not flagged as possibly made up.
• Speed. Without a contact email, Crossref allows 1 search a second; in the test run on Apify that came to about 40 references a minute. With crossrefMailto set, Crossref allows 3 searches a second.
• PDFs need a text layer (scanned PDFs are not read), must be direct links to the PDF file, must be allowed by the site's robots.txt, and must be under 40 MB. A PDF that needs more than about 60% of the run's memory to read, or more than about 3 minutes, is stopped and reported as error; run again with more memory for very large files. Private and local addresses are refused, including through redirects. Sites that block automated visitors are reported as error and not charged.
• Titles in corrected citations keep the capitalisation the publisher registered; APA's sentence case is not applied automatically.
• Metadata errors in the indexes carry through. For example, a book chapter reprinted years later can show the reprint year, which produces a year mismatch.

Data sources and their terms

• Crossref REST API. Since 1 December 2025 the public pool allows 5 single-record and 1 list request a second with 1 at a time; the polite pool (requests with a mailto) allows 10 and 3 a second with 3 at a time (Crossref blog, access and authentication). Crossref describes its metadata as "openly shared with the global scholarly community through our free APIs" (tips for using the REST API). Retraction Watch data is in the REST API, marked with source: retraction-watch, and updated every working day (Crossref: Retraction Watch).
• OpenAlex. CC0 data (api.openalex.org). 10 requests a second without a key and 30 with a free key (authentication). Single-work lookups are free, list requests cost $0.0001 and searches $0.001 of the daily budget (quick reference); without a key the budget is $0.10 a day per IP address, as reported in the API's rate-limit headers.
• DataCite REST API for arXiv and repository DOIs, with a limit of 3,000 requests per 5 minutes per IP address (DataCite support). The Actor stays at 1 request a second.
• doi.org handle API, to tell a DOI that is not registered at all from one whose agency has no metadata.

The Actor identifies itself as PlainsourceCitationChecker in its User-Agent and backs off when an API returns HTTP 429.

FAQ

Is a not_found reference always made up? No. It means none of the three indexes has a matching record. Check the alternatives and notes: a near miss in alternatives often points to the real paper, and works without a DOI may need an OpenAlex key to be found.

Why is a correct citation reported as mismatch? The indexes sometimes hold a different version: a reprint with a later year, a title registered without its subtitle, or a conference paper registered under its journal version. notes and the matched fields show what was compared.

Can I check DOIs only? Yes. Put one DOI per item in References. A registered DOI comes back verified with its record; an unregistered one comes back not_found.

Which email should I give Crossref? Any address where you can be reached about your requests. Crossref uses it to sort traffic into its faster pool and to contact you if your requests cause problems.

Can an AI agent call it? Yes, through the Apify API or the Apify MCP server. For example, with the API client: client.actor("fernandosord/citation-checker-doi-validator").call({ references: ["10.1038/nature14539"] }). The possibleHallucination field and the OUTPUT summary are meant for this use.