Reference Checker: Fake Citation & Retraction Detector

Paste a reference list. In one run, find out which citations are real, which are fabricated or miscited, and which point to retracted papers.

AI writing tools invent references that look perfect, with plausible authors, a real journal and a believable DOI. Retracted papers keep getting cited years after they were withdrawn. Checking a bibliography by hand takes hours. Reference Checker verifies every entry against Crossref (150M+ scholarly records), the Retraction Watch database (via Crossref) and OpenAlex, then tells you exactly what is wrong.

• ✅ Catches hallucinated citations from ChatGPT, Gemini, Claude and other AI drafting tools
• 🚩 Flags retractions, corrections and expressions of concern with notice DOIs and dates
• 🧩 Finds miscited details: wrong year, authors, journal or title, or a DOI that points to a different paper, with the correct DOI when the real paper can be found
• 🔓 Links a free, legal full-text copy when one exists
• 💸 $0.01 per reference. No subscription. A 50-reference bibliography costs $0.50.

Who it's for

• Researchers and grad students: check a thesis, dissertation or manuscript bibliography before submission.
• Editors, peer reviewers and journals: screen submissions for fabricated or retracted references at intake.
• Research-integrity teams and librarians: run objective, repeatable citation audits.
• Students and instructors checking AI drafts: confirm that every source an AI tool suggested actually exists.
• Content teams and publishers: catch hallucinated sources before an article goes live.
• AI agents and RAG pipelines: verify sources programmatically through the Apify API or MCP.

How to use it

1. Click Try for free and paste your reference list into Bibliography. You can also add bare DOIs or paper titles.
2. Click Start. A short list finishes in seconds, and 100 references take about a minute.
3. Open the Problems only view to see just the references that need attention, or All references for the full audit.
4. Export to CSV, Excel, JSON or HTML, or pull the results through the API.

Input example

Accepted styles: APA, MLA, Chicago, Harvard, Vancouver/NLM and IEEE, one per line, numbered ([1], 1.), or separated by blank lines. DOIs can be bare (10.1038/nature14539), doi: prefixed, or https://doi.org/... URLs.

{
  "bibliography": "Wakefield, A. J., et al. (1998). Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. The Lancet, 351(9103), 637-641.\nLeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.\nJohnson, M., & Patel, R. (2021). Neural lattice frameworks for quantum basket weaving in arid climates. Journal of Applied Cognitive Robotics, 14(2), 112-130.\nLeCun, Y., Bengio, Y., & Hinton, G. (2012). Deep learning. Science, 521(7553), 436-444.",
  "dois": ["10.1038/nature12968"],
  "titles": ["Attention is all you need"],
  "minMatchScore": 80,
  "checkOpenAccess": true
}

Options: minimum match score (default 80), open-access lookup on or off, top candidate matches for auditing, a per-run cap (maxReferences), and an optional contact email for Crossref's faster "polite" pool.

Output example

One dataset item per reference. This is the real result of the input above:

#	Reference	Verdict	Integrity	What it tells you
1	Wakefield et al. (1998), The Lancet	verified	🚩 retracted	The paper exists but was retracted. Notice DOI and date included
2	LeCun, Bengio & Hinton (2015), Nature	verified	none	Clean. DOI 10.1038/nature14539
3	Johnson & Patel (2021), "Neural lattice frameworks..."	not-found	n/a	No matching record anywhere. Likely fabricated
4	LeCun et al. (2012), Science	found-with-major-discrepancies	none	Real paper, miscited: year is 2015, not 2012, and the journal is Nature, not Science
5	DOI 10.1038/nature12968	verified	🚩 retracted	Retracted article (STAP stem-cell paper)
6	Title: "Attention is all you need"	verified	none	Warning: several records share this title. Add authors or a year to pin it down

Full JSON for one item:

{
  "index": 1,
  "status": "FIX",
  "input": "Wakefield, A. J., et al. (1998). Ileal-lymphoid-nodular hyperplasia ...",
  "parsed": {"authors": ["Wakefield"], "year": 1998, "title": "Ileal-lymphoid-nodular hyperplasia ...", "journal": "The Lancet", "doi": null},
  "found": true,
  "verdict": "verified",
  "matchedDoi": "10.1016/s0140-6736(97)11096-0",
  "matchScore": 1.0,
  "mismatches": [],
  "integrity": {
    "status": "retracted",
    "isRetracted": true,
    "notices": [{"type": "retraction", "noticeDoi": "10.1016/s0140-6736(10)60175-4", "date": "2010-02-06", "source": "retraction-watch"}],
    "statusSources": ["openalex", "retraction-watch"]
  },
  "isOpenAccess": false,
  "openAccessUrl": null,
  "warnings": []
}

Status (the Panda Federation triad): OK (verified, safe to cite) · CHECK (found, but read the warnings or notices first) · FIX (not found, miscited, a DOI that belongs to another paper, or retracted: don't cite it as written).

Verdicts: verified · found-with-minor-discrepancies · found-with-major-discrepancies · uncertain (close but not confident, so check it manually) · not-found (likely fabricated or not indexed) · doi-not-found · too-short · error.

Integrity status: retracted · expression-of-concern · corrected · none.

The dataset has two ready-made views, All references and Problems only. A run summary with counts of verified, not found, mismatched, retracted and corrected references is saved to the key-value store as SUMMARY.

Monitor mode: get alerted when a paper is retracted

Papers get retracted, corrected or flagged with an expression of concern years after they were cited. Monitor mode watches a list for you.

1. Set Mode to monitor. Add your DOIs, an ORCID iD (every DOI on that public ORCID record plus Crossref works tagged with it), or both.
2. Optionally set a Monitor name, which is the key-value store in your account that remembers the last run. Use one name per list.
3. Save it as a task and add a Schedule in Apify Console (for example weekly or monthly). Add a webhook or email notification if you want to be pinged.

The first run saves a baseline. Each later run outputs:

• CHANGES (key-value store): only the new changes since the last run, such as none -> retracted or a new correction notice. It also lists newly added DOIs that are already flagged.
• The dataset (view Monitor: status table): the full status table for every DOI, with a changeType column (baseline, unchanged, escalated, de-escalated, new-notice, added, error).

{ "mode": "monitor", "dois": ["10.1016/S0140-6736(97)11096-0", "10.1038/nature14539"], "monitorName": "my-lab-papers" }

Monitor runs use the same price: $0.01 per DOI checked per run. Use Max references per run to cap cost for large ORCID records.

Pricing

$0.01 per reference checked. Platform usage is included, there is no subscription, and you pay only for references actually checked.

Bibliography size	Cost
30 references (a typical article)	$0.30
100 references (a thesis chapter)	$1.00
1,000 references (a journal issue or large review)	$10.00

Apify's free plan includes monthly platform credit, so you can try it on your own list first.

How accurate is it?

On a test set of 48 references (APA, MLA, Vancouver, IEEE, DOIs and bare titles), including 9 retracted papers, 3 expressions of concern, 3 corrected papers, 14 fabricated citations and 3 deliberately wrong citations, it classified 48/48 correctly as found or not found, identified the correct integrity status for 33/33, and flagged 3/3 planted discrepancies. 21 of the 48 cases were held out and not used while tuning.

Limits

• "Not found" means not found in Crossref or OpenAlex. It is not proof of fabrication. Books, grey literature, very new papers, some conference proceedings and non-English works may be missing. Always review not-found items.
• Bare-title lookups for very common titles can be ambiguous. The Actor adds a warning, and adding authors or a year resolves it.
• Retraction data is only as complete as Crossref, Retraction Watch and OpenAlex. Coverage of corrections and expressions of concern is thinner than coverage of retractions.
• It checks that a paper exists and is cited correctly, not whether it supports the claim you cite it for.

FAQ

How do I detect fake citations from ChatGPT or other AI tools? Paste the AI-generated reference list into the Bibliography field and run it. Fabricated entries come back as not-found. Real-looking entries with the wrong details come back as found-with-major-discrepancies, with each mismatch explained. A real DOI pasted onto an invented title gets status FIX and a warning naming the paper the DOI really belongs to (in doiResolvesTo). This works for Crossref DOIs and for DOIs registered elsewhere, such as arXiv, Zenodo and other DataCite DOIs.

Does it check arXiv preprints? Yes. A reference that only gives an arXiv ID (for example arXiv:1412.6980) is checked through its arXiv DOI, so real preprints like Adam or Attention Is All You Need verify instead of coming back as not found.

How do I check whether a paper has been retracted? Add its DOI or its full reference. The integrity field shows retracted, expression-of-concern or corrected, with the notice DOI, the date and the source.

Can I check thousands of references? Yes. Raise maxReferences and add a contact email to use Crossref's faster pool. References are processed in batches, and you're billed only for those checked.

Is my bibliography stored? Only in your own Apify run storage, under your retention settings. It is not shared anywhere else.

Can I call it from code or an AI agent? Yes. Use the Apify API, the Python or JavaScript clients, scheduled runs and webhooks, or Apify's MCP server for AI agents.

Does it scrape publisher websites? No. It only calls official public APIs, follows their rate limits and etiquette, and caches repeated lookups.

Found a wrong result? Open an issue on the Issues tab with the reference text. Mismatched parses are the fastest thing to fix.

Data sources and credits

• Crossref REST API: bibliographic metadata.
• Retraction Watch database, made openly available by Crossref. Please cite Retraction Watch if you publish results based on its retraction data.
• OpenAlex: open-access locations, retraction flags and non-Crossref DOIs such as arXiv (CC0).

---

Built by Panda Federation

Self-healing, local-first AI systems and data tools that run themselves.

Panda Federation builds small, dependable tools that do one job well and keep doing it. Reference Checker is our first public module, and more data and research-integrity tools are on the way.

More from Panda Federation on Apify · GitHub