# Citation Checker: Find Fake, Wrong & Retracted References (`reliable_tools/citation-checker`) Actor Citation checker for reference lists: finds fake (hallucinated), wrong and retracted references. Checks DOIs, PMIDs, URLs, BibTeX and free-text citations against Crossref, PubMed and OpenAlex. - **URL**: https://apify.com/reliable_tools/citation-checker.md - **Developed by:** [Reliable Tools](https://apify.com/reliable_tools) (community) - **Categories:** AI, Developer tools, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $2.55 / 1,000 reference resolveds This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Citation Checker: Find Fake, Wrong & Retracted References This citation checker validates a whole reference list in one run and finds the references that are fake (invented, for example by an AI writing tool), wrong (wrong year, wrong author, wrong journal, a DOI that points to another paper) or retracted. Paste citations in any style, DOIs, PubMed IDs, arXiv IDs, URLs or a BibTeX file. Every reference is checked against Crossref, PubMed, OpenAlex, DataCite and doi.org, and you get the official record, the details that do not match, any retraction or correction notice, an open-access link and a probability that the citation is wrong. ### What does Citation Checker do? For each reference it answers four questions: 1. **Does this work exist?** The reference is resolved by its identifier (DOI, PMID, PMC ID, arXiv ID) or searched by its text in Crossref, PubMed, OpenAlex and DataCite. A DOI that doi.org has never registered, a PMID that PubMed does not know, or a URL that returns 404 is reported as such. 2. **Is the citation correct?** The citation as you wrote it is compared field by field with the official record: title, authors (first author in particular), year, journal or venue, volume and first page. You get the list of wrong details with the cited value and the record value, plus a corrected citation built from the record. 3. **Was it retracted or corrected?** Retractions, withdrawals, expressions of concern and corrections come from Crossref (which includes the Retraction Watch database) and from PubMed. 4. **How sure is it?** An AI match score gives the probability that the matched record is the work you meant, and the probability that your citation is fabricated or wrong. The final status is decided by fixed rules on top of these numbers, so the same input always gets the same treatment. For the AI match score, the citation text you entered and the candidate records found in the sources are sent to an external AI service, which only answers fixed comparison questions (same work, which details agree). References given only as a DOI, PMID, PMC ID or arXiv ID that resolve directly skip this step. Without the AI step the Actor falls back to rule-based scores. ### Who uses it? - Researchers, students and supervisors checking a thesis or manuscript bibliography before submission. - Journal editors and publishers screening submitted manuscripts for fake or retracted references. - Medical writers and regulatory teams verifying Vancouver-style references against PubMed. - Law firms and policy teams checking briefs and reports for fabricated references to scholarly articles. - Teams that build AI research assistants, deep-research agents and RAG pipelines and must not ship a fake DOI. ### What data you get | Field | What it is | |---|---| | `status` | `verified`, `mismatch` (the work exists but details are wrong), `ambiguous` (possible match only), `not_found`, `invalid_input`, `source_error` | | `exists` | `yes`, `no` or `unclear` | | `pFabricatedOrWrong` | Probability (0 to 1) that the citation is fabricated or has wrong details | | `sameWorkProbability` | Probability that the matched record is the work you cited | | `mismatchFields`, `mismatches` | Wrong details (`title`, `authors`, `year`, `journal`, `volume`, `pages`, `doi`, `pmid`, `url`, `identifier`) with cited and official values | | `retracted`, `integrityNotice`, `integrityNotices` | Retraction or withdrawal flag, the most serious notice, and all notices with notice DOI, date and source | | `doi`, `pmid`, `pmcid`, `openalexId`, `arxivId` | Resolved identifiers | | `record` | The official record: title, authors, year, journal, volume, issue, pages, type, publisher, citation count | | `openAccessUrl`, `openAccessStatus`, `openAccessLicense` | Free full-text link when one exists | | `suggestedCitation` | A corrected citation built from the official record | | `closestRecord` | For not found references: the closest record and how likely it is the same work | | `parsed` | The fields read from your citation (authors, year, title, journal, volume, pages, identifiers) | | `checks` | The exact comparisons made by code (title similarity, first author, year, journal) | | `recommendation`, `summary` | What to do with the reference, and one sentence for a quick review | ### How to check references step by step 1. Open the Actor and put your references into **References**, one per line. Any citation style works, and so do DOIs, PMIDs, PMC IDs, arXiv IDs and links. 2. Or paste a whole bibliography or a `.bib` file into **Paste a reference list or BibTeX**. 3. Click **Start**. The sample input finishes in well under a minute. 4. Sort by `pFabricatedOrWrong` or filter on `status` and `retracted`, then download as JSON, CSV or Excel. ### What do the status values mean? - `verified`: the work exists and your citation agrees with the record. Check `retracted` anyway. - `mismatch`: the work exists, but at least one detail in your citation is wrong. `mismatches` says which, and `suggestedCitation` gives the corrected reference. - `ambiguous`: a similar record exists but it is not clear it is the same work. Check `closestRecord` or the record by hand. - `not_found`: no matching work in the sources, or the identifier does not exist. These are the likely fabricated references. - `invalid_input`: the line has nothing to check. `source_error`: a source did not answer; please retry. ### Example output ```json { "reference": "2", "input": "LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2017;521(7553):436-444.", "status": "mismatch", "exists": "yes", "pFabricatedOrWrong": 0.95, "sameWorkProbability": 0.95, "mismatchFields": ["year"], "mismatches": [{"field": "year", "cited": 2017, "record": 2015}], "retracted": false, "integrityNotice": null, "doi": "10.1038/nature14539", "pmid": "26017442", "openalexId": "W2919115771", "record": {"title": "Deep learning", "authors": ["Yann LeCun", "Yoshua Bengio", "Geoffrey Hinton"], "year": 2015, "journal": "Nature", "volume": "521", "issue": "7553", "pages": "436-444"}, "openAccessUrl": "https://hal.science/hal-04206682", "suggestedCitation": "LeCun, Y., Bengio, Y., Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444. https://doi.org/10.1038/nature14539", "recommendation": "The work exists but the citation has wrong details: year. Use the suggested citation." } ``` ### How to use Citation Checker through the API Run it from your own code or an AI agent with the Apify API or the Apify client: ```python from apify_client import ApifyClient client = ApifyClient("") run = client.actor("reliable_tools/citation-checker").call(run_input={ "references": ["10.1038/nature14539", "Doe J. Quantum cheese fermentation improves memory. J Imag Sci. 2019;12:45-52."] }) for row in client.dataset(run["defaultDatasetId"]).iterate_items(): print(row["status"], row["pFabricatedOrWrong"], row["summary"]) ``` The run summary with counts per status is stored in the key-value store under `SUMMARY`. ### How much does it cost to check references? Pay per result, no subscription: | Event | Price | When | |---|---|---| | Reference resolved | $0.003 | A reference matched to a real work, with identifiers, official record and open-access link | | Integrity analysis | $0.007 | Mismatch check, retraction and correction notices, corrected citation and AI score for a resolved reference | | Actor start | $0.00005 | Once per run | A checked reference costs $0.01 in total, or $0.003 with **Integrity analysis** turned off. References that are not found, empty lines and references that could not be checked because a source did not answer are free. A 100-reference bibliography with every reference found costs about $1. ### Sources and licences | Source | Used for | Terms | |---|---|---| | Crossref REST API | DOI metadata, bibliographic search, retraction and correction notices | Bibliographic metadata is treated by Crossref as facts, free to reuse; abstracts are not output. Requests use the polite pool with a contact address and at most 3 requests at a time. | | Retraction Watch database (via Crossref) | Retractions, expressions of concern, corrections | Made openly available by Crossref; the source is cited in every notice (`source`: Retraction Watch via Crossref). | | OpenAlex | IDs, open-access links, retraction flag, title search | CC0 | | PubMed E-utilities (NCBI, U.S. National Library of Medicine) | PMID records, PubMed retraction flags, title search, PMC ID conversion | Citation data from PubMed, courtesy of the U.S. National Library of Medicine. Used under the NCBI usage policy: tool name and contact are sent, and all NCBI requests in a run are paced to at most 3 per second (10 with your own key). | | DataCite REST API | arXiv, Zenodo and other DataCite DOIs | Openly available DOI metadata; requests are paced below DataCite's published rate limit | | doi.org (DOI Handle API and content negotiation) | Whether a DOI exists at all, metadata for non-Crossref DOIs | Public resolver | Cited web pages (plain URLs) are opened once, only when the site's robots.txt allows it, to read the page's own citation meta tags. No logins, no paywalled content, no proxies. Request pacing follows each source's published per-second limits within a run. Separate runs that execute at the same time are paced independently. Lookups by identifier are cached for up to 7 days so repeated checks send fewer requests. NCBI and NLM do not guarantee the accuracy, completeness or currency of PubMed data; NCBI's disclaimer and copyright notice apply (https://www.ncbi.nlm.nih.gov/home/about/policies/). This Actor is not affiliated with or endorsed by Crossref, Retraction Watch, OpenAlex, NCBI, NLM, NIH, DataCite or the DOI Foundation. **Personal data.** Records list the authors of each work as the sources publish them (up to 10 names per record). Do not put personal information other than citations into the input. ### FAQ **Can it detect citations made up by AI chatbots and writing tools?** Yes. That is the main use. An invented reference either has no matching record in any source (`not_found`), carries a DOI that doi.org never registered, or mixes a real title with the wrong authors, year or journal (`mismatch`). Each row has a probability so you can sort the worst first. **Does "not found" prove a reference is fake?** No. It means no matching record was found in Crossref, PubMed, OpenAlex or DataCite. Books, reports, theses, court documents and some conference papers have no DOI and may exist elsewhere. Check those by hand; the `closestRecord` field helps. **How does it find retracted papers?** Through the retraction notices that Crossref publishes for every DOI, which include the Retraction Watch database, plus the "Retracted Publication" flag in PubMed and OpenAlex. **Which citation styles work?** APA, Vancouver, AMA, MLA, Harvard, Chicago, IEEE and short forms such as `Smith J et al. 2021 Nature 590:123`. Text in other languages works as long as the title is close to the record. **Can I check a BibTeX file?** Yes. Paste the `.bib` content into **Paste a reference list or BibTeX**. The BibTeX key becomes the `reference` of each row. **Is a preprint the same as the published paper?** The check treats them as the same work. If only the preprint has a DOI (common for conference papers on arXiv), the row is matched to the preprint and the `note` says the published venue could not be confirmed. **Something broke or a field is missing?** Open an issue on the Issues tab and we'll look at it. ### About Reliable Tools This Actor is made by [Reliable Tools](https://apify.com/reliable_tools). We build tools that return clean data from official sources plus an AI score with a probability, so you can filter and act on the results straight away. Our focus is compliance, verification and research data. If a field is missing or a reference doesn't check the way you expect, open an issue on the Issues tab. We read every one and usually reply within a day. You can find our other Actors on our [profile page](https://apify.com/reliable_tools). # Actor input Schema ## `references` (type: `array`): One reference per line: a full citation in any style (APA, Vancouver, MLA, Harvard, IEEE), a DOI, a PMID, a PMC ID, an arXiv ID or a URL. The prefill mixes a correct reference, one with a wrong year, a retracted paper and an invented one. ## `referenceText` (type: `string`): Optional: paste a whole bibliography (one reference per line, numbering like \[1] or 1. is removed) or the content of a .bib file. BibTeX entries are detected automatically. ## `structuredReferences` (type: `array`): Optional: one object per reference with any of "reference" (your ID), "citation" (text), "doi", "pmid", "pmcid", "arxivId", "url", "title", "authors" (list or text), "year", "journal", "volume", "pages". Useful when your data is already split into fields. ## `integrityAnalysis` (type: `boolean`): Adds the field-by-field mismatch check (title, authors, year, journal, volume, pages), retraction, correction and expression of concern notices, a suggested corrected citation and the AI match score. Turn off to only resolve references (exists, IDs, record, open-access link) at the lower price. ## `checkUrls` (type: `boolean`): For plain web links (not DOI, PubMed or arXiv links): checks that the page exists and reads the DOI and title from its citation meta tags. Pages that robots.txt closes to bots are not opened. ## `ncbiApiKey` (type: `string`): Optional: your own free NCBI E-utilities key. It raises the PubMed rate limit from 3 to 10 requests per second, which speeds up large batches. ## `openAlexApiKey` (type: `string`): Optional: your own OpenAlex key. Without it, OpenAlex title search runs within the small free daily allowance and the Actor relies on Crossref, PubMed and DataCite search when that is used up. ## `maxReferences` (type: `integer`): Stops after this many references. ## Actor input object example ```json { "references": [ "LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.", "LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2017;521(7553):436-444.", "Wakefield AJ, Murch SH, Anthony A, et al. Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet. 1998;351(9103):637-641.", "Doe J, Miller K. Quantum cheese fermentation improves working memory in elderly mice. J Imag Sci. 2019;12(3):45-52.", "10.1038/s41586-021-03819-2", "PMID: 26017442" ], "structuredReferences": [], "integrityAnalysis": true, "checkUrls": true, "maxReferences": 1000 } ``` # Actor output Schema ## `results` (type: `string`): Dataset in the references view. ## `summary` (type: `string`): Counts per status and number of retracted references. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "references": [ "LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.", "LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2017;521(7553):436-444.", "Wakefield AJ, Murch SH, Anthony A, et al. Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet. 1998;351(9103):637-641.", "Doe J, Miller K. Quantum cheese fermentation improves working memory in elderly mice. J Imag Sci. 2019;12(3):45-52.", "10.1038/s41586-021-03819-2", "PMID: 26017442" ], "structuredReferences": [] }; // Run the Actor and wait for it to finish const run = await client.actor("reliable_tools/citation-checker").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "references": [ "LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.", "LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2017;521(7553):436-444.", "Wakefield AJ, Murch SH, Anthony A, et al. Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet. 1998;351(9103):637-641.", "Doe J, Miller K. Quantum cheese fermentation improves working memory in elderly mice. J Imag Sci. 2019;12(3):45-52.", "10.1038/s41586-021-03819-2", "PMID: 26017442", ], "structuredReferences": [], } # Run the Actor and wait for it to finish run = client.actor("reliable_tools/citation-checker").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "references": [ "LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.", "LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2017;521(7553):436-444.", "Wakefield AJ, Murch SH, Anthony A, et al. Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet. 1998;351(9103):637-641.", "Doe J, Miller K. Quantum cheese fermentation improves working memory in elderly mice. J Imag Sci. 2019;12(3):45-52.", "10.1038/s41586-021-03819-2", "PMID: 26017442" ], "structuredReferences": [] }' | apify call reliable_tools/citation-checker --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,reliable_tools/citation-checker" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/ChYSnWVQSNSqFjxIt/builds/wTX6e4wxeVgtFer3S/openapi.json