# Reference Checker: Fake Citation & Retraction Detector (`pandadad0214/reference-checker`) Actor Paste a bibliography, DOIs, or titles. Verifies each citation exists (Crossref), flags author/year/journal/title mismatches, retractions, corrections and expressions of concern (Crossref + Retraction Watch), and finds open-access links (OpenAlex). - **URL**: https://apify.com/pandadad0214/reference-checker.md - **Developed by:** [Panda Federation](https://apify.com/pandadad0214) (community) - **Categories:** AI, Education, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Reference Checker: Fake Citation & Retraction Detector **Paste a reference list. In one run, find out which citations are real, which are fabricated or miscited, and which point to retracted papers.** AI writing tools invent references that look perfect, with plausible authors, a real journal and a believable DOI. Retracted papers keep getting cited years after they were withdrawn. Checking a bibliography by hand takes hours. **Reference Checker** verifies every entry against **Crossref** (150M+ scholarly records), the **Retraction Watch** database (via Crossref) and **OpenAlex**, then tells you exactly what is wrong. - ✅ **Catches hallucinated citations** from ChatGPT, Gemini, Claude and other AI drafting tools - 🚩 **Flags retractions, corrections and expressions of concern** with notice DOIs and dates - 🧩 **Finds miscited details**: wrong year, authors, journal or title, or a DOI that points to a different paper, with the correct DOI when the real paper can be found - 🔓 **Links a free, legal full-text copy** when one exists - 💸 **$0.01 per reference.** No subscription. A 50-reference bibliography costs $0.50. ### Who it's for - **Researchers and grad students**: check a thesis, dissertation or manuscript bibliography before submission. - **Editors, peer reviewers and journals**: screen submissions for fabricated or retracted references at intake. - **Research-integrity teams and librarians**: run objective, repeatable citation audits. - **Students and instructors checking AI drafts**: confirm that every source an AI tool suggested actually exists. - **Content teams and publishers**: catch hallucinated sources before an article goes live. - **AI agents and RAG pipelines**: verify sources programmatically through the Apify API or MCP. ### How to use it 1. Click **Try for free** and paste your reference list into **Bibliography**. You can also add bare **DOIs** or **paper titles**. 2. Click **Start**. A short list finishes in seconds, and 100 references take about a minute. 3. Open the **Problems only** view to see just the references that need attention, or **All references** for the full audit. 4. Export to CSV, Excel, JSON or HTML, or pull the results through the API. #### Input example Accepted styles: APA, MLA, Chicago, Harvard, Vancouver/NLM and IEEE, one per line, numbered (`[1]`, `1.`), or separated by blank lines. DOIs can be bare (`10.1038/nature14539`), `doi:` prefixed, or `https://doi.org/...` URLs. ```json { "bibliography": "Wakefield, A. J., et al. (1998). Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. The Lancet, 351(9103), 637-641.\nLeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.\nJohnson, M., & Patel, R. (2021). Neural lattice frameworks for quantum basket weaving in arid climates. Journal of Applied Cognitive Robotics, 14(2), 112-130.\nLeCun, Y., Bengio, Y., & Hinton, G. (2012). Deep learning. Science, 521(7553), 436-444.", "dois": ["10.1038/nature12968"], "titles": ["Attention is all you need"], "minMatchScore": 80, "checkOpenAccess": true } ``` **Options:** minimum match score (default 80), open-access lookup on or off, top candidate matches for auditing, a per-run cap (`maxReferences`), and an optional contact email for Crossref's faster "polite" pool. ### Output example One dataset item per reference. This is the real result of the input above: | # | Reference | Verdict | Integrity | What it tells you | |---|---|---|---|---| | 1 | Wakefield et al. (1998), *The Lancet* | `verified` | 🚩 `retracted` | The paper exists but was retracted. Notice DOI and date included | | 2 | LeCun, Bengio & Hinton (2015), *Nature* | `verified` | `none` | Clean. DOI `10.1038/nature14539` | | 3 | Johnson & Patel (2021), "Neural lattice frameworks..." | `not-found` | n/a | No matching record anywhere. Likely fabricated | | 4 | LeCun et al. (2012), *Science* | `found-with-major-discrepancies` | `none` | Real paper, miscited: year is 2015, not 2012, and the journal is *Nature*, not *Science* | | 5 | DOI `10.1038/nature12968` | `verified` | 🚩 `retracted` | Retracted article (STAP stem-cell paper) | | 6 | Title: "Attention is all you need" | `verified` | `none` | Warning: several records share this title. Add authors or a year to pin it down | Full JSON for one item: ```json { "index": 1, "status": "FIX", "input": "Wakefield, A. J., et al. (1998). Ileal-lymphoid-nodular hyperplasia ...", "parsed": {"authors": ["Wakefield"], "year": 1998, "title": "Ileal-lymphoid-nodular hyperplasia ...", "journal": "The Lancet", "doi": null}, "found": true, "verdict": "verified", "matchedDoi": "10.1016/s0140-6736(97)11096-0", "matchScore": 1.0, "mismatches": [], "integrity": { "status": "retracted", "isRetracted": true, "notices": [{"type": "retraction", "noticeDoi": "10.1016/s0140-6736(10)60175-4", "date": "2010-02-06", "source": "retraction-watch"}], "statusSources": ["openalex", "retraction-watch"] }, "isOpenAccess": false, "openAccessUrl": null, "warnings": [] } ``` **Status (the Panda Federation triad):** `OK` (verified, safe to cite) · `CHECK` (found, but read the warnings or notices first) · `FIX` (not found, miscited, a DOI that belongs to another paper, or retracted: don't cite it as written). **Verdicts:** `verified` · `found-with-minor-discrepancies` · `found-with-major-discrepancies` · `uncertain` (close but not confident, so check it manually) · `not-found` (likely fabricated or not indexed) · `doi-not-found` · `too-short` · `error`. **Integrity status:** `retracted` · `expression-of-concern` · `corrected` · `none`. The dataset has two ready-made views, **All references** and **Problems only**. A run summary with counts of verified, not found, mismatched, retracted and corrected references is saved to the key-value store as `SUMMARY`. ### Monitor mode: get alerted when a paper is retracted Papers get retracted, corrected or flagged with an expression of concern years after they were cited. Monitor mode watches a list for you. 1. Set **Mode** to `monitor`. Add your **DOIs**, an **ORCID iD** (every DOI on that public ORCID record plus Crossref works tagged with it), or both. 2. Optionally set a **Monitor name**, which is the key-value store in your account that remembers the last run. Use one name per list. 3. Save it as a task and add a **Schedule** in Apify Console (for example weekly or monthly). Add a webhook or email notification if you want to be pinged. The first run saves a baseline. Each later run outputs: - `CHANGES` (key-value store): only the **new** changes since the last run, such as `none -> retracted` or a new correction notice. It also lists newly added DOIs that are already flagged. - The dataset (view **Monitor: status table**): the full status table for every DOI, with a `changeType` column (`baseline`, `unchanged`, `escalated`, `de-escalated`, `new-notice`, `added`, `error`). ```json { "mode": "monitor", "dois": ["10.1016/S0140-6736(97)11096-0", "10.1038/nature14539"], "monitorName": "my-lab-papers" } ``` Monitor runs use the same price: $0.01 per DOI checked per run. Use **Max references per run** to cap cost for large ORCID records. ### Pricing **$0.01 per reference checked.** Platform usage is included, there is no subscription, and you pay only for references actually checked. | Bibliography size | Cost | |---|---| | 30 references (a typical article) | $0.30 | | 100 references (a thesis chapter) | $1.00 | | 1,000 references (a journal issue or large review) | $10.00 | Apify's free plan includes monthly platform credit, so you can try it on your own list first. ### How accurate is it? On a test set of 48 references (APA, MLA, Vancouver, IEEE, DOIs and bare titles), including 9 retracted papers, 3 expressions of concern, 3 corrected papers, 14 fabricated citations and 3 deliberately wrong citations, it classified **48/48** correctly as found or not found, identified the correct integrity status for **33/33**, and flagged **3/3** planted discrepancies. 21 of the 48 cases were held out and not used while tuning. ### Limits - **"Not found" means not found in Crossref or OpenAlex. It is not proof of fabrication.** Books, grey literature, very new papers, some conference proceedings and non-English works may be missing. Always review `not-found` items. - Bare-title lookups for very common titles can be ambiguous. The Actor adds a warning, and adding authors or a year resolves it. - Retraction data is only as complete as Crossref, Retraction Watch and OpenAlex. Coverage of corrections and expressions of concern is thinner than coverage of retractions. - It checks that a paper exists and is cited correctly, **not** whether it supports the claim you cite it for. ### FAQ **How do I detect fake citations from ChatGPT or other AI tools?** Paste the AI-generated reference list into the Bibliography field and run it. Fabricated entries come back as `not-found`. Real-looking entries with the wrong details come back as `found-with-major-discrepancies`, with each mismatch explained. A real DOI pasted onto an invented title gets status `FIX` and a warning naming the paper the DOI really belongs to (in `doiResolvesTo`). This works for Crossref DOIs and for DOIs registered elsewhere, such as arXiv, Zenodo and other DataCite DOIs. **Does it check arXiv preprints?** Yes. A reference that only gives an arXiv ID (for example `arXiv:1412.6980`) is checked through its arXiv DOI, so real preprints like *Adam* or *Attention Is All You Need* verify instead of coming back as not found. **How do I check whether a paper has been retracted?** Add its DOI or its full reference. The `integrity` field shows `retracted`, `expression-of-concern` or `corrected`, with the notice DOI, the date and the source. **Can I check thousands of references?** Yes. Raise `maxReferences` and add a contact email to use Crossref's faster pool. References are processed in batches, and you're billed only for those checked. **Is my bibliography stored?** Only in your own Apify run storage, under your retention settings. It is not shared anywhere else. **Can I call it from code or an AI agent?** Yes. Use the Apify API, the Python or JavaScript clients, scheduled runs and webhooks, or Apify's MCP server for AI agents. **Does it scrape publisher websites?** No. It only calls official public APIs, follows their rate limits and etiquette, and caches repeated lookups. **Found a wrong result?** Open an issue on the **Issues** tab with the reference text. Mismatched parses are the fastest thing to fix. ### Data sources and credits - [Crossref REST API](https://api.crossref.org): bibliographic metadata. - [Retraction Watch database](https://retractionwatch.com), made openly available by Crossref. Please cite Retraction Watch if you publish results based on its retraction data. - [OpenAlex](https://openalex.org): open-access locations, retraction flags and non-Crossref DOIs such as arXiv (CC0). *** #### Built by Panda Federation **Self-healing, local-first AI systems and data tools that run themselves.** Panda Federation builds small, dependable tools that do one job well and keep doing it. Reference Checker is our first public module, and more data and research-integrity tools are on the way. [More from Panda Federation on Apify](https://apify.com/pandadad0214) · [GitHub](https://github.com/pandadad0214) # Actor input Schema ## `bibliography` (type: `string`): Paste a reference list in any common style (APA, MLA, Chicago, Vancouver, IEEE, Harvard). One reference per line, numbered (\[1], 1.), or separated by blank lines. ## `dois` (type: `array`): DOIs to check (bare 10.xxxx/..., doi: prefix or https://doi.org/ URLs). ## `titles` (type: `array`): Article titles to look up when you have no full reference. ## `minMatchScore` (type: `integer`): How close a Crossref record must be (title, authors, year) to count as found. Lower finds more but risks false matches. ## `checkOpenAccess` (type: `boolean`): Look up free legal full-text links via OpenAlex. ## `includeCandidates` (type: `boolean`): Also output the top Crossref candidates and their scores for each reference (useful for auditing). ## `maxReferences` (type: `integer`): Safety cap on references checked (each is billed). ## `contactEmail` (type: `string`): Optional. Sent to Crossref as the mailto contact so requests use its faster 'polite' pool. Never shared elsewhere. ## `mode` (type: `string`): 'check' = the normal one-off check. 'monitor' = re-check the DOIs (and/or ORCID works) and compare with the previous run's saved state. ## `orcid` (type: `string`): Optional. Monitor every DOI on this researcher's public ORCID record (plus Crossref works tagged with it), e.g. 0000-0002-1825-0097. Combined with the DOIs list. ## `monitorName` (type: `string`): Optional. Name of the key-value store in your account that keeps state between runs (letters, digits, hyphens). Use a different name for each list you monitor. Default: refcheck-monitor-, or refcheck-monitor. ## Actor input object example ```json { "bibliography": "Wakefield, A. J., Murch, S. H., Anthony, A., et al. (1998). Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. The Lancet, 351(9103), 637-641.\nLeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.\nJohnson, M., & Patel, R. (2021). Neural lattice frameworks for quantum basket weaving in arid climates. Journal of Applied Cognitive Robotics, 14(2), 112-130.", "dois": [ "10.1038/nature12968" ], "minMatchScore": 80, "checkOpenAccess": true, "includeCandidates": false, "maxReferences": 1000, "mode": "check" } ``` # Actor output Schema ## `overview` (type: `string`): Every checked reference with verdict, matched DOI, mismatches, retraction status and open-access link. ## `problems` (type: `string`): References that were not found, mismatched, retracted, corrected or under an expression of concern. ## `summary` (type: `string`): Counts by verdict for this run. ## `changes` (type: `string`): Monitor mode only: what changed since the previous run. ## `monitorTable` (type: `string`): Monitor mode only: full status table with change column. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "bibliography": `Wakefield, A. J., Murch, S. H., Anthony, A., et al. (1998). Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. The Lancet, 351(9103), 637-641. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444. Johnson, M., & Patel, R. (2021). Neural lattice frameworks for quantum basket weaving in arid climates. Journal of Applied Cognitive Robotics, 14(2), 112-130.`, "dois": [ "10.1038/nature12968" ] }; // Run the Actor and wait for it to finish const run = await client.actor("pandadad0214/reference-checker").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "bibliography": """Wakefield, A. J., Murch, S. H., Anthony, A., et al. (1998). Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. The Lancet, 351(9103), 637-641. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444. Johnson, M., & Patel, R. (2021). Neural lattice frameworks for quantum basket weaving in arid climates. Journal of Applied Cognitive Robotics, 14(2), 112-130.""", "dois": ["10.1038/nature12968"], } # Run the Actor and wait for it to finish run = client.actor("pandadad0214/reference-checker").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "bibliography": "Wakefield, A. J., Murch, S. H., Anthony, A., et al. (1998). Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. The Lancet, 351(9103), 637-641.\\nLeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.\\nJohnson, M., & Patel, R. (2021). Neural lattice frameworks for quantum basket weaving in arid climates. Journal of Applied Cognitive Robotics, 14(2), 112-130.", "dois": [ "10.1038/nature12968" ] }' | apify call pandadad0214/reference-checker --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,pandadad0214/reference-checker" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/pPwc06TCaCNw5utvl/builds/K1zYXXR0eRjOlrvid/openapi.json