Research Papers API for Agents — OpenAlex + arXiv + Crossref, One Schema

SEO title: Research Papers API for Agents — OpenAlex, arXiv, Crossref SEO description: Search OpenAlex, arXiv and Crossref for scholarly papers in one unified JSON schema: title, authors, citations, DOI, abstract, open-access PDF link. Built for AI research agents. No API key needed.

One actor, one schema, three open scholarly APIs. Search by topic/keyword and get back title, authors, year, venue, citation count, DOI, abstract, and an open-access PDF link where one exists — normalized the same way whether the paper came from OpenAlex, arXiv, or Crossref. Built for AI research agents (MCP-friendly): token-efficient by design, with an explicit fields filter and an include_abstract switch for when you only need metadata.

Use with AI agents (MCP / LangChain)

This actor is built to be called as a tool by an AI agent, not just from the Console. The tool definition (name, description, arguments) is generated automatically from this actor's title and input schema, so an LLM can pick it and fill the arguments correctly.

MCP (Claude Desktop / Cursor / VS Code) — add to your MCP client config:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com",
      "headers": { "Authorization": "Bearer <APIFY_TOKEN>" },
      "actors": ["westerly_breaker/research-papers-api"]
    }
  }
}

LangChain:

from langchain_apify import ApifyActorsTool

papers = ApifyActorsTool("westerly_breaker/research-papers-api")  # reads APIFY_TOKEN from env
result = papers.invoke({"query": "retrieval augmented generation", "max_items": 5,
                        "include_abstract": False})

Why agents like it: no API key for the upstream sources, one unified schema across OpenAlex/arXiv/ Crossref, and token-efficient output — use fields to return only what you need and include_abstract: false to drop the most token-expensive field when you only want metadata.

Standby (low-latency HTTP API, no cold start) — Standby mode is enabled for this actor, so you can skip the batch run/dataset round-trip entirely and call it as a plain HTTP API. Authenticate with your own Apify token (Authorization: Bearer <APIFY_TOKEN>, or ?token=<APIFY_TOKEN>):

curl -H "Authorization: Bearer <APIFY_TOKEN>" \
  "https://westerly-breaker--research-papers-api.apify.actor/search?query=large+language+models&max_items=5&include_abstract=false"

or POST the same fields as a JSON body (identical shape to the Console/API input):

curl -X POST "https://westerly-breaker--research-papers-api.apify.actor/search" \
  -H "Authorization: Bearer <APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"query": "large language models", "max_items": 5, "include_abstract": false}'

Both return {"query": ..., "count": ..., "items": [...]} directly in the HTTP response — no dataset, no Actor.push_data. GET / is a free health check ({"status": "ready", ...}, never charged). Query parameters are plain strings, coerced to the right type server-side (max_items=5 → 5, include_abstract=false → false, sources=openalex,arxiv or repeated sources= params → a list); invalid input returns HTTP 400 with a speaking error message rather than failing a run. Each paper still costs exactly one paper-result charge, same as the batch flow — a budget-limited caller gets back fewer, fully-paid items rather than a response it wasn't billed for the tail of.

Recipes

Copy-paste starting points — each returns finished, deduplicated JSON in one call.

Build a literature-review / RAG corpus. Pull the top recent papers across OpenAlex + arXiv + Crossref in one deduplicated set, abstracts included for embedding:

curl -X POST "https://westerly-breaker--research-papers-api.apify.actor/search" \
  -H "Authorization: Bearer <APIFY_TOKEN>" -H "Content-Type: application/json" \
  -d '{"query": "retrieval augmented generation", "sources": ["openalex","arxiv","crossref"], "max_items": 50}'

Give an agent a token-efficient search tool. Drop the abstract (the most expensive field) and restrict fields so an LLM tool call gets back only what it needs to pick papers:

curl -H "Authorization: Bearer <APIFY_TOKEN>" \
  "https://westerly-breaker--research-papers-api.apify.actor/search?query=diffusion+models&include_abstract=false&fields=title,doi,year,citations&max_items=10"

Filter to high-impact, recent, open-access work. Narrow to papers you can actually read the PDF of:

from langchain_apify import ApifyActorsTool
papers = ApifyActorsTool("westerly_breaker/research-papers-api").invoke(
    {"query": "mRNA vaccine", "year_from": 2022, "min_citations": 50, "open_access_only": True})

Why this exists

OpenAlex, arXiv, and Crossref are all free, open, unauthenticated APIs — no key, no scraping, no ToS risk. But each has its own shape, its own quirks, and its own gaps:

• OpenAlex is the richest source (citations, concepts, open-access links, DOI) but returns the abstract as an abstract_inverted_index (a {word: [positions]} map, not text) — this actor rebuilds it into readable text.
• arXiv returns Atom XML, not JSON, has zero citation data, but always has a PDF link.
• Crossref has strong DOI/citation/venue coverage but abstracts are rare (and arrive as JATS XML fragments when present), and there's no reliable open-access signal.

This actor queries whichever source(s) you ask for, normalizes every result into one schema, deduplicates across sources by DOI, and applies your filters (year, minimum citations, open-access-only) consistently regardless of which source(s) a paper came from.

Input

Field	Type	Default	Description
query	string	—	Required. Search keyword(s) or phrase, e.g. "large language models", "CRISPR gene editing".
sources	array of strings	["openalex"]	Which source(s) to query: "openalex", "arxiv", "crossref".
year_from	integer	—	Only papers published in/after this year. Omit for no lower bound.
min_citations	integer	—	Only papers with at least this many citations. arXiv has no citation data at all, so with this filter set, arXiv results are always excluded (unknown ≠ zero). Omit to skip.
open_access_only	boolean	false	Only papers with a usable oa_pdf_url. Mostly narrows OpenAlex/arXiv — Crossref rarely has a reliable OA signal (see Limitations).
fields	array of strings	[] (= all)	Restrict the returned fields, for token efficiency. source/title are always kept regardless.
include_abstract	boolean	true	Master switch for the abstract — the most token-expensive field. When false, abstract is always null, even if you also listed it in fields.
max_items	integer (1–200)	25	Maximum UNIQUE papers returned across all requested sources combined, after cross-source DOI deduplication.

Worked example — input

{
  "query": "large language models",
  "sources": ["openalex"],
  "max_items": 5,
  "include_abstract": true
}

Worked example — output (2 of 5 items, real data, 2026-07-06)

[
  {
    "source": "openalex",
    "title": "Large language models in medicine",
    "authors": ["Arun James Thirunavukarasu", "Darren Shu Jeng Ting", "Kabilan Elangovan", "Laura Gutiérrez", "Ting Fang Tan", "Daniel Shu Wei Ting"],
    "year": 2023,
    "venue": "Nature Medicine",
    "citations": 3371,
    "doi": "10.1038/s41591-023-02448-8",
    "abstract": null,
    "oa_pdf_url": null,
    "concepts": ["Medicine", "Computer science", "Intensive care medicine", "Computational biology", "Biology"],
    "url": "https://openalex.org/W4384561707"
  },
  {
    "source": "openalex",
    "title": "Large language models encode clinical knowledge",
    "authors": ["Karan Singhal", "Shekoofeh Azizi", "Tao Tu", "..."],
    "year": 2023,
    "venue": "Nature",
    "citations": 3243,
    "doi": "10.1038/s41586-023-06291-2",
    "abstract": "Abstract Large language models (LLMs) have demonstrated impressive capabilities, but the bar for clinical applications is high...",
    "oa_pdf_url": "https://www.nature.com/articles/s41586-023-06291-2.pdf",
    "concepts": ["Psychology", "Computer science", "Medicine", "Session (web analytics)"],
    "url": "https://openalex.org/W4386362180"
  }
]

abstract: null above is not a bug: some OpenAlex works genuinely have no abstract_inverted_index even though the field is present in the API response (publisher-restricted indexing) — confirmed live against the OpenAlex API directly for that work ID.

Output schema

Field	Type	Notes
source	string	"openalex", "arxiv", or "crossref"
title	string	Paper title
authors	array of strings	Author display names, source's listed order
year	integer or null	Publication year
venue	string or null	Journal/conference/repository. null for arXiv preprints with no recorded journal_ref
citations	integer or null	Times cited. Always null for arXiv (no such data from that source)
doi	string or null	Bare DOI (no https://doi.org/ prefix), used for cross-source dedup. Often null for arXiv preprints
abstract	string or null	null when include_abstract: false, or when the source has none
oa_pdf_url	string or null	Direct PDF link. Always present for arXiv; best-effort and sometimes still paywalled in practice for Crossref (see Limitations)
concepts	array of strings	OpenAlex concepts (top 10), arXiv subject categories (e.g. "cs.CL"), or [] for Crossref
url	string	Link to the paper's landing page

Pricing (pay-per-event)

Event	Price	When it's charged
paper-result	$0.002	Once per unique paper returned

Plus Apify's own apify-actor-start synthetic event (first 5 seconds of compute free, platform-managed — never charged from this actor's code).

Example: 1000 papers returned: 1000 × $0.002 = $2.00.

If a run's cost would exceed the Max total charge USD you set for it, the actor stops delivering further papers at exactly that point — every delivered item was paid for, and nothing paid-for is ever dropped. It never crashes and never produces unbilled/"free" results past the limit. Because ALL upstream fetching for this actor happens once per requested source, up front (before any charging starts — see "How it works" below), a budget stop never leaves a half-fetched, wasted upstream request behind either.

Error messages

• Missing query: "Missing 'query': provide a search keyword or phrase..." — add the field; it's the only required input.
• Unsupported sources value: "'sources' contains unsupported value(s) [...]. Supported values are: ['arxiv', 'crossref', 'openalex']."
• year_from/min_citations wrong type or out of range: states the expected range and that omitting the field skips the filter entirely.
• Unsupported fields value: lists the allowed output field names.
• max_items out of range: "'max_items' must be between 1 and 200, got N."
• 0 results with otherwise valid input: not an error — the run succeeds, but the log has an explicit WARNING: 0 results produced despite valid input (...) line and the run's status message says so.

How it works

1. Validate input.
2. Query every requested source exactly once each (one HTTP request per source, not one per item/page) — up to a modest multiple of max_items raw candidates per source, capped at 100.
3. Normalize every result into the unified schema above.
4. Filter each source's results independently (year_from/min_citations/open_access_only), applied the same way regardless of source.
5. Round-robin interleave across sources: source 1's top result, source 2's top result, source 3's top result, source 1's 2nd result, and so on (a source that runs out of results is simply skipped from then on). With a single requested source this is a no-op — order is unchanged.
6. Deduplicate by DOI over that interleaved order (papers without a DOI always survive this step — there's no key to dedupe them on; on a DOI collision, whichever copy comes first in the interleaved order — in practice usually the earlier-listed source — wins).
7. Cap at max_items unique papers, then charge-and-deliver each one.

Known limitations (documented, not hidden)

• Fewer than max_items results are possible even when more exist upstream. Each source is fetched exactly once per run (no pagination) at up to min(max_items × 3, 100) raw candidates; if year_from/min_citations/open_access_only filter out a lot of that batch, or cross-source DOI dedup removes duplicates, the final count can come in under max_items. This keeps upstream cost/latency to one bounded request per source regardless of how large max_items is — a deliberate MVP simplification, not pagination-until-full.
• min_citations always excludes arXiv results when set, because arXiv exposes no citation data at all (its citations field is always null, and null can never satisfy "at least N citations").
• Crossref's oa_pdf_url is best-effort, not a reliable open-access signal. Crossref has no is_oa-style flag the way OpenAlex does; this actor only fills the field when a link entry explicitly advertises content-type: application/pdf (excluding Crossref's own Similarity-Check links) — and even then, the link often still requires a subscription in practice. A more reliable answer would mean integrating Unpaywall, which is out of scope for this MVP.
• Crossref abstracts are present on only a minority of records, and are JATS XML fragments when present — this actor does a best-effort tag-strip (regex), not a full JATS parse.
• Crossref has no concept/topic classification comparable to OpenAlex's concepts — always [] for that source.
• A multi-source request returns a round-robin MIX of sources, not a source-priority ordering. With max_items set lower than the combined per-source results, the output is built by interleaving each requested source's own top results in turn (source 1's #1, source 2's #1, source 3's #1, source 1's #2, ...) — a source is never allowed to silently fill the entire quota on its own just because it was listed first or returned more raw candidates.
• DOI dedup keeps the first occurrence in that interleaved order — if the same paper is found via two sources, whichever source is listed first in sources usually "wins" the returned metadata for that paper (since its copy lands earlier in the round-robin), but this is a side effect of interleave order, not a separate source-priority rule.

Data sources (public, no API key, no scraping)

• OpenAlex Works API: https://api.openalex.org/works
• arXiv API: https://export.arxiv.org/api/query (Atom XML)
• Crossref Works API: https://api.crossref.org/works

All three are open, unauthenticated, rate-limit-friendly APIs meant for exactly this kind of programmatic use — no login, no key, no ToS-risk scraping. This actor identifies itself via a descriptive User-Agent header (no contact email/mailto param needed).