arXiv Papers Scraper — Authors, Abstracts & PDFs

Search arXiv and get one row per paper: title, the full abstract, every author, categories, the bare arXiv id and its version, a direct PDF link, DOI and journal reference.

No API key, no account. $0.01 per paper — a search that matches nothing is recorded and not charged.

Input

{ "query": "diffusion model", "category": "cs.LG", "maxPapers": 100, "sort": "submitted" }

arXiv query syntax and bulk searches:

{ "queries": ["au:Hinton AND cat:cs.LG", "ti:\"state space model\""], "maxPapers": 50 }

Field	Type	Notes
query	string	Free text or arXiv syntax — au:, ti:, abs:, cat:, joined with AND/OR.
queries	string[]	Up to 10 searches per run.
maxPapers	integer	1–500 per search. You are charged per paper returned.
sort	enum	relevance, submitted, updated.
category	string	Restrict to one category — cs.LG, cs.CV, math.ST, q-bio.NC.

Output

{
  "ok": true,
  "arxivId": "2201.00978",
  "version": 1,
  "title": "PyramidTNT: Improved Transformer-in-Transformer Baselines…",
  "abstract": "Transformer networks have achieved great progress…",
  "authors": ["Kai Han", "Jianyuan Guo", "Yehui Tang", "Yunhe Wang"],
  "firstAuthor": "Kai Han",
  "authorCount": 4,
  "primaryCategory": "cs.CV",
  "categories": ["cs.CV", "cs.LG"],
  "publishedAt": "2022-01-04T04:56:57Z",
  "updatedAt": "2022-01-04T04:56:57Z",
  "isRevised": false,
  "absUrl": "http://arxiv.org/abs/2201.00978v1",
  "pdfUrl": "https://arxiv.org/pdf/2201.00978v1",
  "doi": null,
  "journalRef": null,
  "totalMatches": 178100
}

What it gets right

• ⭐ The id and the version are separate. arXiv returns 2201.00978v1. The v1 belongs in a download URL and breaks a citation or a cross-reference lookup. You get arxivId: "2201.00978" and version: 1, plus a pdfUrl that keeps the version because that is where it matters.
• Titles and abstracts are one line. The Atom feed wraps them over many lines with indentation; raw, they arrive full of newlines and double spaces and break every CSV.
• isRevised is derived and checked. A paper whose updated differs from published has been revised — useful, and asserted to agree with its own inputs so it cannot drift into decoration.
• Category filters are applied server-side, folded into the query, so you do not pay for rows you would filter out yourself.
• It is deliberately unhurried. arXiv asks callers to pause ~3 seconds between requests, and this does. A scraper that hammers a free scholarly API gets the whole platform blocked.
• totalMatches tells you whether 25 rows is the answer or the tip of 178,000.

Pricing

$0.01 per paper returned. No start fee. A search matching nothing produces a paperCount: 0 record and is not charged.

Limits

• 10 searches per run, up to 500 papers each.
• Returns metadata and the abstract, not the full text. Use pdfUrl for the paper itself.
• doi and journalRef exist only where the author supplied them — most preprints have neither.
• arXiv's relevance ranking is its own; this does not re-rank.

FAQ

Do I need an API key? No — arXiv's API is open.

Does arXiv query syntax work? Yes: au:, ti:, abs:, cat:, all:, with AND/OR/ANDNOT.

Why is it slower than other Actors? The 3-second pause between pages is deliberate and is what arXiv asks for.

Changelog

• 0.1 (2026-09-10) — first release. One row per paper, id/version split, normalised titles and abstracts, revised flag, category filter, polite paging.