arXiv Scraper — Papers, Abstracts & Authors

Search arXiv's preprint catalogue with the full query grammar — or fetch exact IDs — and get clean, structured paper metadata. Title, abstract, authors, categories, PDF link, DOI and journal reference. Free keyless API, robust XML parsing, polite rate-limiting.

Ideal for research alerts ("new papers in my field"), LLM training datasets, literature reviews and competitive-science tracking.

What you get

Per paper:

• Core metadata — arXiv ID, title, full abstract (summary), author list
• Dates — published and updated timestamps
• Categories — primaryCategory plus all categories (e.g. cs.LG, stat.ML)
• Publication links — doi, journalRef, author comment
• Direct URLs — pdfUrl (the PDF) and absUrl (the abstract page)

Output sample

{
  "arxivId": "1706.03762v7",
  "title": "Attention Is All You Need",
  "summary": "The dominant sequence transduction models are based on complex recurrent...",
  "authors": ["Ashish Vaswani", "Noam Shazeer", "Niki Parmar"],
  "published": "2017-06-12T17:57:34Z",
  "updated": "2023-08-02T00:41:18Z",
  "primaryCategory": "cs.CL",
  "categories": ["cs.CL", "cs.LG"],
  "doi": null,
  "journalRef": null,
  "comment": "15 pages, 5 figures",
  "pdfUrl": "http://arxiv.org/pdf/1706.03762v7",
  "absUrl": "http://arxiv.org/abs/1706.03762v7"
}

Input

Field	Type	Default	Description
searchQuery	string	—	arXiv query syntax: all:transformer, cat:cs.LG AND ti:diffusion, au:hinton
idList	array	—	Fetch specific arXiv IDs, e.g. ["1706.03762","2005.14165"]
sortBy	string	submittedDate	relevance, lastUpdatedDate or submittedDate
sortOrder	string	descending	ascending or descending
maxResults	integer	100	Cap on papers returned (paged in batches of 100)

Provide a searchQuery or an idList. Query prefixes: ti (title), au (author), abs (abstract), cat (category), all.

Example input

{
  "searchQuery": "cat:cs.LG AND all:diffusion",
  "sortBy": "submittedDate",
  "sortOrder": "descending",
  "maxResults": 100
}

Why this actor

• Full query grammar — not a thin one-off scraper: field prefixes, boolean operators, sort options, and exact-ID lookup.
• Correct & complete parsing — proper Atom XML parsing pulls all authors, both categories, DOI and journal ref that thinner scrapers drop.
• Reliable at scale — pages in batches with arXiv's requested politeness delay and backoff on 429/5xx.

Pricing

Pay per result — $1.50 per 1,000 results (one result = one paper). No subscription or platform fees; you pay only for the results you get.

Notes & limits

• Source is arXiv's official free Atom API — no key, no anti-bot.
• The actor waits ~3s between pages as arXiv asks; large pulls take proportionally longer.
• Public metadata; you are responsible for compliant use. Not affiliated with arXiv/Cornell.

Related actors

• CrossRef Scraper — 150M+ published scholarly works with DOIs and citation counts (pairs with arXiv to cover preprints + published).
• Semantic Scholar Scraper — papers, citations and influence metrics.
• Open Library Scraper — books and ISBN metadata.