arXiv Papers Scraper: Search Papers, Abstracts & Authors

Search arXiv and get clean, structured paper metadata: title, abstract, authors (with affiliations when listed), arXiv ID and version, primary and cross-list categories, submission and update dates, DOI, journal reference, comments, and links to the abstract page and PDF. Search by keywords, category (cs.AI, cs.LG, hep-th...), author, submission date range or a list of arXiv IDs.

It reads the official arXiv API, so you need no login, API key, browser or proxy setup of your own. $0.90 per 1,000 papers.

What you can do with it

• Daily new-paper feeds: schedule a run for the latest cs.CL or q-bio submissions and send them to Slack, email, Notion or Google Sheets.
• Literature reviews and research monitoring: pull every paper matching a topic in a date range, with abstracts, into a spreadsheet.
• RAG and AI datasets: build a corpus of abstracts and metadata to embed, classify or summarize with an LLM.
• Author and lab tracking: follow what specific researchers publish.
• Bibliometrics: count papers by category, month or cross-list, or collect DOIs and journal references.

How to use it

1. Type one or more searches, pick categories (or leave everything empty for the latest cs.AI papers).
2. Optionally add authors, a submission date range, a sort order and a Max papers cap.
3. Run it, then download the results as JSON, CSV or Excel, or connect them to any Apify integration.

Input

Field	What it does
queries	Free-text searches, one per line, each run with the filters below; every word must match, "exact phrase" in quotes. Rows carry the query that found them; a paper found twice is returned once.
maxItemsPerQuery	Cap per search when you give several (empty = only maxItems applies).
categories	arXiv category codes, matched with OR (cs.AI, cs.LG, stat.ML, astro-ph...).
authors	Author names, matched with OR.
dateFrom, dateTo	First-submission date range (YYYY-MM-DD).
ids	Specific arXiv IDs or abs/pdf links (1706.03762, 2005.14165v4, hep-th/9901001).
searchQuery	Advanced arXiv query syntax (ti:, abs:, au:, cat:, AND/OR/ANDNOT).
sortBy, sortOrder	Relevance, submission date or last-updated date; newest or oldest first.
maxItems	Max papers for the whole run (default 50, up to 10,000). Also caps your cost.

Filled fields are combined with AND. Example:

{ "queries": ["retrieval augmented generation"], "categories": ["cs.CL", "cs.IR"], "dateFrom": "2026-01-01", "maxItems": 200 }

Several topics in one run, 100 papers each:

{ "queries": ["mixture of experts", "speculative decoding", "\"state space model\""], "categories": ["cs.LG", "cs.CL"], "maxItemsPerQuery": 100, "maxItems": 300 }

Output

One item per paper:

{
  "arxivId": "1706.03762", "version": 7,
  "title": "Attention Is All You Need",
  "summary": "The dominant sequence transduction models are based on complex recurrent or convolutional neural networks ...",
  "authors": [{"name": "Ashish Vaswani", "affiliations": []}],
  "authorNames": ["Ashish Vaswani", "Noam Shazeer", "..."],
  "primaryCategory": "cs.CL", "categories": ["cs.CL", "cs.LG"],
  "published": "2017-06-12T17:57:34Z", "updated": "2023-08-02T00:41:18Z",
  "doi": null, "journalRef": null, "comment": "15 pages, 5 figures",
  "absUrl": "https://arxiv.org/abs/1706.03762v7", "pdfUrl": "https://arxiv.org/pdf/1706.03762v7",
  "source": "arXiv API (export.arxiv.org)",
  "attribution": "Thank you to arXiv for use of its open access interoperability. Metadata from arXiv.org (CC0)."
}

Field	Meaning
arxivId, version	arXiv identifier without version, and the latest version number.
title, summary	Title and abstract (whitespace cleaned).
authors, authorNames	Authors with affiliations (when the authors provided them), and a plain name list.
primaryCategory, categories	Main category and all cross-lists.
published, updated	First submission (v1) and latest version timestamps, UTC.
doi, journalRef, comment	Author-supplied publication info; null when not given.
absUrl, pdfUrl	Links to the paper on arXiv.org.
source, attribution	Where the data came from and the arXiv acknowledgement.

Pricing

Pay per result: $0.90 per 1,000 papers (paper-scraped). The maxItems cap applies before you are charged, and the run stops at your spending limit. A search that finds nothing fails instead of returning an empty dataset.

Limits and notes

• Speed: arXiv asks API users to make at most one request every 3 seconds on a single connection. This actor follows that rule, so it returns about 100 papers every 5 seconds (500 papers in about 25 seconds).
• Proxy included: arXiv often turns away requests from shared cloud servers, so the actor connects through Apify Proxy, keeping one IP per run. This costs you nothing beyond the per-paper price.
• arXiv's API returns at most about 30,000 results per search. Split very large pulls by date range.
• Returns metadata only. PDFs are linked, not downloaded: the papers' copyright belongs to their authors, and arXiv asks that readers be sent to arXiv.org for the full text.
• Affiliations, DOI and journal references are filled only when authors supplied them.
• Abstract and metadata are released by arXiv under CC0. Thank you to arXiv for use of its open access interoperability.
• This actor is not affiliated with, endorsed by or supported by arXiv or Cornell University.