arXiv papers search — abstracts, authors, categories (API)

One row per arXiv paper from arXiv's public API (https://export.arxiv.org/api/query). Search by plain text or arXiv query syntax, category and submission date; sort by relevance, last update or submission date; get arXiv id, title, authors, abstract, primary and other categories, published/updated dates, DOI, journal reference, author comment and the PDF/abstract links — normalised, one flat row per paper, ready for CSV or JSON.

The default input (cs.AI papers on "large language models", newest first, 100 rows) succeeds within 5 minutes and returns rows — Apify auto-tests it daily.

Data source and status

Publisher	arXiv (Cornell University / arXiv.org)
API	https://export.arxiv.org/api/query — public, no key (documentation: https://info.arxiv.org/help/api/user-manual.html)
What this Actor reads	Plain GET requests with search_query, start, max_results=100, sortBy, sortOrder; the Atom feed's opensearch:totalResults and <entry> elements; one request every 3 seconds
Refresh cadence	New submissions are announced Sunday–Thursday around 20:00 US Eastern (arXiv's daily mailing); the API reflects them then. The Actor reads live on every run; fetched_at on each row is the fetch time
Terms / attribution	Metadata is provided by arXiv under the CC0 1.0 public-domain dedication (https://info.arxiv.org/help/api/tou.html); "Thank you to arXiv for use of its open access interoperability." Full texts remain under their authors' licences. The API terms ask for no more than one request every 3 seconds — this Actor complies
Known caveats	doi, journal_ref and comment are empty when the authors gave none; published is the first-version submission date and updated the latest-version date; the abstract is cut at 4,000 characters (abstract_truncated says so); pdf_url is empty for the few withdrawn papers without a PDF; a malformed arXiv-syntax query is rejected by the API and the run fails with its message
arXiv rate limits	export.arxiv.org/api/query answers 429 Rate exceeded. when arXiv's own backend is busy, independently of how slowly you ask: on 2026-09-13 it answered 429 to single requests from two unrelated networks — including a one-paper id_list lookup — while /oai2 and the arXiv website answered 200 from the same machines. Each page is therefore retried with growing back-off for up to ~3.5 minutes (obeying Retry-After when arXiv sends one). If arXiv never answers, the run fails rather than reporting an empty result. If it stops answering after rows have been written, the run finishes SUCCEEDED with the reason in RUN_SUMMARY.cut_short and in the run's status message, so a partial read is always labelled as one

Honesty note: the rows are the API's own fields renamed and flattened — nothing is inferred, summarised or classified by this Actor, and a run stops with an error rather than guess when the API's shape changes.

Identification: the requests carry a product-token User-Agent and the operator's contact address in the standard From: header (RFC 9110 §10.1.2).

Input

Field	Type	Meaning
query	string	Plain text (several words → all:"phrase") or arXiv syntax passed verbatim when it has a field prefix (ti:, au:, abs:, all:, cat: …), AND/OR/ANDNOT, parentheses or quotes
categories	array	Optional, e.g. cs.AI, cs.CL, stat.ML; any listed category matches. List: https://arxiv.org/category_taxonomy
date_from / date_to	YYYY-MM-DD	Optional, inclusive, on the submission date
sort	relevance / lastUpdatedDate / submittedDate	Default submittedDate
sort_order	descending / ascending	Default descending
max_records	integer	Default 200 (prefilled 100), at most 2,000
request_interval_ms	integer	Default and floor 3000
contact_email	string	Sent in the From: header

At least one of query, categories, date_from/date_to is required.

Example — papers with "transformer" in the title in cs.CL or cs.LG submitted in August 2026, by relevance:

{ "query": "ti:transformer", "categories": ["cs.CL", "cs.LG"], "date_from": "2026-08-01", "date_to": "2026-08-31", "sort": "relevance", "max_records": 25 }

Output (dataset row)

arxiv_id (versioned, e.g. 2609.04180v1), arxiv_id_base, version, title, authors, abstract (≤ 4,000 chars), abstract_truncated, primary_category, categories, published, updated, doi, journal_ref, comment, pdf_url, abs_url, fetched_at, source.

A run summary (filters, the exact search_query sent, the API's own match count, pages, requests, whether the pay-per-event budget stopped the run) is stored as RUN_SUMMARY in the run's key-value store.

Pricing (pay per event)

Event	Price
run-start — once per run	$0.10
record — per row written	$0.005

The default 100-paper pull costs $0.60; 1,000 papers cost $5.10. Rows stop when your run's maximum charge is reached; the run summary says so.

Operator

Steelyard Ventures LLC (RetrainMap) — info@steelyardclinical.com. Not affiliated with arXiv or Cornell University. The Actor writes only to its own dataset and key-value store; it stores no credentials and sends nothing else.