arXiv Research Paper Scraper

arXiv Research Paper Scraper

Collect public research-paper metadata from arXiv: complete abstracts, all listed authors and published affiliations, subject categories with their taxonomy, original and updated publication timestamps, version numbers and public paper/PDF links. Source comments, journal references, DOI and report numbers are included when available.

Use it to build literature lists, follow a topic or organize public research metadata. The Actor collects metadata and links; paper files remain on arXiv.

Start a search

Enter a search phrase and choose the maximum paper count. For example:

{
  "query": "large language models",
  "maxItems": 50
}

The original defaults remain all fields, relevance order, descending direction and at most 50 papers. A query is required; an empty configuration fails with no records. The API can return an empty result for a valid query, which also produces a failed run with no useful papers.

Search by title, author, abstract, category, identifier or the other offered arXiv fields. Choose submitted date or updated date when chronology matters. Multiple queries share one global maximum. Duplicate papers are removed by their version-independent arXiv identifier unless deduplication is disabled.

searchQuery and searchKeywords are query aliases; maxResults aliases the maximum paper count. A maximum alias overrides an injected default of 50. Conflicting non-default maxima fail before requests. Alternatively, supply public arXiv query URLs through sourceUrls or startUrls; each source retains its own query and sort values. Do not combine source URLs with query fields.

The optional page budget applies independently to each query. Without it, the budget derives from the requested maximum and 50-paper page size, preserving the earlier pagination behavior. An explicit lower page budget may return partial results. Source/page/position fields identify where each paper was found.

Actual output example

This excerpt is based on a saved local Actor output row from October 7, 2026. Its abstract is a 12-word documentation excerpt; the stored row preserves the complete abstract.

{
  "arxivId": "2610.00223",
  "version": 1,
  "title": "A Holistic Assessment of the Carbon Footprint of Noor, a Very Large Arabic Language Model",
  "authors": [
    "Imad Lakim",
    "Ebtesam Almazrouei",
    "Ibrahim Abu Alhaol",
    "Merouane Debbah",
    "Julien Launay"
  ],
  "authorDetails": [
    { "name": "Imad Lakim" },
    { "name": "Ebtesam Almazrouei" },
    { "name": "Ibrahim Abu Alhaol" },
    { "name": "Merouane Debbah" },
    { "name": "Julien Launay" }
  ],
  "abstract": "As ever larger language models grow more ubiquitous, it is crucial to",
  "primaryCategory": "cs.CL",
  "categories": [
    "cs.CL"
  ],
  "classifications": [
    { "term": "cs.CL", "scheme": "http://arxiv.org/schemas/atom" }
  ],
  "publishedAt": "2026-09-22T11:25:39Z",
  "updatedAt": "2026-09-22T11:25:39Z",
  "abstractUrl": "https://arxiv.org/abs/2610.00223",
  "pdfUrl": "https://arxiv.org/pdf/2610.00223v1"
}

The duplicate id field was removed: it repeated arxivId exactly and carried no version information; version remains available separately. Existing consumers can replace id with arxivId. Run instrumentation scrapedAt was removed from each paper row; run completion and request timestamps remain in the summary. authors and categories remain their original string arrays, including every native category term. authorDetails carries each author and all affiliation elements when supplied by arXiv; classifications preserves native category terms with their schemes so ACM or MSC labels remain distinguishable.

The Paper overview shows the complete abstract alongside identity, authors, category, dates and source links. Additional views show publication details and source provenance. Undisclosed optional fields are omitted. PDF links identify public source files without copying or hosting those files.

Confirmed saved papers remain successful if a later request fails; the summary then marks partial results and explains the failure. No saved useful papers means FAILED. Only genuine paper records enter the dataset; notices and counts remain in the run summary. You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

Request settings, cost and support

Developer Options bound timeouts, retries and response size. API requests run sequentially with at least three seconds between attempts, including different queries and retries. A smaller requested delay still uses that required minimum. Run only one instance against arXiv at a time so your combined traffic respects the publisher limit. Access denials and rate-limit responses stop without bypasses.

Compute cost depends on requested papers, pages and waiting time; check current pricing in Apify Console. For help, provide the run ID, public query and expected result through the Actor Issues tab.

The Actor is independent of arXiv. arXiv API terms permit descriptive metadata reuse under CC0, require one request every three seconds over a single connection, and keep paper-content redistribution subject to its separate license.