arXiv Scraper: 12 Fields, PDF Links, No API Key

Pay only for results delivered. Browse all Actors.

💰 From $1.00 / 1,000 results.

Search arXiv preprints and get 12 fields per paper: arXiv ID, title, full abstract, authors, all categories, primary category, published and updated dates, DOI where one is registered, a direct PDF link, the abstract-page URL and a capture timestamp. Filter by keyword, arXiv category and a date floor, and sort by submission date, relevance or last update.

Pure HTTP against the official arXiv API. No API key, no login, no browser, up to 2,000 papers per run.

Why use this arXiv scraper

• PDF link on every row. pdf_url is constructed for each paper, so a download or full-text pipeline needs no second lookup.
• All categories, not just the primary one. categories[] carries every cross-list, which is how you catch a paper that is primarily cs.LG but also stat.ML.
• Three sort orders. submittedDate for a new-paper feed, relevance for a topic search, lastUpdatedDate for revisions.
• A date floor that stops the run early. With sortBy: "submittedDate" and a dateFrom, the Actor stops as soon as it pages past your cutoff instead of walking the whole result set.
• Paced for arXiv. Requests are spaced just over three seconds apart, which is arXiv's own recommended rate and what keeps a large run from being throttled.

Monitor new papers in an arXiv category

{
  "query": "large language models",
  "category": "cs.CL",
  "sortBy": "submittedDate",
  "dateFrom": "2026-07-01",
  "maxResults": 200
}

Newest first with a date floor is the standard research-monitoring shape. Save it as a Task and schedule it.

Narrow a broad topic to one arXiv category

{
  "query": "reinforcement learning",
  "category": "cs.AI",
  "sortBy": "submittedDate",
  "maxResults": 500
}

Query and category are combined with AND, so this returns reinforcement-learning papers whose classification includes cs.AI. A query term is always required, leave it out and the Actor falls back to machine learning.

Find the most relevant papers on a topic

{
  "query": "retrieval augmented generation evaluation",
  "sortBy": "relevance",
  "maxResults": 300
}

Relevance sort is the right choice for a literature scan; submission-date sort is for a feed.

Track revisions to existing work

{
  "query": "diffusion transformer",
  "sortBy": "lastUpdatedDate",
  "maxResults": 200
}

published_date and updated_date are separate fields, so you can see exactly which papers were revised and when.

What data does the arXiv scraper return

{
  "arxiv_id": "2607.01234v1",
  "title": "Scaling Retrieval-Augmented Generation to Long Documents",
  "abstract": "We study retrieval-augmented generation in settings where the evidence...",
  "authors": ["Jane Doe", "John Smith", "Alice Chen"],
  "categories": ["cs.CL", "cs.AI", "cs.IR"],
  "primary_category": "cs.CL",
  "published_date": "2026-07-02T14:21:03Z",
  "updated_date": "2026-07-09T08:15:44Z",
  "doi": null,
  "pdf_url": "https://arxiv.org/pdf/2607.01234v1",
  "url": "https://arxiv.org/abs/2607.01234v1",
  "scraped_at": "2026-08-01T21:00:00.000Z"
}

Field	Description
arxiv_id	arXiv identifier including version suffix
title, abstract	Whitespace-normalised title and abstract
authors[]	Author names in submission order
categories[], primary_category	All arXiv categories and the primary one
published_date, updated_date	First submission and latest revision timestamps
doi	Journal DOI where the authors registered one
pdf_url, url	Direct PDF and abstract-page links
scraped_at	Capture timestamp

Most preprints have no doi, it is only present once the work is published in a venue and the authors add it. That field comes back empty rather than fabricated.

How the scraper works without an API key

The arXiv API is a public Atom feed. This Actor combines your query and category into an arXiv search_query, requests 100 entries per page, parses the Atom XML, and pauses 3.1 seconds between pages in line with arXiv's rate guidance. With a dateFrom and submission-date sort, it stops paging the moment it crosses your cutoff.

Common arXiv categories

Category	Field
cs.AI	Artificial intelligence
cs.CL	Computation and language (NLP)
cs.LG	Machine learning
cs.CV	Computer vision
stat.ML	Machine learning (statistics)
quant-ph	Quantum physics
q-bio	Quantitative biology
econ.EM	Econometrics

What can you build with arXiv data

Research monitoring. A daily feed of new preprints in your categories and keywords.

AI and ML trend analysis. Publication volume by category and month, or author-collaboration graphs.

RAG over preprints. Abstracts plus PDF links for a full-text ingestion pipeline.

Talent and lab tracking. Which groups are publishing in a subfield, and how often.

How much does it cost to scrape arXiv

Pay per paper delivered: $0.002 on the Apify Free plan, $0.001 on Gold and above. Searches that match nothing are never charged.

How do I use arXiv in Claude or ChatGPT

https://mcp.apify.com/?tools=themineworks/arxiv-preprint-search

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });

const run = await client.actor('themineworks/arxiv-preprint-search').call({
  query: 'large language models',
  category: 'cs.CL',
  sortBy: 'submittedDate',
  maxResults: 200,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();

arXiv scraper FAQ

Do I need an API key? No. The arXiv API is open.

Can I search a category without a keyword? No. A query term is always applied, if you leave query empty the Actor substitutes machine learning. Always supply a real query alongside category.

Why is doi usually empty? Preprints only carry a DOI once the authors register a published version.

How fast does a large run go? arXiv asks for roughly one request per three seconds, so a 2,000-paper run takes around a minute of paging. That pacing is deliberate.

Does dateFrom filter server-side? arXiv has no date parameter in this API, so the Actor applies the floor as it pages and stops once it crosses it. Pair it with sortBy: "submittedDate" for the intended behaviour.

Complete your research intel pipeline

• Crossref Scraper, the published record and citation counts.
• OpenCitations Scraper, the citation graph around a DOI.
• PubMed Scraper, the biomedical literature with MeSH terms.

Found a bug or want a field added? Open an issue on the Actor's Apify Console page.

Related guides

• arXiv Scraper: Search AI, Physics, and Biology Preprints with PDF Links via API

Last verified: 2026-08