arXiv Papers Scraper - Preprints, Authors & PDFs

Thirdwatch portfolio: 8,800+ users across 140+ public Actors, 2M+ records delivered, >95% run success. Explore all Thirdwatch Actors.

Search arXiv papers — titles, authors, abstracts, categories, and PDF links from the open preprint index.

What you get

Search arXiv papers — titles, authors, abstracts, categories, and PDF links from the open preprint index.

Why use this arXiv Papers scraper

• Full paper metadata — title, authors, abstract, subject categories, published date, and arXiv/PDF links.
• Free open index — arXiv's public API requires no key and no browser.
• Research-grade rows — one clean record per paper for literature pipelines.

Scrape arXiv Papers data in minutes

{
  "queries": [
    "large language models"
  ]
}

{
  "queries": [
    "transformer architecture"
  ]
}

Input parameters

Parameter	Required	Description
queries	Yes	Topics, paper titles, authors, abstracts, or arXiv categories to search. Default: ['large language models'].
searchField	No	Apply each query to all metadata or a specific arXiv field. One of: all, title, author, abstract, category. Default: all.
sortBy	No	Order results using arXiv's supported sort fields. One of: relevance, submittedDate, lastUpdatedDate. Default: submittedDate.
sortOrder	No	Return newest or highest values first with descending order, or reverse them. One of: descending, ascending. Default: descending.
maxResultsPerQuery	No	Stop after this many unique papers for each search term. Default: 100.

Output fields

• arxiv_id
• title
• summary
• published
• updated
• authors
• categories
• primary_category
• doi
• journal_reference
• comment
• paper_url
• pdf_url
• query
• source

Example result

{
  "arxiv_id": "2608.27458v1",
  "title": "The Atacama Cosmology Telescope: Constraints on the anisotropic screening and birefringence effects with DR6",
  "summary": "While we have learned much from recent surveys of the mm-wave sky, many secondary anisotropies of the CMB remain under-e…",
  "published": "2026-08-27T17:59:59Z",
  "updated": "2026-08-27T17:59:59Z",
  "authors": [
    "Darby M. Kramer",
    "Alexander van Engelen",
    "Frank J. Qu",
    "…"
  ],
  "categories": [
    "astro-ph.CO"
  ],
  "primary_category": "astro-ph.CO",
  "doi": null,
  "journal_reference": null,
  "comment": "31 pages, 24 figures. To be submitted to ApJ",
  "paper_url": "http://arxiv.org/abs/2608.27458v1",
  "pdf_url": "https://arxiv.org/pdf/2608.27458v1",
  "query": "large language models",
  "source": "arXiv"
}

Common use cases

• Monitor new papers in a field
• Build literature datasets
• Feed summarization and RAG pipelines
• Track authors and labs publishing on arXiv

Compared to alternatives

• vs other arXiv Papers scrapers on the Store — pay-per-result billing means empty or failed searches cost nothing; you are not charged for a run that returns zero rows.
• vs building your own — no proxy management, markup-change maintenance, or infrastructure to run; paste inputs and consume the dataset.

Limitations

• arXiv only — paywalled venues need institutional access.
• Abstracts are included; full PDF text is linked, not extracted.
• The public API rate-limits heavy pagination — bound runs with maxResults.

FAQ

Do I need an account or cookies?

No — the Actor reads public data only. There is no login to configure and no account to risk.

What happens when a search returns nothing?

The run completes with zero (or an explicit error) rows. Per-result billing means you are not charged for empty output.

Can I run it on a schedule?

Yes — attach an Apify schedule or use the saved tasks to re-run the same inputs daily or weekly.

Related guides and Actors

• Analyze Arxiv Papers With Python
• Build Arxiv Research Dataset
• Monitor New Ai Papers On Arxiv

Last verified: 2026-09

More scrapers at thirdwatch.dev.