arXiv Research Papers & Abstracts Scraper

Here is one real result, with every field the actor returns:

{
  "arxivId": "1603.03851v3",
  "title": "Superconductivity driven by pairing of the coherent parts of the physical electrons",
  "authors": ["Yuehua Su", "Chao Zhang"],
  "authorCount": 2,
  "authorsDetailed": [
    { "name": "Yuehua Su", "affiliation": null },
    { "name": "Chao Zhang", "affiliation": null }
  ],
  "affiliations": null,
  "summary": "How the superconductivity in unconventional superconductors emerges from the diverse mother normal states is still a big puzzle...",
  "categories": ["cond-mat.supr-con"],
  "primaryCategory": "cond-mat.supr-con",
  "published": "2016-03-12T02:43:01Z",
  "updated": "2017-12-15T13:49:39Z",
  "doi": "10.1016/j.physc.2017.12.001",
  "journalRef": "Physica C: Superconductivity and its applications 546 (2018) 21-27",
  "comment": "9 pages; Revised version submitted to Physica C: Superconductivity and its applications",
  "pdfUrl": "https://arxiv.org/pdf/1603.03851v3",
  "absUrl": "https://arxiv.org/abs/1603.03851v3",
  "version": 3,
  "htmlUrl": "https://arxiv.org/html/1603.03851v3",
  "doiUrl": "https://doi.org/10.1016/j.physc.2017.12.001",
  "source": "arXiv",
  "observedAt": "2026-08-14T06:11:27.923Z",
  "aiSummary": null,
  "aiKeywords": null,
  "aiField": null,
  "error": null
}

The most complete arXiv papers scraper available. It returns every field the arXiv API exposes per paper (arXiv ID, title, authors with affiliations, abstract, subject categories, DOI, journal reference, comment, publication and update dates, and PDF/HTML/abstract links), plus optional AI add-ons for a plain-English summary, keywords, and a field classification, and gives you four filters to target exactly the papers you need.

📥 Input · 📤 Output · 💰 Pricing · ▶️ Examples

Table of contents

• What it does
• Quickstart
• Input reference
• Output reference
• Example output record
• Run via API and CLI
• Fetch results
• Billing and limits
• FAQ and troubleshooting

What it does

The actor searches arXiv by keyword, author, or subject (or a native arXiv field query), applies your field, sort, and date filters, paginates through the matching preprints, and writes one normalized record per paper to the run's dataset. Each record includes the arXiv ID, title, authors (plain and detailed with affiliations), abstract, subject categories, DOI, journal reference, and direct PDF, HTML, and abstract links. Missing source values are returned as null. Three optional AI add-ons (paid plans only) can add a plain-English summary, topical keywords, and a field classification.

Quickstart

Open the actor, paste this into the input, and press Run. It returns the 10 most relevant papers on large language models.

{
  "searchQuery": "large language models",
  "maxPapers": 10,
  "field": "all",
  "sortBy": "relevance"
}

Set field to ti, abs, au, or cat to match a plain phrase against a single field, or pass a native query such as ti:transformer AND cat:cs.CL.

Input reference

Field	Type	Required	Default	Description
searchQuery	string	no	large language models	Words, phrase, or native arXiv query with field prefixes such as ti:transformer AND cat:cs.CL.
field	enum	no	all	Which field to match a plain phrase against. One of all, ti (title), abs (abstract), au (author), cat (category code).
sortBy	enum	no	relevance	Order results. One of relevance, newest (submitted), lastUpdated.
dateFrom	string	no	(none)	Only keep papers submitted on or after this date (YYYY-MM-DD).
dateTo	string	no	(none)	Only keep papers submitted on or before this date (YYYY-MM-DD).
maxPapers	integer	no	10	Maximum papers to collect. Free Apify plans are capped at 10 per run.
withAiSummary	boolean	no	false	AI add-on (paid plans only). 1 to 2 sentence plain-English summary from the abstract. Billed only when produced.
withAiKeywords	boolean	no	false	AI add-on (paid plans only). 5 to 10 topical keywords per paper. Billed only when produced.
withAiField	boolean	no	false	AI add-on (paid plans only). Classify the paper's academic field. Billed only when produced.

Output reference

One dataset item per paper. Types: string, integer, string[], object[], or null when the source value is absent.

Field	Type	Description
arxivId	string	arXiv identifier with version, for example 1603.03851v3.
title	string	Paper title.
authors	string[]	Author names.
authorCount	integer	Number of authors.
authorsDetailed	object[]	Authors with affiliation {name, affiliation}.
affiliations	string[]	Distinct affiliations when present, else null.
summary	string	Full abstract text.
categories	string[]	All arXiv subject category codes.
primaryCategory	string	Primary subject category code.
published	string	ISO 8601 submission date.
updated	string	ISO 8601 last-update date.
doi	string	DOI, or null.
journalRef	string	Journal reference, or null.
comment	string	Author comment (for example page count), or null.
pdfUrl	string	Direct PDF URL.
absUrl	string	Abstract page URL.
htmlUrl	string	HTML version URL.
doiUrl	string	Resolved DOI URL, or null.
version	integer	Version number of the paper.
source	string	Data source label (arXiv).
observedAt	string	ISO 8601 timestamp of when the record was collected.
aiSummary	string	Plain-English summary from the AI add-on, or null.
aiKeywords	string[]	Topical keywords from the AI add-on, or null.
aiField	string	Academic field from the AI add-on, or null.
error	string	null on success. On a failed run, a single item with a populated error field is written instead.

Example output record

Real record from a live run (input {"searchQuery": "graphene superconductivity", "maxPapers": 3}):

{
  "arxivId": "1603.03851v3",
  "title": "Superconductivity driven by pairing of the coherent parts of the physical electrons",
  "authors": ["Yuehua Su", "Chao Zhang"],
  "authorCount": 2,
  "categories": ["cond-mat.supr-con"],
  "primaryCategory": "cond-mat.supr-con",
  "published": "2016-03-12T02:43:01Z",
  "updated": "2017-12-15T13:49:39Z",
  "doi": "10.1016/j.physc.2017.12.001",
  "journalRef": "Physica C: Superconductivity and its applications 546 (2018) 21-27",
  "comment": "9 pages; Revised version submitted to Physica C: Superconductivity and its applications",
  "pdfUrl": "https://arxiv.org/pdf/1603.03851v3",
  "absUrl": "https://arxiv.org/abs/1603.03851v3",
  "version": 3,
  "htmlUrl": "https://arxiv.org/html/1603.03851v3",
  "doiUrl": "https://doi.org/10.1016/j.physc.2017.12.001",
  "source": "arXiv",
  "observedAt": "2026-08-14T06:11:27.923Z",
  "error": null
}

Run via API and CLI

Start a run and wait for it to finish, then read the dataset. Replace <TOKEN> with your Apify API token.

Run synchronously and get dataset items in one call:

curl -X POST "https://api.apify.com/v2/acts/scrapers_lat~arxiv-papers-scraper/run-sync-get-dataset-items?token=<TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"searchQuery":"quantum computing","field":"ti","maxPapers":25,"sortBy":"newest"}'

Start a run asynchronously (native query):

curl -X POST "https://api.apify.com/v2/acts/scrapers_lat~arxiv-papers-scraper/runs?token=<TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"searchQuery":"ti:transformer AND cat:cs.CL","maxPapers":100,"dateFrom":"2024-01-01"}'

Apify CLI:

apify call scrapers_lat/arxiv-papers-scraper \
  --input '{"searchQuery":"protein folding","field":"abs","maxPapers":50}'

Fetch results

Every run writes to a dataset. Fetch items as JSON, CSV, or Excel by changing format:

# JSON
curl "https://api.apify.com/v2/datasets/<DATASET_ID>/items?token=<TOKEN>&clean=true&format=json"

# CSV
curl "https://api.apify.com/v2/datasets/<DATASET_ID>/items?token=<TOKEN>&clean=true&format=csv"

# Paginate large datasets
curl "https://api.apify.com/v2/datasets/<DATASET_ID>/items?token=<TOKEN>&offset=1000&limit=1000"

<DATASET_ID> is returned as defaultDatasetId in the run object. Use offset and limit to page through large result sets. clean=true drops empty and internal fields.

Billing and limits

• Pay per result. You are charged per paper returned (result event). See the pricing tab for the current per-result price.
• No charge on failure. If a run errors or matches nothing, the actor writes a single item with a populated error field and does not charge for it.
• Spend cap respected. Set maxTotalChargeUsd on the run; once reached, the actor stops emitting and charging further billable results.
• Free Apify plans are capped at 10 papers per run. Upgrade for higher maxPapers.
• AI add-ons (withAiSummary, withAiKeywords, withAiField) require a paid plan and are charged per paper only when the model returns usable output.

FAQ and troubleshooting

A search returned 0 papers. Why? The query matched nothing, or the date range excluded everything. Zero-result runs are not charged. Loosen the query, set field to all, or widen the dates.

How do I search by author? Set field to au and put the name in searchQuery, or use a native query such as au:hinton.

Can I use native arXiv query syntax? Yes. Put an expression such as ti:transformer AND cat:cs.CL in searchQuery; field prefixes take precedence over the field selector.

How do I get the newest papers on a topic? Set sortBy to newest and optionally add dateFrom.

Why are DOI and journalRef null for some papers? Preprints are often posted before publication, so those fields are empty until the paper is published. Missing values are returned as null, never invented.

Is this an official arXiv tool? No. This actor is independent and has no affiliation with arXiv or Cornell University. It reads only data that is publicly available through the arXiv API.

Related scrapers

• Archive.org Scraper: Internet Archive items and metadata.
• App Store Reviews Scraper: Apple App Store reviews and ratings.
• openFDA Food Recalls Scraper: US FDA food recalls and enforcement.
• SEC EDGAR Company Filings Scraper: SEC filings by ticker or CIK.

More scrapers at scrapers.lat

Built and maintained by scrapers.lat, where we publish scrapers for US and Latin American public platforms: company registries, government data, finance, e-commerce and more. Browse the catalog or request a custom scraper at scrapers.lat.

---

Independent tool, not affiliated with arXiv or Cornell University. Accesses only publicly available arXiv data.