📚 arXiv Scraper

🚀 Export open-access research in seconds. Query 2M+ preprints from arXiv by keyword, author, or category, and pull titles, abstracts, authors, DOIs, and PDF URLs into a clean dataset. No API key, no registration, no XML parsing.

The arXiv Scraper queries the public arXiv API (export.arxiv.org) and returns 14 fields per paper, including arxivId, title, authors, full abstract, primary and secondary categories, DOI, journal reference, publication and update dates, and a direct PDF URL. arXiv is the world's largest open-access preprint archive for physics, mathematics, computer science, quantitative biology, statistics, and economics.

The archive spans every major quantitative discipline and 2+ million papers going back to 1991. This Actor converts arXiv query syntax into a structured dataset available as CSV, Excel, JSON, or XML in under five minutes. All filtering happens server-side, so you skip the Atom XML parser entirely.

🎯 Target Audience	💡 Primary Use Cases
Academic researchers, ML engineers, data scientists, literature review teams, citation tracking tools, competitive-intelligence analysts, journalists, educators	Literature reviews, citation graphs, trend tracking, paper discovery, LLM training corpora, author profiling, category monitoring

📋 What the arXiv Scraper does

Three filtering workflows in a single run:

• 🔍 Keyword search. Full-text queries across title, abstract, and metadata using arXiv query syntax.
• 👤 Author search. Pull every paper by a given author using the au: prefix.
• 📂 Category filter. Restrict by arXiv subject category (e.g., cs.LG, math.PR, physics.optics).

Each record includes the arxivId, title, author list, full abstract, primary category and all secondary categories, DOI, journal reference, comment field, publication and update timestamps, plus direct links to the abstract page and the PDF.

💡 Why it matters: arXiv is the default publication channel for machine learning, theoretical physics, and mathematics. Tracking new papers manually is slow, and the official API returns Atom XML that most teams do not want to parse. This Actor returns a flat JSON dataset ready for downstream ingestion.

📊 Data fields

Each record includes: absUrl, arxivId, authors, categories, comment, doi, journalRef, pdfUrl, primaryCategory, published, scrapedAt, summary, title, updated, version. All 15 field names come from a real production run, so what you see here is what lands in your dataset.

⚠️ Good to Know: arXiv enforces a rate limit on its public API. The Actor paces requests to stay within policy, so very large runs (10,000+ papers) naturally take longer. Plan accordingly for literature-review pipelines.

🚀 How to use

1. 📝 Sign up. Create a free account with $5 credit (takes 2 minutes).
2. 🌐 Open the Actor. Go to the arXiv Scraper page on the Apify Store.
3. 🎯 Set input. Enter an arXiv query (e.g., cat:cs.LG AND all:diffusion), pick a sort order, and set maxItems.
4. 🚀 Run it. Click Start and let the Actor collect your papers.
5. 📥 Download. Grab your results in the Dataset tab as CSV, Excel, JSON, or XML.

⏱️ Total time from signup to downloaded dataset: 3-5 minutes. No coding required.

🔗 Recommended Actors

• 🤗 Hugging Face Model Scraper - ML model metadata, downloads, and benchmarks
• 📦 Hugging Face Datasets Scraper - Open datasets for ML and NLP research
• 👨‍🔬 Semantic Scholar Author Profiles Scraper - Author citations, h-index, and affiliations
• 🧬 PubMed Scraper - Biomedical literature from the NIH database
• 📖 Open Library Authors Scraper - Author records from the Internet Archive's Open Library

💡 Pro Tip: browse the complete ParseForge collection for more research and reference-data scrapers.

⚠️ Disclaimer: this Actor is an independent tool and is not affiliated with, endorsed by, or sponsored by arXiv, Cornell University, or the Simons Foundation. All trademarks mentioned are the property of their respective owners. Only publicly available open-access preprint metadata is collected.

🆘 Need Help?

If you hit a bug, have questions about setup, or need a scraper we haven't built yet, open our contact form or write to parseforge@protonmail.com. We also take on paid custom data projects.

For faster answers, join our Discord. It's the best place to get support and suggest new actors.