arXiv Papers Scraper

🔎 arXiv paper search for research feeds

arXiv Papers Scraper searches public arXiv paper records and saves one dataset row per matching paper. Use it to collect paper metadata for literature monitoring, category feeds, author tracking, date-limited searches, or advanced arXiv queries without user credentials or an API key.

• Run an arXiv papers scraper to collect matching research records for a dataset or dashboard.
• Use an arXiv paper monitor to watch recurring searches and return newly delivered papers.
• Build an arXiv category feed for selected category codes such as cs.AI or stat.ML.
• Run arXiv author search workflows to track papers from specific researchers.
• Create an arXiv metadata export with IDs, titles, authors, categories, dates, DOI values, and URLs.
• Use advanced arXiv search when simple keywords are not precise enough for your query.
• Collect arXiv PDF URLs for direct source links without downloading or hosting files.

📦 What data you get

Each saved result represents one arXiv paper record. Depending on source metadata and your input settings, rows can include:

Field	Description
arxivId	Source-native arXiv identifier.
title	Paper title as published by arXiv.
authors and authorCount	Authors in source order and the number of listed authors.
abstract	Paper abstract when requested and available.
primaryCategory and categories	Primary arXiv category and all listed categories, including cross-listings.
isCrossListed	Whether the paper appears in more than one arXiv category.
submittedDate and updatedDate	Initial submission date and most recent update date.
abstractUrl and pdfUrl	Canonical arXiv abstract page URL and direct source PDF URL.
doi, comment, journalRef	Optional DOI, author comment, and journal reference when arXiv provides them.
abstractReadMinutes	Estimated abstract reading time when the abstract is included.
metadataCompleteness	Ratio from 0 to 1 for useful metadata completeness in the emitted row.
extractionTimestamp	ISO timestamp when the Actor extracted the record.

▶️ How to run it

1. Enter a free-text search query, choose arXiv categories, add author, title, or abstract filters, or use the advanced arXiv query field.
2. Optionally filter by submitted-date range and choose sorting by relevance, newest submitted papers, or recently updated papers.
3. Set the maximum number of papers and decide whether to include abstracts.
4. Turn on Only new papers for scheduled monitoring when you want recurring runs to focus on newly delivered papers.
5. Start the Actor and export the dataset from Apify in your preferred format.

⚙️ Input options

Input	What it does
Search query	Searches public arXiv records by general free text.
arXiv categories	Limits results to selected arXiv category codes.
Advanced arXiv query	Uses arXiv advanced query syntax for precise searches.
Author filter	Filters by paper authors.
Title filter	Filters by terms in the paper title.
Abstract filter	Filters by terms in the abstract.
Submitted from / Submitted to	Filters by submitted-date range.
Sort results by	Orders by relevance, newest submitted papers, or recently updated papers.
Include abstracts	Adds abstracts and abstract read-minute estimates when available.
Only new papers	Returns papers not delivered by earlier runs with the same input.
Maximum papers	Caps the number of paper records returned.

🧾 Output example

{
  "arxivId": "2401.12345",
  "title": "Example Paper Title",
  "authors": ["Jane Doe", "John Smith"],
  "authorCount": 2,
  "abstract": "This paper studies an example research problem.",
  "primaryCategory": "cs.LG",
  "categories": ["cs.LG", "stat.ML"],
  "isCrossListed": true,
  "submittedDate": "2024-01-23T00:00:00.000Z",
  "updatedDate": "2024-02-01T00:00:00.000Z",
  "abstractUrl": "https://arxiv.org/abs/2401.12345",
  "pdfUrl": "https://arxiv.org/pdf/2401.12345",
  "doi": "10.0000/example.doi",
  "comment": "12 pages, 3 figures",
  "journalRef": "Example Journal 1 (2024)",
  "abstractReadMinutes": 1,
  "metadataCompleteness": 1,
  "extractionTimestamp": "2026-07-08T12:00:00.000Z"
}

Optional fields such as abstract, doi, comment, and journalRef appear only when requested or available from arXiv.

💳 Pricing

This actor uses pay-per-event pricing. You are charged only when the Actor saves an accepted arXiv paper record to the dataset. Empty runs, setup checks, failed runs, and searches with no matching papers are not charged as paper results.

🔌 Integrations

Use this actor with Apify integrations, API clients, webhooks, scheduled runs, and dataset exports. Send results to spreadsheets, BI tools, databases, or your own app after each run.

❓ FAQ

🔁 Can I use this as an arXiv paper monitor?

Yes. Use categories, queries, authors, title or abstract filters, or date ranges with scheduled Apify runs. The Only new papers option is designed for recurring monitoring so feeds can focus on newly delivered papers.

🔍 Does it support advanced arXiv search syntax?

Yes. You can use the advanced arXiv query input when simple search terms, category filters, author filters, title terms, or abstract terms are not specific enough.

📄 Does it download PDF files?

No. The Actor returns the direct source pdfUrl from arXiv. It does not download, mirror, host, or transform PDF files.

🧠 Does it summarize or rank papers?

No. It returns source metadata and derived mechanical fields such as author count, cross-listing status, abstract read minutes, and metadata completeness. It does not create LLM summaries, recommendations, rankings, or quality judgments.

🔐 Do I need an arXiv account or API key?

No. The Actor searches public arXiv paper records without requiring user credentials or an API key.

📭 What happens if no papers match my query?

A no-result search is treated as a successful empty run. The pricing model charges only for accepted paper records saved to the dataset.

🧩 Is this an alternative to the arXiv API?

Use it when you want arXiv paper search and monitoring inside Apify with scheduling, webhooks, API access, and dataset exports. If you only need to call arXiv directly from your own code, the source API may be enough.

📝 Changelog

• 0.1.0: Initial release.

🆘 Support

For issues, questions, or feature requests, file a ticket and I'll fix or implement it in less than 24h 🫡

🔗 Other actors

• URL to BibTeX Converter ↗ - Use it for related arxiv and papers workflows.
• ORCID Researcher Profile Search ↗ - Use it for related syntax and doi workflows.
• World Bank Projects Scraper ↗ - Use it for related abstract and education workflows.
• CMS Open Payments Scraper ↗ - Use it for related education workflows.
• GitHub Security Advisories Scraper ↗ - Use it for related feeds workflows. Made with ❤️ by Maxime Dupré