arXiv Paper Search Scraper

Search arXiv papers by keyword and narrow results by author, research category or publication date. Export titles, abstracts, authors and paper links for literature reviews, research tracking or an AI knowledge pipeline.

🚀 Quick start

Enter a search phrase and an explicit result limit. After the run, open the dataset and export CSV, Excel or JSON.

{
  "query": "large language models",
  "categories": ["cs.CL", "cs.AI"],
  "sortBy": "submittedDate",
  "sortOrder": "descending",
  "maxItems": 5
}

This searches for matching papers; it does not download PDF contents or summarize the research.

Search controls

Input	Meaning
query	Required keyword or phrase across supported arXiv metadata fields
categories	arXiv codes such as cs.AI, cs.CL, cs.LG or stat.ML; multiple codes are alternatives
author	Optional author-name filter, combined with the query
startDate, endDate	Inclusive publication-date bounds in YYYY-MM-DD format
sortBy	relevance, submittedDate or lastUpdatedDate
sortOrder	ascending or descending
maxItems	Maximum saved papers, up to 5,000
startIndex	Zero-based result offset for continuing an interrupted search

Category, author and text constraints are combined. Date filters use the paper's publication date; they do not mean its most recent revision date. Narrow filters may return fewer papers than the limit.

📄 Output

Each row represents a matching paper's metadata.

Fields	What you can use them for
arxivId, atomId	Identify the paper and deduplicate exports
paperTitle, summary, authors	Review titles, abstracts and author lists
primaryCategory, categories	Group papers by research area
publishedAt, updatedAt	Distinguish publication and update dates
doi, journalRef, comment	Available publication references and notes
absUrl, pdfUrl	Link to the abstract page and available PDF
query, position	Trace the search and returned position

Missing optional publication references are null; a missing DOI is not itself an extraction failure. Follow pdfUrl with a separate document workflow if you need full text.

Research workflows

• Literature discovery: search a phrase, review summary, then follow absUrl or pdfUrl for the original paper.
• Author research: add author while keeping a meaningful query; inspect the returned author lists.
• Recurring topic tracking: schedule the same input, store earlier results and compare arxivId to identify new papers. The Actor does not maintain that history or generate alerts itself.

Pricing and limits

Actor event fees in USD, at the Free-plan rate:

Charge	Price
Start one run	$0.005
1,000 saved papers	$0.575

A run saving 1,000 papers has about $0.58 in Actor event fees. Results depend on the query, upstream availability and run limits. Check the Pricing tab for current rates, plan discounts and additional charges.

Public arXiv endpoints can be slow or temporarily unavailable. A fallback metadata route may use different matching or ordering from the main search API. Treat fallback results as a bounded metadata search, not a guarantee of exhaustive or identical ranking.

API and AI agents

Use the API tab for the full integration details.

$npm install apify-client

import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('fetch_cat/arxiv-paper-search-scraper').call({
  query: 'large language models',
  maxItems: 5,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Connect through the official Apify plugin. Example request: “Use fetch_cat/arxiv-paper-search-scraper to find five papers about large language models and return their titles, abstracts and paper links.” Have the agent inspect the input schema first.

Getting started with Apify

New to Apify? This official introduction explains inputs, runs and exports using another Actor. Use the settings above for this scraper; the Console layout may differ.

FAQ and troubleshooting

Why did I get fewer papers than requested?

The limit is a cap. Check category and date constraints, then inspect RUN_SUMMARY and any saved warnings before treating the result as complete.

Can I resume a time-bounded search?

When PENDING_WORK is available, use its nextStart as startIndex with the same search settings. Retain the earlier dataset separately and deduplicate combined results by arxivId.

Can I search only by an author?

The current schema requires query. Combine it with author rather than submitting an author-only input.

Does this provide full-paper text or generated summaries?

No. summary is the paper's abstract metadata. PDF links do not mean the PDF was downloaded or analyzed.

Related Actors

• Google Scholar Profiles Scraper — Research public scholar profiles.
• GitHub Repositories Search Scraper — Find related research implementations.
• Hacker News Search Scraper — Explore developer discussions about research topics.
• RSS Feed Reader — Collect publisher and research feeds.
• Web Page to Markdown Extractor — Prepare linked web pages for document workflows.

Support

Found an error or have a request? Open an issue with the run ID, a non-sensitive input example and the expected result. For custom additions, email hello@fetchcat.dev. Feedback and reviews help us improve the Actor.

Made with care for curious people, AI agents and the possibilities in data. ♥