arXiv Paper Search Scraper
Pricing
from $0.30 / 1,000 paper results
arXiv Paper Search Scraper
Search arXiv papers by keyword, author, category, and date using public paper metadata.
Pricing
from $0.30 / 1,000 paper results
Rating
0.0
(0)
Developer
Fetch Cat
Maintained by CommunityActor stats
0
Bookmarked
1
Total users
0
Monthly active users
24 days ago
Last modified
Categories
Features
Share
Search arXiv papers by keyword and narrow results by author, research category or publication date. Export titles, abstracts, authors, DOI and paper links for literature reviews, research tracking or an AI knowledge pipeline.
This Actor returns public paper metadata, not PDF contents or generated research summaries. Results can be partial, and backup searches may match or order papers differently.
Quick start
Enter a search phrase and a small result limit. After the run, open the dataset and export CSV, Excel or JSON.
{"query": "large language models","categories": ["cs.CL", "cs.AI"],"sortBy": "submittedDate","sortOrder": "descending","maxItems": 5}
Prefer a video guide? This short Apify walkthrough shows you how to set up and run an Actor. It uses a different Actor, but the setup steps are the same.
What you get
- Paper details: titles, abstracts and author lists, with publication and update dates when available.
- Research links: arXiv abstract and PDF URLs, plus DOI and journal references when provided.
- Search context: the original query and result position alongside each paper, so you can trace an export back to its search.
Who is it for?
- Researchers and students building a reading list around a defined topic.
- Literature-review teams screening abstracts before reading the full papers.
- Research data teams collecting public paper metadata for a searchable catalogue or an AI knowledge workflow.
Research workflows
- Build a reading shortlist. Search a phrase such as “reinforcement learning” and narrow it by category or publication date. Review
paperTitle,authorsandsummary, then followabsUrlto decide which papers to read. The summary is the paper's abstract, not an AI assessment. - Explore an author's work on a topic. Combine a meaningful query with an author name, then inspect the returned author lists and publication references. An author-only search is not supported, and a name match does not establish an author's identity.
- Track research updates. Request submission-date sorting and compare separate exports by
arxivIdin your own workflow. Review warnings before comparing them, because backup searches may use a different order. The Actor does not maintain history or send alerts.
Input settings
| Setting | API key | Meaning |
|---|---|---|
| Search query | query | Required keyword or phrase, not advanced arXiv query syntax |
| Research categories | categories | Optional category codes such as cs.AI or stat.ML. Multiple codes are alternatives |
| Author | author | Optional name, combined with the query |
| Publication dates | startDate, endDate | Inclusive YYYY-MM-DD bounds on initial publication |
| Sorting | sortBy, sortOrder | Relevance, submission date or update date. Ascending or descending |
| Result limit | maxItems | Maximum saved papers: 1–5,000. Defaults to 50 when omitted |
| Result offset | startIndex | Zero-based search offset. Defaults to 0 |
See each input's help for defaults and limitations. Text, category and author constraints are combined. Dates do not filter by the most recent revision. Narrow filters may return fewer papers than the limit.
Example inputs
Use the Quick start input as a small first run. Adjust the fields described above for your own data.
Output data
Each dataset row represents a paper's metadata. The following is a shape-only illustration, not a real paper or a promised result:
{"arxivId": "<arXiv identifier>","paperTitle": "<paper title>","summary": "<abstract text>","authors": ["<author name>"],"absUrl": "<abstract page URL>","pdfUrl": "<PDF URL or null>","doi": null,"position": 1}
| Fields | Contents |
|---|---|
arxivId, atomId | arXiv identifier and abstract URL. Version suffixes may differ between searches |
paperTitle, summary, authors | Title, abstract metadata and author names |
primaryCategory, categories | Primary category when available and returned category tags |
publishedAt, updatedAt | Publication and update dates. Precision depends on available metadata |
doi, journalRef, comment | Optional publication references and notes. Unavailable values are null |
absUrl, pdfUrl | Abstract page and PDF links, not downloaded documents |
query, position | Original search phrase and scan position. Date filtering can leave gaps |
RUN_SUMMARY in the key-value store records saved/scanned counts, request attempts, timestamps and warnings when it can be saved. When a run stops before its time limit, PENDING_WORK may contain a continuation offset. These records are best-effort, not a guarantee that every interrupted run can resume.
Download your results
Open your run's dataset, choose Export and select JSON, CSV or Excel. JSON keeps nested fields together. CSV and Excel are useful when you want to review the records in a spreadsheet.
See Apify's download and export guide for the available formats and field options.
Pricing
The start event charges once when a valid search begins. A search that later fails can still have a start charge and saved results.
The result event charges per paper saved to the dataset, not per scanned paper. There are no paid status or error rows.
See the live Pricing tab for current rates and plan discounts.
Tips for best results
- Start with a specific phrase. Use a small
maxItemsfor your first run, then check whether the titles and abstracts match your topic before increasing it. - Add filters gradually. Category, author and date constraints work together. If the sample is too narrow, relax one filter at a time while keeping a meaningful query.
- Check the abstract before following the PDF. Use
summaryto screen papers, then open the linked document for the full text. The Actor exports the link without reading the PDF. - Review warnings before combining exports. Check saved rows and
RUN_SUMMARYwhen available, particularly if you are comparing updates or continuing from an offset.
Limits and caveats
- Coverage can be partial. Public arXiv services can be slow or unavailable. The result limit is a cap, not a guarantee of that many matches.
- Backup search results can differ. A backup metadata search can use broader word matching and different ordering. Requested sorting and
startIndexare not guaranteed there. - No saved papers means a failed run. This does not establish that no matching papers exist on arXiv. A search that fails can still have a start charge and saved results.
- Dates refer to initial publication. Date filters do not select by the most recent revision. Optional references can be missing, and identifier version suffixes may vary between searches.
- Metadata only. The Actor does not extract full text, assess research quality, collect citation counts or access login-only data. It is an independent tool, not an official arXiv service.
Automation and integrations
Save your settings for next time
Once you have an input that works for you, save it as an Apify task. This keeps your settings together so you can use them again without filling in the form from scratch.
Run on a schedule
Need fresh results regularly? You can schedule the Actor or a saved task to run at a time that suits you.
Each scheduled run collects the data again. If you want to compare results or highlight changes, add that step in your own workflow.
Send results to your other tools
Use n8n, Make or Zapier to connect this scraper to the rest of your workflow. For example, you could pass its results to an AI assistant or add records to a spreadsheet. Follow the Apify integration guides for your chosen tool.
For a custom connection, an Apify webhook can notify your app when a run finishes or fails. Your app can then fetch the results or handle the failure.
API usage
Want to collect this data from your own app or automation? The examples below send input to the scraper and show how to collect its results.
The Node.js and Python examples wait for the run to finish before reading its results. The cURL example starts the run and returns its details. Wait for that run to finish, then use its defaultDatasetId to fetch the results.
For setup instructions, see Apify's JavaScript client guide, Python client guide and Actor runs API reference.
cURL
curl -X POST 'https://api.apify.com/v2/acts/fetch_cat~arxiv-paper-search-scraper/runs' \-H "Authorization: Bearer $APIFY_TOKEN" \-H 'Content-Type: application/json' \--data @- <<'JSON'{"query": "large language models","categories": ["cs.CL","cs.AI"],"sortBy": "submittedDate","sortOrder": "descending","maxItems": 5}JSON
Node.js
import { ApifyClient } from 'apify-client';const client = new ApifyClient({ token: process.env.APIFY_TOKEN });const run = await client.actor('fetch_cat/arxiv-paper-search-scraper').call({"query": "large language models","categories": ["cs.CL","cs.AI"],"sortBy": "submittedDate","sortOrder": "descending","maxItems": 5});const { items } = await client.dataset(run.defaultDatasetId).listItems();console.log(items);
Python
import osfrom apify_client import ApifyClientclient = ApifyClient(os.environ['APIFY_TOKEN'])run = client.actor('fetch_cat/arxiv-paper-search-scraper').call(run_input={'query': 'large language models', 'categories': ['cs.CL', 'cs.AI'], 'sortBy': 'submittedDate', 'sortOrder': 'descending', 'maxItems': 5})print(client.dataset(run['defaultDatasetId']).list_items().items)
MCP and AI agents
You can ask your AI assistant to collect data with arXiv Paper Search Scraper and help you work with the results. The scraper returns the data, and your assistant can then summarize it, compare records or prepare notes.
Connect your assistant through the Apify MCP server. MCP gives your assistant a way to find this scraper, run it and read the results. Follow the official setup and sign-in guide for your app.
Easier setup with an Apify plugin
If you use a supported coding assistant, the official Apify plugin can set up the connection for you. It connects your assistant to Apify and includes instructions for common tasks. Follow the guide for your particular app.
Use Apify AI in Console
Prefer to keep everything in Apify? Open Apify AI in Console and ask it to find Fetch Cat's arXiv Paper Search Scraper. You can describe the data you want and ask questions about the results.
Apify AI is currently in beta. It asks for confirmation before starting an Actor, and normal run charges apply.
Choose your assistant
- Claude and Claude Code (Anthropic): Apify setup guide.
- ChatGPT and Codex (OpenAI): Apify setup guides.
- Grok Bot (xAI): Apify plugin guide.
- Gemini CLI (Google): MCP server setup guide. Use the server URL below with the HTTP configuration supported by Gemini CLI.
Connect just this scraper
To keep your setup focused on this scraper, add this server configuration to a compatible MCP client:
{"mcpServers": {"apify": {"url": "https://mcp.apify.com?tools=fetch_cat/arxiv-paper-search-scraper"}}}
Sign in to Apify when your client asks. If your client uses a Bearer token instead, follow its setup guide and keep the token in its secure configuration. Configuration formats vary by app, so use the linked guide if your client needs a different format.
Use Apify's general server
Already connected to the general Apify MCP server at https://mcp.apify.com? You can use it to find this scraper without adding another server. Ask your assistant to use arXiv Paper Search Scraper by Fetch Cat, or give it the identifier fetch_cat/arxiv-paper-search-scraper.
Your assistant can check the available settings and retrieve the results from the returned run. If the run is still working, it should follow that run rather than start it again. Both connection options use Apify's hosted service.
Example requests
Give your assistant the inputs it needs along with a request like one of these:
- “Find up to ten arXiv papers about retrieval augmented generation in the language-computation category. Give me a reading-list table with titles, authors, abstracts and links to the papers.”
- “Find up to five arXiv papers about neural networks that list Geoffrey Hinton as an author. Show the returned author lists, publication dates and DOI links where available so I can check the matches.”
- “Collect up to ten arXiv papers about reinforcement learning first published during 2025. Ask for newest submissions first, show their abstracts and source links, and tell me if the search reports any warnings.”
Normal Apify charges apply. Check the live Pricing tab before asking your assistant to run a larger batch.
Pay with x402
If your AI agent has a funded wallet, it can pay for Apify usage through x402. This experimental option lets an agent use Apify without a conventional account.
Apify AGI provides a prepaid API token with a spending cap after payment. Your agent can use that token to run this scraper through the Apify API or an authenticated MCP connection.
Some MCP setups also support paying for each request directly. That is a different payment option, so check the official guide for its availability.
Follow the official guide for wallet setup, supported payment options and current terms. Apify handles the payments. Keep wallet keys and payment tokens in your agent's secure configuration.
Other ways for agents to pay
Apify also supports MPP for prepaid access through AGI and Skyfire for eligible Actors. If your agent already uses one of these, see the agentic payment guide to choose the right setup.
FAQ
Is this an arXiv paper search API alternative?
It packages public paper metadata as an Apify dataset for exports and programmatic workflows. It does not reproduce every arXiv search feature. In particular, query is a literal phrase rather than advanced field-prefix syntax.
Can I search only by an author?
No. query is required. Combine it with author rather than submitting an author-only input.
Why did I get fewer papers than requested, or a failed run?
The limit is a cap. Narrow category/date filters and upstream availability affect results. Check saved data and RUN_SUMMARY warnings when available. No saved matches currently produces a failed run. It is not proof of an exhaustive search.
Can I resume a time-bounded search?
When PENDING_WORK is available, use its nextStart as startIndex with the same search settings. Keep the earlier dataset separately and check combined results for duplicates. Continuation applies to the main search. Backup results may not honor that offset or the requested order.
Does this provide full-paper text or generated summaries?
No. summary is abstract metadata. PDF links do not mean the PDF was downloaded or analyzed. DOI, journal references and comments can be missing.
Related Actors
- Google Scholar Profiles Scraper — Research public scholar profiles.
- GitHub Repositories Search Scraper — Find related research implementations.
- Hacker News Search Scraper — Explore developer discussions about research topics.
- RSS Feed Reader — Collect publisher and research feeds.
- Web Page to Markdown Extractor — Prepare linked web pages for document workflows.
Support and feedback
For custom additions or integrations, email hello@fetchcat.dev.
Found an error or have a feature request? Open an issue with the run URL, a non-sensitive input example and what you expected.
Need help sharing it? See Apify’s guide to sharing a run when reporting an issue. Sharing a run can give the developer access to its input, logs and results, so use a run that contains no sensitive data. Please leave out API tokens, passwords, cookies, private documents and personal data. Use a public sample URL or a redacted example instead.
Tried this scraper? We'd love to hear how it worked for you. Leave a review to help others choose the right tool and help us improve it. Your feedback means a lot to us.
Made with care for curious people, AI agents and the possibilities in data. ♥