ArXiv Papers Scraper
Under maintenancePricing
from $0.10 / 1,000 dataset items
ArXiv Papers Scraper
Under maintenanceSearch and scrape academic papers from the arXiv API by keyword, category, or author.
Pricing
from $0.10 / 1,000 dataset items
Rating
0.0
(0)
Developer
Moeeze Hassan
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
4 days ago
Last modified
Share
Search and scrape academic papers from the arXiv API by keyword, category, or author. The actor queries the arXiv Atom XML feed and returns structured paper records including title, authors, abstract, category, and direct PDF/abstract URLs.
Features
- Full-text and metadata search via the arXiv search API
- Optional category filter (e.g.
cs.AI,cs.CL,stat.ML) - Sort by relevance or date, ascending or descending
- Direct links to the PDF and abstract pages
Input Fields
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
query | string | ✅ | "" | Search query (keyword, author name, or phrase). |
maxResults | integer | ❌ | 20 | Maximum number of papers to return (1–200). |
sortBy | string (enum) | ❌ | relevance | Sort results by relevance or date. |
sortOrder | string (enum) | ❌ | descending | Sort order: descending or ascending. |
category | string | ❌ | "" | Optional arXiv category filter (e.g. cs.AI, cs.CL, stat.ML). |
At least a
queryorcategorymust be provided.
Output Fields
Each pushed dataset item is a paper record with the following fields:
| Field | Type | Description |
|---|---|---|
id | string | arXiv paper ID (e.g. 2301.12345v1). |
title | string | Paper title. |
authors | array<string> | List of author names. |
abstract | string | Paper abstract (whitespace-collapsed). |
publishedDate | string | Original publication date (ISO-8601). |
updatedDate | string | Last updated date (ISO-8601). |
primaryCategory | string | Primary arXiv category. |
pdfUrl | string | Direct URL to the PDF. |
arxivUrl | string | URL to the arXiv abstract page. |
Usage Example
Search for papers on "transformer attention"
{"query": "transformer attention","maxResults": 20,"sortBy": "relevance","sortOrder": "descending"}
Get the latest 10 papers in the cs.AI category
{"query": "","category": "cs.AI","maxResults": 10,"sortBy": "date","sortOrder": "descending"}
Search for a specific author with a category filter
{"query": "Yann LeCun","category": "cs.LG","maxResults": 50,"sortBy": "date","sortOrder": "descending"}
Run via the Apify CLI
$apify run arxiv-papers-scraper --input '{"query":"transformer attention","maxResults":20}'
Data Source
Papers are fetched from the arXiv export API at http://export.arxiv.org/api/query, which returns an Atom XML feed.