# arXiv Scraper — Papers, Abstracts, Authors & PDFs (`jeffev/arxiv-scraper`) Actor

Search arXiv papers by keywords, categories (cs.AI, cs.CL, q-fin...), authors and submission date range, newest first or by relevance: title, abstract, authors and affiliations, categories, dates, DOI, journal reference, comments and PDF links. Official arXiv API.

- **URL**: https://apify.com/jeffev/arxiv-scraper.md
- **Developed by:** [Jefferson Valandro](https://apify.com/jeffev) (community)
- **Categories:** Developer tools, News
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.35 / 1,000 papers

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## arXiv Scraper — Papers, Abstracts, Authors & PDFs

Search **arXiv** by keywords, **categories** (cs.AI, cs.CL, cs.LG, stat.ML, q-fin...), authors and **submission date range**, newest first or by relevance, or fetch papers by ID or URL. For each paper: title, **abstract**, authors (and affiliations when given), primary and all categories, submission and update dates, version, **DOI, journal reference**, comments and the **PDF link**. Uses the **official arXiv API**.

Built for **AI and research monitoring, literature reviews, trend analysis, newsletters, and AI / RAG datasets**.

### Why this one

- **Category + date filters** combined with any keywords, so you get, for example, every cs.AI paper about agents submitted this month.
- **arXiv query syntax** supported (`ti:`, `abs:`, `au:`).
- **Respects the arXiv API terms** (one request every 3 seconds) for reliable runs.

### Input

LLM agent papers in cs.AI submitted since October:

```json
{
  "searchQueries": ["llm agents"],
  "categories": ["cs.AI"],
  "dateFrom": "2026-10-01",
  "sortBy": "submitted",
  "maxPapersPerSearch": 500
}
```

### Output

| Field | Description |
|---|---|
| `arxivId`, `version`, `title`, `abstract`, `url`, `pdfUrl` | paper |
| `authors`, `authorsWithAffiliation` | authors |
| `primaryCategory`, `categories` | subjects |
| `published`, `updated`, `doi`, `journalRef`, `comment` | metadata |

### Example output

One real item from a test run (long texts and lists shortened):

```json
{
  "type": "paper",
  "arxivId": "2611.03434",
  "version": "v1",
  "title": "Optimize the Harness, Not the Model: Fitting a Minimal Agent to a Self-Hostable LLM for Investigative Analysis",
  "abstract": "Government intelligence teams need LLM agents that can analyze large, messy evidence collections, but security and cost constraints often rule out hosted fro...",
  "authors": [
    "Shai Nahum Gefen",
    "Vered Chassida",
    "Dmitry Basin"
  ],
  "authorsWithAffiliation": [],
  "primaryCategory": "cs.AI",
  "categories": [
    "cs.AI",
    "cs.CL",
    "cs.MA"
  ],
  "published": "2026-11-02T16:53:29Z",
  "updated": "2026-11-02T16:53:29Z",
  "doi": "10.1145/3799682.3839256",
  "journalRef": null,
  "comment": "3 pages, 1 table. Industry Day talk at CIKM 2026, Rome, 7 November 2026",
  "url": "https://arxiv.org/abs/2611.03434",
  "pdfUrl": "https://arxiv.org/pdf/2611.03434v1",
  "search": "llm agents"
}
```

### Notes

- Thank you to arXiv for use of its open access interoperability.

### Feedback

Found a problem or need a field? Open an issue in the **Issues** tab: every issue is read and answered. If the actor helped you, a review on the Store helps a lot.

### How much does it cost?

You pay only for results delivered, at a fixed price per item. **Apify platform usage is included** in that price, so there are no compute costs on top. Errors and items that couldn't be found are free.

| What you get | Price | Example |
|---|---|---|
| Paper | $0.0005 each | 1,000 papers = $0.50 |

The Apify **Free plan includes $5 of monthly credit**, enough for about **10,000 papers** per month at no cost. You can also set a maximum spend per run.

On paid Apify plans it gets cheaper automatically: **10% off on Starter, 20% on Scale and 30% on Business**.

# Actor input Schema

## `searchQueries` (type: `array`):

Keywords (searched in all fields) or arXiv query syntax: ti:transformer, abs:"diffusion model", au:hinton.

## `categories` (type: `array`):

arXiv categories, e.g. cs.AI, cs.CL, cs.LG, cs.CV, stat.ML, q-fin.TR, econ.EM, physics.optics. Applied to every search (or used alone).

## `maxPapersPerSearch` (type: `integer`):

Limit per search.

## `sortBy` (type: `string`):

Order of results.

## `dateFrom` (type: `string`):

YYYY-MM-DD

## `dateTo` (type: `string`):

YYYY-MM-DD

## `authors` (type: `array`):

Author names, e.g. 'Yann LeCun'.

## `arxivIds` (type: `array`):

e.g. 2310.06825 or https://arxiv.org/abs/2310.06825

## Actor input object example

```json
{
  "searchQueries": [
    "llm agents"
  ],
  "categories": [
    "cs.AI"
  ],
  "maxPapersPerSearch": 50,
  "sortBy": "submitted"
}
```

# Actor output Schema

## `results` (type: `string`):

arXiv papers.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQueries": [
        "llm agents"
    ],
    "categories": [
        "cs.AI"
    ],
    "maxPapersPerSearch": 50
};

// Run the Actor and wait for it to finish
const run = await client.actor("jeffev/arxiv-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "searchQueries": ["llm agents"],
    "categories": ["cs.AI"],
    "maxPapersPerSearch": 50,
}

# Run the Actor and wait for it to finish
run = client.actor("jeffev/arxiv-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQueries": [
    "llm agents"
  ],
  "categories": [
    "cs.AI"
  ],
  "maxPapersPerSearch": 50
}' |
apify call jeffev/arxiv-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,jeffev/arxiv-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/EdtzqkFtrcKVkpEzU/builds/hOFKFWmfboCYXByk8/openapi.json