# arXiv papers search - abstracts, authors, categories (API) (`retrainmap/arxiv-papers`) Actor

Search arXiv's public API by plain text or arXiv query syntax, category and submission date; one row per paper with arXiv id, title, authors, abstract, categories, published/updated dates, DOI, journal reference and PDF/abstract links. Default: newest cs.AI papers on large language models.

- **URL**: https://apify.com/retrainmap/arxiv-papers.md
- **Developed by:** [RetrainMap Team](https://apify.com/retrainmap) (community)
- **Categories:**
- **Stats:** 2 total users, 1 monthly users, 60.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 records

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## arXiv papers search — abstracts, authors, categories (API)

One row per arXiv paper from arXiv's public API (https://export.arxiv.org/api/query).
Search by plain text or arXiv query syntax, category and submission date; sort by
relevance, last update or submission date; get arXiv id, title, authors, abstract,
primary and other categories, published/updated dates, DOI, journal reference, author
comment and the PDF/abstract links — normalised, one flat row per paper, ready for CSV or
JSON.

**The default input (cs.AI papers on "large language models", newest first, 100 rows)
succeeds within 5 minutes and returns rows** — Apify auto-tests it daily.

### Data source and status

| | |
|---|---|
| Publisher | arXiv (Cornell University / arXiv.org) |
| API | https://export.arxiv.org/api/query — public, no key (documentation: https://info.arxiv.org/help/api/user-manual.html) |
| What this Actor reads | Plain GET requests with `search_query`, `start`, `max_results=100`, `sortBy`, `sortOrder`; the Atom feed's `opensearch:totalResults` and `<entry>` elements; one request every 3 seconds |
| Refresh cadence | New submissions are announced **Sunday–Thursday around 20:00 US Eastern** (arXiv's daily mailing); the API reflects them then. The Actor reads live on every run; `fetched_at` on each row is the fetch time |
| Terms / attribution | Metadata is provided by arXiv under the CC0 1.0 public-domain dedication (https://info.arxiv.org/help/api/tou.html); "Thank you to arXiv for use of its open access interoperability." Full texts remain under their authors' licences. The API terms ask for no more than one request every 3 seconds — this Actor complies |
| Known caveats | `doi`, `journal_ref` and `comment` are empty when the authors gave none; `published` is the first-version submission date and `updated` the latest-version date; the abstract is cut at 4,000 characters (`abstract_truncated` says so); `pdf_url` is empty for the few withdrawn papers without a PDF; a malformed arXiv-syntax query is rejected by the API and the run fails with its message |
| arXiv rate limits | `export.arxiv.org/api/query` answers `429 Rate exceeded.` when arXiv's own backend is busy, independently of how slowly you ask: on **2026-09-13** it answered 429 to single requests from two unrelated networks — including a one-paper `id_list` lookup — while `/oai2` and the arXiv website answered 200 from the same machines. Each page is therefore retried with growing back-off for up to ~3.5 minutes (obeying `Retry-After` when arXiv sends one). If arXiv never answers, the run **fails** rather than reporting an empty result. If it stops answering *after* rows have been written, the run finishes SUCCEEDED with the reason in `RUN_SUMMARY.cut_short` and in the run's status message, so a partial read is always labelled as one |

Honesty note: the rows are the API's own fields renamed and flattened — nothing is
inferred, summarised or classified by this Actor, and a run stops with an error rather
than guess when the API's shape changes.

Identification: the requests carry a product-token User-Agent and the operator's
contact address in the standard `From:` header (RFC 9110 §10.1.2).

### Input

| Field | Type | Meaning |
|---|---|---|
| `query` | string | Plain text (several words → `all:"phrase"`) or arXiv syntax passed verbatim when it has a field prefix (`ti:`, `au:`, `abs:`, `all:`, `cat:` …), `AND`/`OR`/`ANDNOT`, parentheses or quotes |
| `categories` | array | Optional, e.g. `cs.AI`, `cs.CL`, `stat.ML`; any listed category matches. List: https://arxiv.org/category\_taxonomy |
| `date_from` / `date_to` | `YYYY-MM-DD` | Optional, inclusive, on the submission date |
| `sort` | `relevance` / `lastUpdatedDate` / `submittedDate` | Default `submittedDate` |
| `sort_order` | `descending` / `ascending` | Default `descending` |
| `max_records` | integer | Default 200 (prefilled 100), at most 2,000 |
| `request_interval_ms` | integer | Default and floor 3000 |
| `contact_email` | string | Sent in the `From:` header |

At least one of `query`, `categories`, `date_from`/`date_to` is required.

Example — papers with "transformer" in the title in cs.CL or cs.LG submitted in August 2026, by relevance:

```json
{ "query": "ti:transformer", "categories": ["cs.CL", "cs.LG"], "date_from": "2026-08-01", "date_to": "2026-08-31", "sort": "relevance", "max_records": 25 }
```

### Output (dataset row)

`arxiv_id` (versioned, e.g. `2609.04180v1`), `arxiv_id_base`, `version`, `title`, `authors`,
`abstract` (≤ 4,000 chars), `abstract_truncated`, `primary_category`, `categories`,
`published`, `updated`, `doi`, `journal_ref`, `comment`, `pdf_url`, `abs_url`, `fetched_at`,
`source`.

A run summary (filters, the exact `search_query` sent, the API's own match count, pages,
requests, whether the pay-per-event budget stopped the run) is stored as `RUN_SUMMARY` in
the run's key-value store.

### Pricing (pay per event)

| Event | Price |
|---|---|
| `run-start` — once per run | $0.10 |
| `record` — per row written | $0.005 |

The default 100-paper pull costs $0.60; 1,000 papers cost $5.10. Rows stop when your run's
maximum charge is reached; the run summary says so.

### Operator

Steelyard Ventures LLC (RetrainMap) — info@steelyardclinical.com. Not affiliated with
arXiv or Cornell University. The Actor writes only to its own dataset and key-value
store; it stores no credentials and sends nothing else.

# Actor input Schema

## `query` (type: `string`):

Plain text (several words are searched as a phrase in all fields, e.g. large language models -> all:"large language models") or arXiv query syntax passed verbatim when it contains a field prefix (ti:, au:, abs:, all:, cat:, jr:, co:, rn:, id:), AND / OR / ANDNOT, parentheses or double quotes - e.g. ti:transformer AND au:vaswani.

## `categories` (type: `array`):

Optional arXiv categories, e.g. cs.AI, cs.CL, stat.ML, math.OC, q-bio.QM; a paper in any listed category matches. Full list: https://arxiv.org/category\_taxonomy

## `date_from` (type: `string`):

Optional, inclusive; filters on the first submission date (arXiv submittedDate).

## `date_to` (type: `string`):

Optional, inclusive.

## `sort` (type: `string`):

arXiv sortBy: relevance, lastUpdatedDate or submittedDate.

## `sort_order` (type: `string`):

arXiv sortOrder.

## `max_records` (type: `integer`):

Stop after this many papers have been written. Read 100 per request with a 3-second pause between requests (arXiv's rate rule), so 2,000 papers take about one minute.

## `request_interval_ms` (type: `integer`):

arXiv asks for no more than one request every 3 seconds; the floor is 3000 ms.

## `contact_email` (type: `string`):

Sent in the standard From: request header so the data owner can reach the operator (RFC 9110 s10.1.2).

## Actor input object example

```json
{
  "query": "large language models",
  "categories": [
    "cs.AI"
  ],
  "sort": "submittedDate",
  "sort_order": "descending",
  "max_records": 100,
  "request_interval_ms": 3000,
  "contact_email": "info@steelyardclinical.com"
}
```

# Actor output Schema

## `results` (type: `string`):

Every matched paper as one row: arXiv id, title, authors, abstract, categories, published/updated dates, DOI, journal reference, comment, PDF and abstract links.

## `results_csv` (type: `string`):

The same rows as a CSV file.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "large language models",
    "categories": [
        "cs.AI"
    ],
    "max_records": 100
};

// Run the Actor and wait for it to finish
const run = await client.actor("retrainmap/arxiv-papers").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "query": "large language models",
    "categories": ["cs.AI"],
    "max_records": 100,
}

# Run the Actor and wait for it to finish
run = client.actor("retrainmap/arxiv-papers").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "large language models",
  "categories": [
    "cs.AI"
  ],
  "max_records": 100
}' |
apify call retrainmap/arxiv-papers --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,retrainmap/arxiv-papers"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/21akczOrTjH1mx8vm/builds/S7zlwDwhXKjAlHcpX/openapi.json