# Inspire Hep Scraper (`automation-lab/inspire-hep-literature-search`) Actor

Search official INSPIRE HEP literature and export normalized papers, abstracts, authors, affiliations, collaborations, citations, identifiers, dates, and canonical links.

- **URL**: https://apify.com/automation-lab/inspire-hep-literature-search.md
- **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community)
- **Categories:** Education
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.32 / 1,000 paper extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Inspire Hep Scraper

Search **inspire hep** literature and export normalized paper records from the official INSPIRE HEP API.
Build recurring research datasets with titles, abstracts, authors, affiliations, collaborations, citation counts, subjects, publication dates, identifiers, full-text links, and canonical source URLs.

The Actor is designed for physicists, librarians, scientometric analysts, and data teams that need structured INSPIRE records without maintaining API pagination and normalization code.

### What does Inspire Hep Scraper do?

The Actor turns an INSPIRE literature search into integration-ready dataset rows.
You can:

- search by free text or native INSPIRE query syntax;
- filter by author, collaboration, subject, and inclusive date range;
- sort by recency, citation count, or relevance;
- export up to 10,000 records per run;
- retain both normalized fields and authoritative source links;
- schedule the same query and compare datasets downstream.

It reads the public official JSON API directly.
It does not scrape rendered pages, require login credentials, or download full-text documents.

### Who is it for?

#### High-energy-physics researchers

Create a compact evidence set for a topic, collaborator, or experiment before a literature review.

#### Research librarians

Export stable identifiers, publication references, author affiliations, and source provenance into catalog or discovery workflows.

#### Scientometric analysts

Collect citation counts, citation links, subjects, collaborations, DOI values, and arXiv identifiers for reproducible analysis.

#### Data and automation teams

Run a date-bounded query on a schedule, deliver results to a warehouse, and detect new or updated records with your preferred integration.

### Why use this Actor?

- **Official source:** results come from `inspirehep.net/api/literature`.
- **Normalized rows:** nested API metadata becomes predictable dataset fields.
- **Research context:** citations, collaborations, subjects, author IDs, affiliations, DOIs, and arXiv IDs remain together.
- **Bounded execution:** `maxItems`, request timeout, pagination, and retry limits prevent uncontrolled runs.
- **Transparent provenance:** every row includes canonical page, record API, citation-query, source, query, rank, and retrieval time.
- **No account required:** supported public searches use the anonymous INSPIRE HEP API.

### What data can I extract?

| Field | Meaning |
| --- | --- |
| `inspireId` | Stable literature record ID |
| `controlNumber` | INSPIRE control number when supplied |
| `title` | Primary paper title |
| `abstract` | Primary abstract, or `null` when unavailable |
| `authors` | Structured names, INSPIRE author IDs, ORCID values, and affiliations |
| `authorNames` | Flat author-name list for spreadsheets |
| `affiliations` | Unique normalized and raw affiliations across authors |
| `collaborations` | Named collaborations such as ATLAS or CMS |
| `citationCount` | Current citation count reported by INSPIRE |
| `citationCountWithoutSelfCitations` | Citation count excluding self-citations when available |
| `dois` | DOI values |
| `arxivIds` | Linked arXiv identifiers |
| `texkeys` | INSPIRE TeX citation keys |
| `documentTypes` | Record document types |
| `subjects` | INSPIRE subject classifications |
| `publicationInfo` | Journal, volume, year, page, and article reference data |
| `earliestDate` | Earliest known literature date |
| `preprintDate` | Preprint date |
| `createdAt` | Record creation timestamp |
| `updatedAt` | Record update timestamp |
| `fullTextUrls` | Public document links exposed by INSPIRE |
| `canonicalUrl` | Canonical human-readable literature page |
| `apiUrl` | Official record API URL |
| `citationsUrl` | Official API query for citing papers |
| `query` | Effective query sent to INSPIRE |
| `rank` | One-based export rank |
| `totalResultsForQuery` | Total matches reported by INSPIRE |
| `source` | `INSPIRE HEP` |
| `retrievedAt` | UTC retrieval timestamp |

Field availability follows the source record.
An empty author, DOI, abstract, collaboration, or full-text list is valid when INSPIRE does not expose that metadata.

### How to use Inspire Hep Scraper

1. Open the Actor in Apify Console.
2. Enter at least one query or structured filter.
3. Choose a sort order.
4. Set the maximum number of papers.
5. Click **Start**.
6. Open the **Dataset** tab to review, download, or integrate the rows.

A useful first run is:

```json
{
  "collaboration": "ATLAS",
  "sort": "mostrecent",
  "maxItems": 10
}
```

### Input parameters

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `query` | string | — | Free text or native INSPIRE literature query syntax |
| `author` | string | — | Author name such as `Witten, Edward` |
| `collaboration` | string | — | Collaboration such as `ATLAS`, `CMS`, or `LIGO` |
| `subject` | string | — | INSPIRE subject such as `Theory-HEP` |
| `dateFrom` | string | — | Inclusive lower date in `YYYY`, `YYYY-MM`, or `YYYY-MM-DD` |
| `dateTo` | string | — | Inclusive upper date in the same format |
| `sort` | string | `mostrecent` | `mostrecent`, `mostcited`, or `relevance` |
| `maxItems` | integer | `100` | Maximum saved papers, from 1 to 10,000 |
| `maxRequestRetries` | integer | `3` | Retries for timeout, HTTP 429, and temporary server failures, from 0 to 5 |

All supplied search clauses are combined with `and`.
For example, `subject: Theory-HEP` plus a 2025 range searches only matching records in that period.

### Search examples

#### Search a research topic

```json
{
  "query": "quantum gravity",
  "sort": "mostcited",
  "maxItems": 25
}
```

#### Search an author

```json
{
  "author": "Witten, Edward",
  "sort": "mostcited",
  "maxItems": 50
}
```

#### Export collaboration literature

```json
{
  "collaboration": "ATLAS",
  "sort": "mostrecent",
  "maxItems": 100
}
```

#### Build a date-bounded subject dataset

```json
{
  "subject": "Theory-HEP",
  "dateFrom": "2025-01-01",
  "dateTo": "2025-12-31",
  "sort": "mostrecent",
  "maxItems": 500
}
```

#### Use native INSPIRE syntax

```json
{
  "query": "refersto:recid:451647",
  "sort": "mostcited",
  "maxItems": 100
}
```

Native query syntax is passed to INSPIRE inside parentheses and can be combined with the structured filters.
Invalid syntax returns a failed run with the API error instead of a misleading empty dataset.

### Output example

This abbreviated shape reflects real current Actor output:

```json
{
  "inspireId": "101338",
  "controlNumber": 101338,
  "title": "Particle Creation by Black Holes",
  "abstract": "In the quantum theory of fields in curved space-time...",
  "authorNames": ["Hawking, S.W."],
  "affiliations": ["Cambridge U., DAMTP"],
  "collaborations": [],
  "citationCount": 13978,
  "dois": ["10.1007/BF02345020"],
  "arxivIds": [],
  "documentTypes": ["article"],
  "subjects": ["Gravitation and Cosmology"],
  "canonicalUrl": "https://inspirehep.net/literature/101338",
  "apiUrl": "https://inspirehep.net/api/literature/101338",
  "query": "(quantum gravity)",
  "rank": 1,
  "source": "INSPIRE HEP"
}
```

The full dataset also contains structured authors, publication references, source timestamps, full-text URLs, citation URL, total match count, and retrieval timestamp.

### How much does it cost to export INSPIRE HEP papers?

Pay-per-event pricing contains a **$0.005 run start** and one `paper` event for every normalized dataset row.
On the BRONZE plan, a paper costs **$0.003872**.

| Papers | BRONZE example |
| ---: | ---: |
| 1 | $0.008872 |
| 10 | $0.043720 |
| 100 | $0.392200 |
| 1,000 | $3.877000 |

The formula is `$0.005 + paper count × plan paper price`.
No paper event is charged for a duplicate, rejected, empty, or failed record.
Apify platform usage may also apply according to your Apify plan.

### Recurring literature monitoring

The Actor returns snapshots, not alerts or change records.
To build a monitor:

1. choose a subject, author, collaboration, or native query;
2. add a recent date boundary;
3. save the Actor input as an Apify Task;
4. schedule that Task;
5. send each dataset to a database, webhook workflow, or cloud storage;
6. compare stable `inspireId` values and `updatedAt` timestamps downstream.

This keeps monitoring logic explicit and avoids claiming that every run contains only newly changed records.

### Dataset export and integrations

The default dataset works with Apify's standard export formats, including JSON, CSV, XML, Excel, and RSS where supported.
Common workflows include:

- Google Sheets for a review queue;
- BigQuery or Snowflake for scientometric analysis;
- Make or Zapier for notifications;
- webhooks for ingestion after a successful run;
- Python notebooks for citation-network analysis;
- an LLM retrieval pipeline that stores titles and abstracts with source links.

Use `inspireId` as the stable deduplication key.
Use `updatedAt` to detect source metadata changes.

### Run with the Apify API

Replace `<APIFY_TOKEN>` with your Apify API token.

#### cURL

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~inspire-hep-literature-search/runs?token=<APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"collaboration":"ATLAS","sort":"mostrecent","maxItems":25}'
```

#### JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/inspire-hep-literature-search').call({
  collaboration: 'ATLAS',
  sort: 'mostrecent',
  maxItems: 25,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

#### Python

```python
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/inspire-hep-literature-search').call(run_input={
    'subject': 'Theory-HEP',
    'dateFrom': '2025-01-01',
    'dateTo': '2025-12-31',
    'maxItems': 100,
})
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)
```

### Use with MCP and AI assistants

Add the Apify MCP server in Claude Code:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/inspire-hep-literature-search"
```

#### Claude Desktop setup

Add this HTTP MCP server in Claude Desktop's connector settings.

#### Cursor setup

Add the same HTTP endpoint in Cursor's MCP settings.

#### VS Code setup

Add the endpoint to your VS Code MCP server configuration.

Claude Desktop, Cursor, and VS Code can use this equivalent JSON:

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/inspire-hep-literature-search"
    }
  }
}
```

Example prompts:

- “Export the 20 most-cited INSPIRE HEP papers matching quantum gravity.”
- “Find 25 recent ATLAS collaboration records and summarize the citation counts.”
- “Create a 2025 Theory-HEP dataset with DOI and arXiv identifiers.”

Always verify generated scientific conclusions against the source records.

### Reliability and retry behavior

Each API request has a 30-second timeout.
The Actor retries network failures, HTTP 429 responses, and temporary HTTP 5xx responses with bounded exponential backoff and jitter.
It does not blindly retry invalid queries or other deterministic HTTP 4xx responses.

Pagination uses a stable page size of up to 250 records.
The Actor deduplicates by `inspireId` and stops as soon as `maxItems` is reached or INSPIRE returns no more results.

If a transient outage continues beyond `maxRequestRetries`, the run fails so scheduled workflows can detect the problem.

### Limits and data interpretation

- INSPIRE controls search matching, ranking, record coverage, and citation metrics.
- Citation counts can change as INSPIRE updates its graph.
- Some records have no abstract, authors, affiliation, DOI, arXiv ID, or full-text URL.
- `fullTextUrls` are links only; this Actor does not download PDFs.
- A broad query can match many records; `maxItems` limits only this Actor's export.
- Date filters use INSPIRE's literature `date` query field.
- The Actor does not provide jobs, conferences, profiles, or private data.
- Scheduled comparisons and alerts must be implemented with Apify Tasks or downstream tooling.

### Legality and responsible use

INSPIRE HEP is a public scholarly information service.
Use the exported metadata in accordance with INSPIRE's terms, applicable database rights, publisher rights, and your organization's research policies.

Do not treat an accessible full-text link as permission to redistribute copyrighted documents.
Do not use affiliation or author metadata for harassment, discrimination, or unsolicited personal targeting.
Keep canonical source links so users can verify context and corrections.

### Troubleshooting

#### The run says I must provide a filter

At least one of `query`, `author`, `collaboration`, `subject`, `dateFrom`, or `dateTo` is required.
This prevents accidental exports of the complete literature index.

#### INSPIRE returns HTTP 400

The free-form `query` likely contains invalid native syntax, or a structured value produced an unsupported expression.
Test a simpler topic query, then add filters one at a time.
Invalid syntax is not retried.

#### The dataset is empty

Open the equivalent query on INSPIRE and confirm records currently match.
Check author spelling, collaboration naming, subject capitalization, and the date range.
A valid zero-match search completes successfully with no charged paper events.

#### Some metadata fields are empty

This normally means the official record does not contain that metadata.
Use `canonicalUrl` to inspect the record and `apiUrl` to compare the underlying JSON.

#### A scheduled run failed after earlier success

The official service may be temporarily unavailable or rate-limiting requests.
Keep bounded retries enabled and let the failed run signal your automation rather than accepting an incomplete snapshot.

### Frequently asked questions

#### Does this Actor require an INSPIRE account?

No. It uses the public literature API without user credentials.

#### Can I search multiple criteria together?

Yes. The Actor joins your free query, author, collaboration, subject, and date clauses with `and`.

#### Can I export more than one API page?

Yes. Pagination continues automatically up to `maxItems`, with a maximum of 10,000 papers per run.

#### Does it return citing papers?

Every row includes `citationsUrl`.
You can also use native syntax such as `refersto:recid:451647` in `query` to export the citing literature itself.

#### Does it download PDFs?

No. It returns public links exposed by INSPIRE, which keeps runs lightweight and avoids unnecessary transfer.

#### Is the output real-time?

It is a snapshot of the official API response at `retrievedAt`, not a stream.

### Related automation-lab Actors

- [arXiv Paper Search & Export](https://apify.com/automation-lab/arxiv-paper-search-export) for arXiv-first topic, category, author, and date exports.
- [PubMed Article Search Scraper](https://apify.com/automation-lab/pubmed-article-search-scraper) for biomedical article metadata, abstracts, MeSH terms, and affiliations.

Choose this Actor when INSPIRE's high-energy-physics curation, citation graph, collaborations, subjects, and canonical literature IDs are the required source of truth.

# Actor input Schema

## `query` (type: `string`):

Free-text or INSPIRE search syntax, for example "quantum gravity" or "refersto:recid:451647". Combined with the structured filters below.

## `author` (type: `string`):

Author name to match, such as "Witten, Edward".

## `collaboration` (type: `string`):

Collaboration name to match, such as ATLAS, CMS, or LIGO.

## `subject` (type: `string`):

INSPIRE subject term, such as Theory-HEP, Experiment-HEP, or Gravitation and Cosmology.

## `dateFrom` (type: `string`):

Earliest literature date, inclusive, in YYYY, YYYY-MM, or YYYY-MM-DD format.

## `dateTo` (type: `string`):

Latest literature date, inclusive, in YYYY, YYYY-MM, or YYYY-MM-DD format.

## `sort` (type: `string`):

Order returned by the official INSPIRE HEP API.

## `maxItems` (type: `integer`):

Maximum number of normalized literature records to save.

## `maxRequestRetries` (type: `integer`):

Retries for timeouts, rate limits, and temporary server errors. Invalid queries are not retried.

## Actor input object example

```json
{
  "query": "quantum gravity",
  "sort": "mostrecent",
  "maxItems": 10,
  "maxRequestRetries": 3
}
```

# Actor output Schema

## `dataset` (type: `string`):

Dataset containing normalized INSPIRE HEP papers.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "quantum gravity",
    "maxItems": 10,
    "maxRequestRetries": 3
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/inspire-hep-literature-search").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "query": "quantum gravity",
    "maxItems": 10,
    "maxRequestRetries": 3,
}

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/inspire-hep-literature-search").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "quantum gravity",
  "maxItems": 10,
  "maxRequestRetries": 3
}' |
apify call automation-lab/inspire-hep-literature-search --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,automation-lab/inspire-hep-literature-search"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/czitWnvujz9Tp7Rmy/builds/SSBGyD5bDF8AGdogL/openapi.json