Inspire Hep Scraper

Search inspire hep literature and export normalized paper records from the official INSPIRE HEP API. Build recurring research datasets with titles, abstracts, authors, affiliations, collaborations, citation counts, subjects, publication dates, identifiers, full-text links, and canonical source URLs.

The Actor is designed for physicists, librarians, scientometric analysts, and data teams that need structured INSPIRE records without maintaining API pagination and normalization code.

What does Inspire Hep Scraper do?

The Actor turns an INSPIRE literature search into integration-ready dataset rows. You can:

• search by free text or native INSPIRE query syntax;
• filter by author, collaboration, subject, and inclusive date range;
• sort by recency, citation count, or relevance;
• export up to 10,000 records per run;
• retain both normalized fields and authoritative source links;
• schedule the same query and compare datasets downstream.

It reads the public official JSON API directly. It does not scrape rendered pages, require login credentials, or download full-text documents.

Who is it for?

High-energy-physics researchers

Create a compact evidence set for a topic, collaborator, or experiment before a literature review.

Research librarians

Export stable identifiers, publication references, author affiliations, and source provenance into catalog or discovery workflows.

Scientometric analysts

Collect citation counts, citation links, subjects, collaborations, DOI values, and arXiv identifiers for reproducible analysis.

Data and automation teams

Run a date-bounded query on a schedule, deliver results to a warehouse, and detect new or updated records with your preferred integration.

Why use this Actor?

• Official source: results come from inspirehep.net/api/literature.
• Normalized rows: nested API metadata becomes predictable dataset fields.
• Research context: citations, collaborations, subjects, author IDs, affiliations, DOIs, and arXiv IDs remain together.
• Bounded execution: maxItems, request timeout, pagination, and retry limits prevent uncontrolled runs.
• Transparent provenance: every row includes canonical page, record API, citation-query, source, query, rank, and retrieval time.
• No account required: supported public searches use the anonymous INSPIRE HEP API.

What data can I extract?

Field	Meaning
inspireId	Stable literature record ID
controlNumber	INSPIRE control number when supplied
title	Primary paper title
abstract	Primary abstract, or null when unavailable
authors	Structured names, INSPIRE author IDs, ORCID values, and affiliations
authorNames	Flat author-name list for spreadsheets
affiliations	Unique normalized and raw affiliations across authors
collaborations	Named collaborations such as ATLAS or CMS
citationCount	Current citation count reported by INSPIRE
citationCountWithoutSelfCitations	Citation count excluding self-citations when available
dois	DOI values
arxivIds	Linked arXiv identifiers
texkeys	INSPIRE TeX citation keys
documentTypes	Record document types
subjects	INSPIRE subject classifications
publicationInfo	Journal, volume, year, page, and article reference data
earliestDate	Earliest known literature date
preprintDate	Preprint date
createdAt	Record creation timestamp
updatedAt	Record update timestamp
fullTextUrls	Public document links exposed by INSPIRE
canonicalUrl	Canonical human-readable literature page
apiUrl	Official record API URL
citationsUrl	Official API query for citing papers
query	Effective query sent to INSPIRE
rank	One-based export rank
totalResultsForQuery	Total matches reported by INSPIRE
source	INSPIRE HEP
retrievedAt	UTC retrieval timestamp

Field availability follows the source record. An empty author, DOI, abstract, collaboration, or full-text list is valid when INSPIRE does not expose that metadata.

How to use Inspire Hep Scraper

1. Open the Actor in Apify Console.
2. Enter at least one query or structured filter.
3. Choose a sort order.
4. Set the maximum number of papers.
5. Click Start.
6. Open the Dataset tab to review, download, or integrate the rows.

A useful first run is:

{
  "collaboration": "ATLAS",
  "sort": "mostrecent",
  "maxItems": 10
}

Input parameters

Parameter	Type	Default	Description
query	string	—	Free text or native INSPIRE literature query syntax
author	string	—	Author name such as Witten, Edward
collaboration	string	—	Collaboration such as ATLAS, CMS, or LIGO
subject	string	—	INSPIRE subject such as Theory-HEP
dateFrom	string	—	Inclusive lower date in YYYY, YYYY-MM, or YYYY-MM-DD
dateTo	string	—	Inclusive upper date in the same format
sort	string	mostrecent	mostrecent, mostcited, or relevance
maxItems	integer	100	Maximum saved papers, from 1 to 10,000
maxRequestRetries	integer	3	Retries for timeout, HTTP 429, and temporary server failures, from 0 to 5

All supplied search clauses are combined with and. For example, subject: Theory-HEP plus a 2025 range searches only matching records in that period.

Search examples

Search a research topic

{
  "query": "quantum gravity",
  "sort": "mostcited",
  "maxItems": 25
}

Search an author

{
  "author": "Witten, Edward",
  "sort": "mostcited",
  "maxItems": 50
}

Export collaboration literature

{
  "collaboration": "ATLAS",
  "sort": "mostrecent",
  "maxItems": 100
}

Build a date-bounded subject dataset

{
  "subject": "Theory-HEP",
  "dateFrom": "2025-01-01",
  "dateTo": "2025-12-31",
  "sort": "mostrecent",
  "maxItems": 500
}

Use native INSPIRE syntax

{
  "query": "refersto:recid:451647",
  "sort": "mostcited",
  "maxItems": 100
}

Native query syntax is passed to INSPIRE inside parentheses and can be combined with the structured filters. Invalid syntax returns a failed run with the API error instead of a misleading empty dataset.

Output example

This abbreviated shape reflects real current Actor output:

{
  "inspireId": "101338",
  "controlNumber": 101338,
  "title": "Particle Creation by Black Holes",
  "abstract": "In the quantum theory of fields in curved space-time...",
  "authorNames": ["Hawking, S.W."],
  "affiliations": ["Cambridge U., DAMTP"],
  "collaborations": [],
  "citationCount": 13978,
  "dois": ["10.1007/BF02345020"],
  "arxivIds": [],
  "documentTypes": ["article"],
  "subjects": ["Gravitation and Cosmology"],
  "canonicalUrl": "https://inspirehep.net/literature/101338",
  "apiUrl": "https://inspirehep.net/api/literature/101338",
  "query": "(quantum gravity)",
  "rank": 1,
  "source": "INSPIRE HEP"
}

The full dataset also contains structured authors, publication references, source timestamps, full-text URLs, citation URL, total match count, and retrieval timestamp.

How much does it cost to export INSPIRE HEP papers?

Pay-per-event pricing contains a $0.005 run start and one paper event for every normalized dataset row. On the BRONZE plan, a paper costs $0.003872.

Papers	BRONZE example
1	$0.008872
10	$0.043720
100	$0.392200
1,000	$3.877000

The formula is $0.005 + paper count × plan paper price. No paper event is charged for a duplicate, rejected, empty, or failed record. Apify platform usage may also apply according to your Apify plan.

Recurring literature monitoring

The Actor returns snapshots, not alerts or change records. To build a monitor:

1. choose a subject, author, collaboration, or native query;
2. add a recent date boundary;
3. save the Actor input as an Apify Task;
4. schedule that Task;
5. send each dataset to a database, webhook workflow, or cloud storage;
6. compare stable inspireId values and updatedAt timestamps downstream.

This keeps monitoring logic explicit and avoids claiming that every run contains only newly changed records.

Dataset export and integrations

The default dataset works with Apify's standard export formats, including JSON, CSV, XML, Excel, and RSS where supported. Common workflows include:

• Google Sheets for a review queue;
• BigQuery or Snowflake for scientometric analysis;
• Make or Zapier for notifications;
• webhooks for ingestion after a successful run;
• Python notebooks for citation-network analysis;
• an LLM retrieval pipeline that stores titles and abstracts with source links.

Use inspireId as the stable deduplication key. Use updatedAt to detect source metadata changes.

Run with the Apify API

Replace <APIFY_TOKEN> with your Apify API token.

cURL

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~inspire-hep-literature-search/runs?token=<APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"collaboration":"ATLAS","sort":"mostrecent","maxItems":25}'

JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/inspire-hep-literature-search').call({
  collaboration: 'ATLAS',
  sort: 'mostrecent',
  maxItems: 25,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/inspire-hep-literature-search').call(run_input={
    'subject': 'Theory-HEP',
    'dateFrom': '2025-01-01',
    'dateTo': '2025-12-31',
    'maxItems': 100,
})
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)

Use with MCP and AI assistants

Add the Apify MCP server in Claude Code:

claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/inspire-hep-literature-search"

Claude Desktop setup

Add this HTTP MCP server in Claude Desktop's connector settings.

Cursor setup

Add the same HTTP endpoint in Cursor's MCP settings.

VS Code setup

Add the endpoint to your VS Code MCP server configuration.

Claude Desktop, Cursor, and VS Code can use this equivalent JSON:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/inspire-hep-literature-search"
    }
  }
}

Example prompts:

• “Export the 20 most-cited INSPIRE HEP papers matching quantum gravity.”
• “Find 25 recent ATLAS collaboration records and summarize the citation counts.”
• “Create a 2025 Theory-HEP dataset with DOI and arXiv identifiers.”

Always verify generated scientific conclusions against the source records.

Reliability and retry behavior

Each API request has a 30-second timeout. The Actor retries network failures, HTTP 429 responses, and temporary HTTP 5xx responses with bounded exponential backoff and jitter. It does not blindly retry invalid queries or other deterministic HTTP 4xx responses.

Pagination uses a stable page size of up to 250 records. The Actor deduplicates by inspireId and stops as soon as maxItems is reached or INSPIRE returns no more results.

If a transient outage continues beyond maxRequestRetries, the run fails so scheduled workflows can detect the problem.

Limits and data interpretation

• INSPIRE controls search matching, ranking, record coverage, and citation metrics.
• Citation counts can change as INSPIRE updates its graph.
• Some records have no abstract, authors, affiliation, DOI, arXiv ID, or full-text URL.
• fullTextUrls are links only; this Actor does not download PDFs.
• A broad query can match many records; maxItems limits only this Actor's export.
• Date filters use INSPIRE's literature date query field.
• The Actor does not provide jobs, conferences, profiles, or private data.
• Scheduled comparisons and alerts must be implemented with Apify Tasks or downstream tooling.

Legality and responsible use

INSPIRE HEP is a public scholarly information service. Use the exported metadata in accordance with INSPIRE's terms, applicable database rights, publisher rights, and your organization's research policies.

Do not treat an accessible full-text link as permission to redistribute copyrighted documents. Do not use affiliation or author metadata for harassment, discrimination, or unsolicited personal targeting. Keep canonical source links so users can verify context and corrections.

Troubleshooting

The run says I must provide a filter

At least one of query, author, collaboration, subject, dateFrom, or dateTo is required. This prevents accidental exports of the complete literature index.

INSPIRE returns HTTP 400

The free-form query likely contains invalid native syntax, or a structured value produced an unsupported expression. Test a simpler topic query, then add filters one at a time. Invalid syntax is not retried.

The dataset is empty

Open the equivalent query on INSPIRE and confirm records currently match. Check author spelling, collaboration naming, subject capitalization, and the date range. A valid zero-match search completes successfully with no charged paper events.

Some metadata fields are empty

This normally means the official record does not contain that metadata. Use canonicalUrl to inspect the record and apiUrl to compare the underlying JSON.

A scheduled run failed after earlier success

The official service may be temporarily unavailable or rate-limiting requests. Keep bounded retries enabled and let the failed run signal your automation rather than accepting an incomplete snapshot.

Frequently asked questions

Does this Actor require an INSPIRE account?

No. It uses the public literature API without user credentials.

Can I search multiple criteria together?

Yes. The Actor joins your free query, author, collaboration, subject, and date clauses with and.

Can I export more than one API page?

Yes. Pagination continues automatically up to maxItems, with a maximum of 10,000 papers per run.

Does it return citing papers?

Every row includes citationsUrl. You can also use native syntax such as refersto:recid:451647 in query to export the citing literature itself.

Does it download PDFs?

No. It returns public links exposed by INSPIRE, which keeps runs lightweight and avoids unnecessary transfer.

Is the output real-time?

It is a snapshot of the official API response at retrievedAt, not a stream.

Related automation-lab Actors

• arXiv Paper Search & Export for arXiv-first topic, category, author, and date exports.
• PubMed Article Search Scraper for biomedical article metadata, abstracts, MeSH terms, and affiliations.

Choose this Actor when INSPIRE's high-energy-physics curation, citation graph, collaborations, subjects, and canonical literature IDs are the required source of truth.