INSPIRE-HEP Physics Papers Scraper

INSPIRE-HEP Physics Papers Scraper

Scrape high-energy physics papers from INSPIRE-HEP by keyword, author, or title, up to a million per run. Each record includes the title, authors, citation count, publication date, and arXiv ID. No API key required. Export to CSV, JSON, Excel, or XML.

INSPIRE-HEP is the definitive bibliographic database for particle physics, but its web interface is built for manual browsing, not bulk analysis. This Actor reads the public search results directly, accepts plain keywords or the full INSPIRE query syntax, and returns every matching paper in a flat, structured row. You can collect up to a million records in a single run, filtered by recency or citation count.

Who uses it	What they scrape INSPIRE-HEP for
Physics researchers	Build a corpus of papers on a specific theory or phenomenon for a literature review.
Librarians and data curators	Harvest metadata for institutional repositories or research information systems.
Scientometric analysts	Track citation trends and collaboration networks across high-energy physics subfields.
PhD students	Gather the foundational papers in a narrow research area to populate a thesis bibliography.

What it does

This Actor collects INSPIRE-HEP paper records by search query and returns each one as a flat row with title, authors, citation count, publication date, and identifiers.

• 🔍 Full INSPIRE query syntax: search by title, author, exact identifier, or any fielded query the native search supports.
• 📊 Citation-aware sorting: order results by most recent, most cited, or oldest first to match your analysis workflow.
• 📄 Structured flat output: every paper lands as one row with a fixed schema, ready for spreadsheets or databases.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with INSPIRE-HEP data

📈 Build a literature corpus.

A phenomenologist searches for 'title dark matter' and collects the 500 most cited papers to seed a systematic review.

🔗 Harvest author publication lists.

A research group leader queries 'a J.Ellis.1' to pull every paper by a specific INSPIRE author profile into a CV or grant report.

📊 Monitor emerging research.

A journal editor runs a daily scrape for 'title axion' sorted by most recent to spot new preprints before they appear on arXiv alerts.

🗂️ Populate an institutional repository.

A university librarian collects all papers with a given affiliation string to backfill missing metadata in the campus research database.

Why choose this scraper

	What you get
No API key or registration	Reads the public search pages directly so you can start scraping immediately without signing up for a developer token.
Full query language support	Accepts the same fielded syntax as the INSPIRE-HEP web search, including author, title, and identifier lookups.
Up to a million papers per run	Collects large datasets in one go, bounded only by the maxItems you set.
Fixed output schema	Every paper arrives with the same columns, making it trivial to merge runs or feed downstream tools.

How it compares

No other Store actor targets INSPIRE-HEP the same way, so the honest comparison is with the alternatives teams actually weigh.

	INSPIRE-HEP Physics Papers Scraper	Build it in-house	By hand
Setup	Run it now, zero config	Days of engineering	None, but hours per pull
When INSPIRE-HEP changes	Maintained for you	You fix it	You re-learn the page
Proxies, retries, anti-bot	Built in	Your problem	Browser only
Output	Fixed JSON schema, CSV/Excel export	Whatever you build	Copy-paste
Cost	Pay per result	Engineering time	Analyst hours

Configure the run

Drive the Actor with a single search query using plain keywords or INSPIRE syntax, and set a sort order and a maximum paper count to control the output volume. The Input tab lists every parameter.

A first run with the defaults:

{
 "query": "title higgs",
 "maxItems": 10
}

A larger pull:

{
 "query": "title higgs",
 "maxItems": 200
}

Pricing

Pay-per-result: $0.005 per result collected. You pay only for the results written to your dataset.

Results collected	Approximate cost
100 results	$0.50
1,000 results	$5.00
10,000 results	$50.00

New Apify accounts start with $5 in free credit.

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the INSPIRE-HEP Physics Papers Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to INSPIRE-HEP through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

$claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/inspire-hep-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check that your query is valid INSPIRE syntax. Try the same search on inspirehep.net first. Also verify that maxItems is set to at least 1 and that the query returns results for the chosen sort order.

The Actor stops before reaching my maxItems limit.

This happens when the search query returns fewer total results than your maxItems value. Try broadening your query or removing restrictive field prefixes.

Some fields are empty in my output.

Not every paper record in INSPIRE has every field populated. Older papers may lack arXiv IDs or DOIs, and some records may not have a journal reference. This is expected.

The run is taking a long time.

Large maxItems values mean more pages to fetch. INSPIRE-HEP pages load quickly, but collecting hundreds of thousands of records will take time. Reduce maxItems or run during off-peak hours.

I get an error about the query syntax.

INSPIRE query syntax is specific. Use the web search at inspirehep.net to test and refine your query, then paste the working query into the Actor input.

FAQ

Question	Answer
What is INSPIRE-HEP?	INSPIRE-HEP is the high-energy physics information system run by CERN, DESY, Fermilab, and SLAC. It indexes papers, authors, citations, and experimental data in particle physics and related fields.
Do I need an API key or account?	No. This Actor reads the public search pages, so you do not need to register an application or obtain a token.
What query syntax can I use?	You can use plain keywords like 'neutrino oscillation' or the full INSPIRE fielded syntax such as 'title higgs', 'author Witten', or 'a J.Ellis.1' for an exact author profile.
How many papers can I scrape in one run?	You set the maximum with the maxItems field, up to 1,000,000 papers per run. The Actor stops when it reaches that count or when there are no more results.
What fields does each paper record contain?	Each row includes the title, author list, citation count, publication date, journal reference, arXiv ID, DOI, and the INSPIRE record URL. The exact fields are shown in the sample output on the Actor's page.
Can I sort results by citation count?	Yes. Set the sort option to 'Most Cited' to get the highest-impact papers first, or 'Most Recent' for the latest additions.
Is this Actor affiliated with CERN or INSPIRE?	No. This is an independent tool that accesses publicly available search pages. It is not endorsed by or affiliated with CERN or the INSPIRE collaboration.
What output formats are supported?	You can export your dataset to CSV, JSON, Excel, or XML from the Apify platform.
Can I scrape papers by affiliation or experiment?	Yes. INSPIRE supports queries like 'aff CERN' or 'experiment ATLAS'. Any query that works in the web search box will work here.
Does this Actor handle pagination automatically?	Yes. It follows the search result pages until it reaches your maxItems limit or exhausts the results.

Related actors

• google-scholar-scraper: Use this if you need to scrape across all academic disciplines, not high-energy physics.
• arxiv-scraper: Use this if you want to scrape preprints directly from arXiv rather than the curated INSPIRE-HEP database.

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by CERN. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.