# Inspire Hep Papers Scraper (`bigibyte/inspire-hep-papers-scraper`) Actor Search high-energy physics literature and author records instantly by subject or keyword. Extract paper titles, citation counts, arXiv IDs, and author details into CSV, JSON, or Excel. - **URL**: https://apify.com/bigibyte/inspire-hep-papers-scraper.md - **Developed by:** [BigiByte](https://apify.com/bigibyte) (community) - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ### INSPIRE-HEP Scraper: Physics Papers and Authors

INSPIRE-HEP

Explore and extract high-energy physics research literature and author profiles directly from INSPIRE-HEP. **BigiByte**'s INSPIRE-HEP Scraper pulls citations, author names, arXiv IDs, and publication metadata into clean, structured datasetsโ€”saving you from manual database queries and pagination. *** ### What Does This Scraper Do? INSPIRE-HEP is the central digital library for high-energy physics literature, connecting researchers with global preprints, journal articles, and author profiles. This tool helps researchers, academic institutions, and science writers quickly gather comprehensive literature and author records into clean tabular formats. #### Key Use Cases - **Literature Reviews:** Pull papers for specific research topics (e.g., Higgs bosons, quantum field theory, cosmology) with citation counts and arXiv identifiers in a single dataset. - **Author Profiles & Collaborations:** Export matching author records, including names and institutional histories. - **Citation Tracking:** Analyze citation counts stored directly on each paper record to evaluate impact and follow research trajectories. - **Research Operations:** Export capped batches of structured records straight into your data notebooks or local storage. - **Scheduled Refreshes:** Automate periodic data collection runs to keep your reference lists current without manual intervention. *** ### Input Parameters | Parameter | Type | Default | Description | | --- | --- | --- | --- | | `mode` | String | `literature` | Choose `literature` for research papers or `authors` for author records. | | `query` | String | `higgs` | The search term or query string to match against records. | | `type` | String | `literature` | Optional alias for mode. | | `maxItems` | Integer | `100` | Maximum number of records to collect (capped at 1,000). | | `timeoutSecs` | Integer | `300` | Overall execution timeout in seconds. | | `requestTimeoutSecs` | Integer | `30` | Timeout per network request in seconds. | #### Example Input ```json { "mode": "literature", "query": "higgs", "maxItems": 100, "timeoutSecs": 300, "requestTimeoutSecs": 30 } ``` *** ### Extracted Output Data Each saved dataset row contains complete record metadata along with simplified top-level fields for convenience: ```json { "links": { "json": "https://inspirehep.net/api/literature/2181837?format=json", "citations": "https://inspirehep.net/api/literature/?q=refersto%3Arecid%3A2181837" }, "created": "2022-11-15T03:07:31.445632+00:00", "updated": "2023-06-14T16:26:19.477731+00:00", "metadata": { "citation_count": 1, "control_number": 2181837, "titles": [ { "source": "arXiv", "title": "Study of $HZZ$ anomalous couplings by angular differential cross sections" } ], "arxiv_eprints": [ { "categories": ["hep-ph"], "value": "2211.07478" } ] }, "id": "2181837", "paperTitle": "Study of $HZZ$ anomalous couplings by angular differential cross sections", "citationCount": 1, "arxivId": "2211.07478", "scrapedAt": "2026-10-12T00:00:00+00:00" } ``` #### Output Field Reference | Field | Type | Description | | --- | --- | --- | | `links` | Object | Direct links object associated with the record. | | `created` | String | Record creation timestamp. | | `updated` | String | Last update timestamp. | | `metadata` | Object | Detailed record metadata (sensitive fields like email addresses are automatically excluded). | | `id` | String | Unique INSPIRE record identifier. | | `paperTitle` | String | Primary paper title (literature records only). | | `citationCount` | Integer | Total citation count (when available). | | `arxivId` | String | Associated arXiv preprint identifier (when available). | | `authorName` | String | Author name (author records only). | | `scrapedAt` | String | UTC timestamp indicating when the item was collected. | | `error` | String | Populated only if a specific request encounters an issue, ensuring the task continues running smoothly. | *** ### How Data Collection Works - **Optimized Pagination:** Pages through results sequentially while respecting platform rate limits and item windows. - **Automatic Deduplication:** Duplicate record IDs are automatically filtered out to keep your dataset clean and unique. - **Resilient Retries:** Built-in network error handling manages transient rate limits or server timeouts smoothly without crashing the entire run. - **Flexible Export:** Download your collected research datasets anytime in JSON, CSV, or Excel formats. *** ### Related Tools & Custom Development If you plan to map, cross-reference, or join physics literature records with external institutional datasets, quick CSV cleanups or bibliographic mapping passes are standard next steps. **BigiByte** builds custom data scrapers, research feeds, and automated monitoring pipelines for academic institutions and scientific repositories worldwide. Need custom scheduled monitoring, specialized database integration, or tailored data pipelines? **BigiByte** builds custom scrapers and APIs tailored to your project: *** ### Frequently Asked Questions **How many records can I extract per run?** `maxItems` defaults to 100 records and can be adjusted up to a hard cap of 1,000 successful items per run. **Can I export my data directly to CSV or Excel?** Yes! Once a task completes, you can export the full dataset directly to JSON, CSV, or Excel format. **Do author records include email addresses?** No. To maintain privacy and data compliance, any fields resembling email addresses are automatically stripped before records are saved. *** ### Trademark Notice & Disclaimer **Trademark Notice:** INSPIRE-HEP and related names, logos, and trademarks belong to their respective owners. Their use in this documentation is solely to identify the source of data and does not imply affiliation, sponsorship, endorsement, or partnership. **Disclaimer:** This tool is an independent solution developed by BigiByte and is not officially affiliated with, endorsed by, or sponsored by INSPIRE-HEP. All data is collected from publicly accessible records. Users are responsible for adhering to the source's terms of service, rate limits, and licensing requirements. *** ### Is Web Scraping Legal? This tool collects publicly available physics literature and author metadata from INSPIRE-HEP. While extracting public records is standard practice, consider the following best practices: - **Personal Data:** Author names and professional profiles constitute public research data. Ensure compliance with applicable privacy regulations (such as GDPR or CCPA) if storing or processing personal information. - **Terms & Rate Limits:** Respect platform guidelines and rate limits (such as managing request pacing and adhering to metadata licensing terms like CC0 where applicable). Bulk harvesting of restricted metadata fields (such as email addresses) is strictly avoided. - **Legal Context:** This information provides general context and does not constitute legal advice. To learn more, read Apify's guide: [Is web scraping legal?](https://blog.apify.com/is-web-scraping-legal/) # Actor input Schema ## `mode` (type: `string`): Choose literature to export papers, or authors to export author records, from the public INSPIRE HEP API. ## `query` (type: `string`): INSPIRE search string passed as the q parameter. A paper search such as higgs, or an author search such as witten. ## `type` (type: `string`): Optional alias for mode. Accepted values are literature and authors. When both are set, mode is used. ## `maxItems` (type: `integer`): Maximum number of literature or author rows to save in one run. Hard cap is 1000. ## `timeoutSecs` (type: `integer`): Stop the run after this many seconds, even if max items is not reached. ## `requestTimeoutSecs` (type: `integer`): How long to wait for a single INSPIRE HTTP response before treating it as a timeout. ## `proxyConfiguration` (type: `object`): Optional proxy for requests. Leave Apify Proxy off. INSPIRE limits each IP to 15 requests in 5 seconds, and proxies must not be used to get around that limit. ## Actor input object example ```json { "mode": "literature", "query": "higgs", "type": "literature", "maxItems": 100, "timeoutSecs": 300, "requestTimeoutSecs": 30, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `datasetUrl` (type: `string`): Default dataset for this run. ## `itemsUrl` (type: `string`): JSON items from the default dataset. ## `overviewUrl` (type: `string`): Table view of literature or author rows. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "literature", "query": "higgs", "type": "literature", "maxItems": 100, "timeoutSecs": 300, "requestTimeoutSecs": 30, "proxyConfiguration": { "useApifyProxy": false } }; // Run the Actor and wait for it to finish const run = await client.actor("bigibyte/inspire-hep-papers-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "literature", "query": "higgs", "type": "literature", "maxItems": 100, "timeoutSecs": 300, "requestTimeoutSecs": 30, "proxyConfiguration": { "useApifyProxy": False }, } # Run the Actor and wait for it to finish run = client.actor("bigibyte/inspire-hep-papers-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "literature", "query": "higgs", "type": "literature", "maxItems": 100, "timeoutSecs": 300, "requestTimeoutSecs": 30, "proxyConfiguration": { "useApifyProxy": false } }' | apify call bigibyte/inspire-hep-papers-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,bigibyte/inspire-hep-papers-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/nCwxdvG06qe7uAZSx/builds/DUIy8Hp6T8xaUU6n3/openapi.json