HAL Open Science Scraper

HAL Open Science Scraper

Scrape research publications from HAL Open Science by search term, domain, or document type, up to a million per run. Every record comes with its title, authors, abstract, publication date, and open-access links. No API key required. Export to CSV, JSON, Excel, or XML.

HAL is France's national open-access repository for scholarly work, but browsing its web interface or using its API means learning a custom query language and handling pagination yourself. This Actor reads the public search feeds directly, filtered by free-text query, document type, research domain, year, or open-access status, and returns each matching publication in one fixed schema.

Who uses it	What they scrape HAL Open Science for
Research librarians	Building a departmental publication list for an annual report.
PhD students	Gathering the latest preprints and theses in a narrow research area.
Science-of-science analysts	Tracking open-access rates across French institutions over time.
Grant administrators	Verifying that funded projects have deposited their deliverables in HAL.

What it does

This Actor collects HAL Open Science publications by search term, document type, domain, or year, and returns each one as a flat row.

• 🔍 Free-text search: query across titles, abstracts, and keywords with wildcard support (e.g. machine*).
• 📄 Document type filter: restrict to journal articles, conference papers, theses, preprints, and more.
• 🌐 Research domain filter: narrow by HAL domain codes like 'info' for computer science or 'shs' for humanities.
• 📅 Year filter: limit results to a single publication year.
• 🔓 Open-access toggle: collect only records that have a full-text file available.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with HAL Open Science data

📈 Monitor a research field.

A PhD candidate runs a weekly scrape for 'reinforcement learning' preprints to stay current with new submissions.

📋 Build an institutional bibliography.

A librarian collects all journal articles from a lab's HAL collection for an annual activity report.

🔓 Audit open-access compliance.

A research administrator filters a domain's publications by the open-access toggle to measure deposit rates.

🗂️ Populate a literature review.

A researcher gathers all theses and conference papers matching a set of keywords for a systematic review.

Why choose this scraper

	What you get
No API key	Reads HAL's public feeds directly, no registration or authentication needed.
Fixed flat schema	Every publication arrives with the same columns, ready for analysis.
High volume	Collect up to a million records in a single run.
Multi-format export	Download your dataset as CSV, JSON, Excel, or XML.

How it compares

No other Store actor targets HAL Open Science the same way, so the honest comparison is with the alternatives teams actually weigh.

	HAL Open Science Scraper	Build it in-house	By hand
Setup	Run it now, zero config	Days of engineering	None, but hours per pull
When HAL Open Science changes	Maintained for you	You fix it	You re-learn the page
Proxies, retries, anti-bot	Built in	Your problem	Browser only
Output	Fixed JSON schema, CSV/Excel export	Whatever you build	Copy-paste
Cost	Pay per result	Engineering time	Analyst hours

Configure the run

Drive the Actor from a free-text search query, document type, research domain, publication year, and an open-access toggle, alone or together, and filters run as each record is read so only matches reach your dataset. The Input tab lists every parameter.

A first run with the defaults:

{
  "maxItems": 10,
  "query": "machine learning"
}

A larger pull:

{
  "maxItems": 200,
  "query": "machine learning"
}

Pricing

Pay-per-result: $0.00999 per result collected. You pay only for the results written to your dataset.

Results collected	Approximate cost
100 results	$1.00
1,000 results	$9.99
10,000 results	$99.90

New Apify accounts start with $5 in free credit.

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the HAL Open Science Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to HAL Open Science through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

$claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/hal-open-science-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check that your query terms are spelled correctly and that your filters (especially domain code and year) are not too restrictive. Try running with only a broad query first, then add filters one by one.

The run stopped before reaching my maxItems limit.

This means HAL returned no more results for your query and filter combination. Try broadening your search term or removing some filters.

Some records are missing their abstract or author list.

HAL records are deposited by authors and vary in completeness. Missing fields will appear as empty values in your dataset; this is expected for some older or less complete deposits.

I get an error when using a domain code.

Domain codes are case-sensitive short strings like 'info' or 'shs'. Verify the code on hal.science/browse and enter it exactly as shown.

The open-access filter returns fewer results than I expected.

Many HAL records do not have a full-text file attached, even if the abstract is public. The toggle strictly filters for records with a file; disable it to see all metadata.

FAQ

Question	Answer
Do I need a HAL account or API key to use this scraper?	No. The Actor reads HAL's public search pages, so no registration, API key, or authentication is required.
What is a HAL research domain code?	HAL organizes publications by domain codes like 'info' (computer science), 'shs' (humanities), or 'sdv' (life sciences). You can browse the full list at hal.science/browse and enter the code in the domain filter.
Can I search for a phrase or use wildcards?	Yes. The query field supports free-text search across titles, abstracts, and keywords. You can use wildcards like 'machine*' to match 'machine learning' and 'machinery'.
How do I get only open-access publications?	Enable the 'Open Access Only' checkbox in the input. The Actor will then return only records that have a full-text file attached.
What document types can I filter by?	You can choose from journal articles, conference papers, book chapters, books, PhD theses, habilitation theses, reports, presentations, posters, preprints, and others.
How many publications can I scrape in one run?	You can set the maximum up to 1,000,000 publications. The Actor will stop when it reaches that number or when no more results match your filters.
Does this scraper get the full text of articles?	No. It collects the metadata and the link to the full-text file when available, but it does not download the PDF or document body itself.
Can I combine filters, like a domain and a year?	Yes. All filters (query, document type, domain, year, open-access) work together. Only publications that satisfy every active filter are returned.
What output formats are supported?	You can export your dataset to CSV, JSON, Excel, or XML from the Apify platform.
Is this scraper suitable for a full HAL dump?	It is designed for targeted collections. For a complete repository dump, you would need to run many broad queries, which may be slow and is not the intended use case.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by CCSD / CNRS. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.