CORE Open Access Research Scraper

CORE Open Access Research Scraper

Scrape open access research papers from CORE, the world's largest aggregator. Get titles, authors, abstracts, publication years, and full-text links for any query, filtered by year and language. No API key required. Export to CSV, JSON, Excel, or XML.

CORE's official API requires an API key and has rate limits that slow down large-scale research. This Actor queries the public CORE v3 search endpoint directly, so you can collect metadata for thousands of open access papers without registration. Each result is returned as a flat row with consistent fields, ready for analysis or export.

Who uses it	What they scrape CORE for
Academic researchers	Building a literature review dataset for a specific topic
Librarians	Monitoring new open access publications in a subject area
Data scientists	Gathering paper metadata for bibliometric analysis
PhD students	Finding recent papers in their field without manual searching

What it does

This Actor collects open access research paper metadata from CORE by keyword or title query, and returns each paper as a flat row.

• 🔍 Keyword search: Enter any title or keyword and get matching papers from CORE's index.
• 📅 Year filters: Set a minimum and maximum publication year to narrow results to a specific time range.
• 🌐 Language filter: Restrict results to papers written in a specific language using a two-letter ISO code.
• ⚙️ Flexible limits: Collect anywhere from 1 to 1,000,000 papers per run, depending on your needs.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with CORE data

📚 Build a literature review dataset.

A researcher enters a topic keyword and a year range, then exports the resulting paper metadata to a CSV for screening in a spreadsheet.

📈 Track publication trends.

A data scientist runs the Actor monthly with the same query and year filters to collect new papers, then analyzes counts by year and language.

🌍 Monitor open access output by language.

A librarian sets the language code to 'es' and a recent year range to find Spanish-language papers added to CORE in the last quarter.

🔎 Discover papers for a systematic review.

A PhD student searches for a specific title phrase and collects all matching records, then deduplicates by CORE ID before full-text screening.

Why choose this scraper

	What you get
No API key	Uses the public CORE search endpoint, so you can start scraping immediately without registration.
Large scale	Collect up to a million paper records in a single run, limited only by CORE's public search.
Consistent schema	Every paper is returned with the same fields, making it easy to merge datasets across runs.
Export ready	Download results as CSV, JSON, Excel, or XML for use in any analysis tool.

How it compares

No other Store actor targets CORE the same way, so the honest comparison is with the alternatives teams actually weigh.

	CORE Open Access Research Scraper	Build it in-house	By hand
Setup	Run it now, zero config	Days of engineering	None, but hours per pull
When CORE changes	Maintained for you	You fix it	You re-learn the page
Proxies, retries, anti-bot	Built in	Your problem	Browser only
Output	Fixed JSON schema, CSV/Excel export	Whatever you build	Copy-paste
Cost	Pay per result	Engineering time	Analyst hours

Configure the run

Drive the Actor with a search query, and optionally filter by publication year range and language. The maximum number of papers to collect is set per run. The Input tab lists every parameter.

A first run with the defaults:

{
  "maxItems": 10
}

A larger pull:

{
  "maxItems": 200
}

Pricing

Pay-per-result: $0.0085 per result collected. You pay only for the results written to your dataset.

Results collected	Approximate cost
100 results	$0.85
1,000 results	$8.50
10,000 results	$85.00

New Apify accounts start with $5 in free credit.

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the CORE Open Access Research Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to CORE through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

$claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/core-ac-uk-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check your query for typos or overly specific phrases. Try a broader keyword. Also verify that your year range and language code are not too restrictive. CORE's public search may not index very recent papers immediately.

Why are some papers missing from the results?

CORE's public search endpoint may not return all matching records, especially for very large result sets. Try narrowing your query or year range, or run the Actor multiple times with different filters.

The Actor returns fewer papers than my 'Maximum papers' setting.

This means CORE found fewer matching papers than your limit. The Actor returns all available matches. If you expected more, broaden your query or remove filters.

I get an error about the language code.

Make sure you are using a valid two-letter ISO 639-1 code, such as 'en', 'fr', or 'de'. If you are unsure, leave the field empty to include all languages.

The run takes a long time.

Large result sets take longer to collect. Reduce the 'Maximum papers' value or narrow your query to speed up the run.

FAQ

Question	Answer
Do I need a CORE API key to use this Actor?	No. This Actor uses the public CORE search endpoint that does not require an API key. You can run it immediately without any registration.
What data does each paper record include?	Each record includes the paper title, authors, abstract, publication year, language, and links to the full text and CORE page. The exact fields are shown in the sample output.
Can I filter by publication year?	Yes. Use the 'Year from' and 'Year to' inputs to set a minimum and maximum publication year. Both are optional, so you can set only one if needed.
How do I filter by language?	Enter a two-letter ISO 639-1 language code in the 'Language code' field, for example 'en' for English, 'fr' for French, or 'es' for Spanish. Leave it empty to include all languages.
What is the maximum number of papers I can collect?	You can set 'Maximum papers' up to 1,000,000 per run. The actual number returned depends on how many results match your query on CORE.
Can I search for an exact paper title?	Yes. Enter the full title or a distinctive phrase in the 'Paper title or keyword' field. The Actor sends it as a full-text search query to CORE.
What export formats are supported?	You can export the results as CSV, JSON, Excel, or XML directly from the Apify dataset.
Is this Actor suitable for large-scale bibliometric studies?	Yes. You can collect up to a million records per run, and the flat schema makes it easy to combine multiple runs into one dataset.
Does this Actor get the full text of papers?	No. It collects metadata and links to the full text where available, but it does not download the PDF or full text content.
Can I run this Actor on a schedule?	Yes. You can set up a scheduled run in Apify to collect new papers matching your query at regular intervals.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by CORE, The Open University. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.