CORE Open Access Papers Scraper

Scrape 300M+ open-access research papers from CORE, the world's largest aggregator of open-access research. Search any topic and get clean, structured JSON with titles, abstracts, authors, DOIs, publication years, publishers, fields of study, citation counts and direct PDF download links — ready for literature reviews, RAG pipelines and bibliometric analysis.

No API key. No login. No proxy required.

---

Why this actor

CORE indexes papers from more than 10,000 repositories and journals worldwide — university archives, arXiv mirrors, institutional repositories and open-access publishers. This actor turns that index into a spreadsheet or a JSON feed in seconds.

• Zero configuration — type a search query, press Start.
• PDF links included — most records carry a direct download_url to the open-access full text.
• Datacenter-clean — the public CORE search endpoint answers without proxies, so runs stay cheap.
• Permissive output — every field is nullable, so one odd repository record never fails your run.

---

Pricing

Roughly $0.50 per 1,000 papers on the Apify platform (compute units only — no proxy costs, no API subscription). A 100-paper test run costs a few cents.

---

Use cases

• Literature review — pull every paper matching a topic, then sort by year or citation count in Excel.
• Open access research — build a list of freely downloadable PDFs on a subject, with no paywall friction.
• RAG corpora — harvest titles + abstracts + full-text links to feed a retrieval-augmented generation pipeline or a custom research assistant.
• Bibliometrics — analyse publication trends, prolific authors, repository coverage and citation distributions across a field.

---

Input

Field	Type	Default	Description
mode	select	search	Scraping mode. search runs your queries against the CORE index.
queries	array	["machine learning", "climate change"]	One or more free-text queries. CORE query syntax is supported, e.g. title:(deep learning) AND yearPublished>2020.
maxItems	integer	100	Maximum total papers across all queries (max 1000).

Example input

{
    "mode": "search",
    "queries": ["quantum computing", "CRISPR gene editing"],
    "maxItems": 500
}

---

Output

One dataset item per paper.

{
    "core_id": "171513974",
    "title": "Learning curves for decision making in supervised machine learning: a survey",
    "abstract": "Learning curves are a concept from social sciences that has been adopted in the context of machine learning...",
    "authors": ["van Rijn J.N.", "Mohr F."],
    "year": 2024,
    "published_date": "2024-01-01T00:00:00+00:00",
    "doi": "10.1007/s10994-024-06619-7",
    "publisher": null,
    "document_type": null,
    "language": "English",
    "fields_of_study": ["Computer Science"],
    "citation_count": 0,
    "download_url": "https://core.ac.uk/download/641684400.pdf",
    "full_text_url": null,
    "data_providers": ["Intellectum (Universidad de La Sabana)"],
    "url": "https://core.ac.uk/works/171513974",
    "source": "core",
    "scraped_at": "2026-07-28T12:00:00+00:00"
}

Field reference

Field	Description
core_id	CORE work identifier
title	Paper title
abstract	Abstract, capped at 5,000 characters
authors	List of author names
year	Publication year
published_date	Publication date, ISO-8601
doi	Digital Object Identifier
publisher	Publisher name, when reported
document_type	Document type, when reported
language	Language name
fields_of_study	Subject / discipline labels
citation_count	Citations known to CORE
download_url	Direct open-access PDF link
full_text_url	Alternative full-text location
data_providers	Repositories that supplied the record
url	Canonical CORE page
source	Always core
scraped_at	UTC scrape timestamp

Every field is nullable — coverage varies by source repository.

---

Notes and limits

• Results are paginated 50 per request; maxItems caps the total.
• CORE's public endpoint occasionally rate-limits; the actor retries with backoff.
• Full text bodies are not exposed to public API users — use download_url for the PDF.
• Data is aggregated open-access metadata, published by CORE for reuse.

---

Related actors

• OpenAlex Scraper — scholarly works, authors and institutions graph
• Crossref Scraper — DOI metadata for 150M+ registered records
• Unpaywall Scraper — find legal free PDFs for any DOI
• Figshare Scraper — datasets, figures and preprints

---

Support

Issues and feature requests: open an issue on the actor page.