Semantic Scholar Scraper - Papers, Citations, References & Authors

Pull structured academic data from Semantic Scholar: paper titles, abstracts, citation counts, authors, venues, DOIs, open-access PDF links, plus the citation graph and author profiles. Search by keyword, filter by year, field of study, publication type, venue and minimum citations, or look up exact papers by DOI, arXiv ID or Semantic Scholar link.

Features

• Keyword search across the Semantic Scholar corpus, sorted by citations or by publication date
• Filters for publication year range, minimum citations, fields of study, publication types, venue and open-access PDF availability
• Look up papers by DOI, arXiv ID, PubMed ID, CorpusId or a semanticscholar.org paper link, any mix in one run
• Citations of a paper (who cites it) and references of a paper (what it cites), each row linked back to the source paper
• Author profile search with h-index, paper count, citation count, affiliations and homepage
• Up to a thousand papers per page, so long result lists come back quickly
• Flat, typed records ready for CSV, Excel or a database

Who uses this

• Literature reviewers who need the most-cited papers on a topic with abstracts in one export
• Research-intelligence and AI teams building citation datasets, retrieval corpora and recommenders
• Bibliometric analysts tracking a field, venue or author over time
• Lab and library teams assembling reading lists and open-access collections

How it works

1. Choose what to collect: a keyword search, specific papers, a paper's citations, a paper's references, or authors.
2. Add your query or paper IDs and any filters.
3. Set maxItems and run. Rows stream into the dataset as they are collected.

Input

Most-cited papers on a topic

{
    "mode": "search",
    "query": "\"graph neural network\"",
    "sortBy": "citationCount:desc",
    "maxItems": 100
}

Recent open-access medical papers

{
    "mode": "search",
    "query": "immunotherapy",
    "fieldsOfStudy": ["Medicine"],
    "yearFrom": 2023,
    "openAccessOnly": true,
    "sortBy": "publicationDate:desc",
    "maxItems": 200
}

Specific papers

{
    "mode": "papers",
    "paperIds": ["ARXIV:1706.03762", "DOI:10.1109/5.726791"],
    "maxItems": 50
}

Who cites a paper

{
    "mode": "citations",
    "paperIds": ["ARXIV:1706.03762"],
    "maxItems": 500
}

Input parameters

Parameter	Type	Description
mode	string	search, papers, citations, references or authors. Default search.
query	string	Keywords for paper search or a name for author search. Supports quoted phrases, + (and), | (or), - (exclude), * (prefix).
paperIds	array	For papers, citations and references: Semantic Scholar IDs, DOI:..., ARXIV:..., PMID:..., CorpusId:... or paper links.
sortBy	string	Search order: most cited, newest, oldest or paper ID.
yearFrom / yearTo	integer	Publication year bounds for keyword search.
minCitations	integer	Only papers with at least this many citations.
fieldsOfStudy	array	One or more fields, for example Computer Science or Medicine. Empty means all.
publicationTypes	array	Review, journal article, conference paper, dataset and more. Empty means all.
openAccessOnly	boolean	Only papers with a free PDF link.
venue	string	Journal or conference name.
maxItems	integer	Maximum rows to return.

Keyword search matches words in titles and abstracts rather than ranking by relevance, so wrap multi-word topics in quotes ("graph neural network") for a phrase match. Citation and reference lists for a single paper are available up to 10,000 entries.

Output

Paper row (abstract omitted here for length):

{
    "record_type": "paper",
    "paper_id": "162d958ff885f1462aeda91cd72582323fd6a1f4",
    "corpus_id": 14542261,
    "title": "Gradient-based learning applied to document recognition",
    "year": 1998,
    "venue": "Proceedings of the IEEE",
    "journal_volume": "86",
    "journal_pages": "2278-2324",
    "publication_types": ["JournalArticle", "Review"],
    "fields_of_study": ["Computer Science"],
    "citation_count": 63440,
    "influential_citation_count": 6512,
    "reference_count": 147,
    "is_open_access": false,
    "doi": "10.1109/5.726791",
    "authors": ["Yann LeCun", "L. Bottou", "Yoshua Bengio", "P. Haffner"],
    "author_ids": ["1688882", "52184096", "1751762", "1721248"],
    "url": "https://www.semanticscholar.org/paper/162d958ff885f1462aeda91cd72582323fd6a1f4"
}

Field	Description
paper_id, corpus_id, url	Semantic Scholar identifiers and page
title, abstract, year, publication_date	Core bibliographic data
venue, journal_volume, journal_pages	Where it was published
publication_types, fields_of_study	Classification
citation_count, influential_citation_count, reference_count	Impact counts
is_open_access, open_access_pdf_url	Free PDF availability
doi, arxiv_id, pubmed_id	External identifiers
authors, author_ids	Author names and IDs, in matching order
relation, source_paper_id, is_influential, citation_intents	On citation and reference rows: link back to the paper you asked about
author_id, author_name, affiliations, homepage, paper_count, author_citation_count, h_index	Author rows

Fields that Semantic Scholar does not hold for a paper (many records have no abstract or DOI) come back empty.

FAQ

How many results can one run return? Keyword search pages through very large result sets; set maxItems to what you need. Citations and references of a single paper stop at 10,000 entries.

Why does a search return papers that only loosely match? Search matches words anywhere in the title and abstract. Use quotes for exact phrases and - to exclude terms.

Can I get the full text of papers? No. You get metadata, abstracts and links to free PDFs where Semantic Scholar knows of one.

Is every paper included? Rows reflect what Semantic Scholar indexes at run time. Recent papers can appear with zero citations or without an abstract.

What format can I export to? JSON, CSV, Excel, XML or HTML from the dataset tab, or through the Apify integrations.