Semantic Scholar Scraper — Papers, Citations & Authors

Search 200M+ academic papers with citation counts and open-access PDFs — or batch-look-up by DOI/arXiv/PMID — and pull author metrics. Sort by citations, filter by year and field. Free, keyless (optional key raises limits).

Built on the Semantic Scholar Graph API, using the reliable bulk-search and batch endpoints (not the heavily throttled plain search), so large runs actually complete.

What you get

Papers (type: "paper"):

• paperId, title, year, publicationDate, venue
• Impact — citationCount, influentialCitationCount, referenceCount
• Cross-IDs — doi, arxivId, pubMedId
• authors (name + authorId), fieldsOfStudy, publicationTypes
• openAccessPdf (direct PDF URL when available), abstract, url

Authors (type: "author") — name, affiliations, homepage, paperCount, citationCount, hIndex, url

Output sample

{
  "type": "paper",
  "paperId": "204e3073870fae3d05bcbc2f6a8e263d9b72e776",
  "title": "Attention Is All You Need",
  "year": 2017,
  "publicationDate": "2017-06-12",
  "venue": "Neural Information Processing Systems",
  "authors": [{ "name": "Ashish Vaswani", "authorId": "40348417" }],
  "citationCount": 98213,
  "influentialCitationCount": 15042,
  "referenceCount": 41,
  "doi": "10.5555/3295222.3295349",
  "arxivId": "1706.03762",
  "pubMedId": null,
  "fieldsOfStudy": ["Computer Science"],
  "publicationTypes": ["JournalArticle", "Conference"],
  "openAccessPdf": "https://arxiv.org/pdf/1706.03762",
  "abstract": "The dominant sequence transduction models…",
  "url": "https://www.semanticscholar.org/paper/204e3073870fae3d05bcbc2f6a8e263d9b72e776"
}

Modes

• paper-search — search papers by query, sort, year, fieldsOfStudy (reliable bulk endpoint).
• paper-detail — batch-look-up by ids (S2 id, DOI:…, ARXIV:…, PMID:…).
• author-search — find authors by name (throttled pool — add a key).
• author-papers — all papers for given authorIds (throttled pool — add a key).

Example input

{
  "mode": "paper-search",
  "query": "large language models",
  "sort": "citationCount:desc",
  "year": "2020-2024",
  "maxResults": 100
}

By DOI/arXiv:

{ "mode": "paper-detail", "ids": ["ARXIV:1706.03762", "ARXIV:2005.14165"] }

Why this actor

• Runs at scale — uses the bulk/batch pool that isn't rate-throttled, unlike naive scrapers.
• Impact metrics — citation and influential-citation counts, not just titles.
• Direct PDFs & cross-IDs — open-access PDF URLs plus DOI/arXiv/PMID for joining.
• Keyless for papers; add an optional free key for author endpoints and higher throughput.

Pricing

Pay per result — $1.50 per 1,000 results (one paper or author = one result). No subscription or platform fees.

Related actors

• arXiv Scraper — preprints and metadata direct from arXiv.
• CrossRef Scholarly Works Scraper — works and citation links via CrossRef.
• PubChem Scraper — chemical compound data for chem/pharma research.

Notes & limits

• Data is Semantic Scholar's official Graph API; the actor backs off on 429/5xx.
• Author endpoints share a throttled anonymous pool — provide a free API key (from semanticscholar.org/product/api) for reliable author results and higher limits.
• This actor is an independent tool, not affiliated with or endorsed by the Allen Institute for AI / Semantic Scholar.