Google Scholar Scraper — Academic Papers & Citations

Scrape Google Scholar search results into structured JSON — title, authors, publication year, journal/venue, abstract snippet, PDF link, citation count, and Scholar cluster ID. This academic paper scraper returns 17 fields per record including parse_confidence, making it the most complete Google Scholar data extraction tool available. Ideal for systematic literature reviews, citation network analysis, research gap discovery, and AI knowledge base enrichment.

Not affiliated with Google or Google Scholar.

Why this Google Scholar scraper?

• 17 fields per record — more than any competing actor; includes PDF link, cluster ID, related/versions URLs
• parse_confidence on every record — machine-readable freshness signal that drops when Scholar changes its HTML structure, before silent failures hit your pipeline
• Year range filter (yearFrom / yearTo) — narrow results to specific publication windows
• Multi-language support via language parameter
• Residential proxy support for bulk scraping without Scholar rate-limiting
• Structural HTML parsing anchored to data-cid / data-rp / gs_ri / gs_a / gs_rs / gs_fl — not fragile CSS class names that break on every Scholar update

Output Schema

Field	Type	Description
title	string	Paper / book title
authors	list[string]	Author name list
publication_year	integer	Publication year
venue	string	Journal / conference / publisher
snippet	string	Abstract fragment shown by Scholar
result_url	string	Link to the paper or book page
pdf_url	string	Direct PDF link (when available)
pdf_host	string	Domain serving the PDF
citations_count	integer	"Cited by N" count
cluster_id	string	Scholar internal cluster ID (stable)
related_url	string	"Related articles" Scholar URL
versions_url	string	"All N versions" Scholar URL
result_type	string	article / book / citation / patent
query	string	Input query (for batch traceability)
rank	integer	0-based rank in result set
parse_confidence	float	0.0–1.0 parse quality signal
warnings	list[string]	Machine-readable issue codes
scraped_at	string	ISO-8601 UTC timestamp

Input

{
  "query": "deep learning image recognition",
  "maxResults": 100,
  "yearFrom": 2020,
  "yearTo": 2024,
  "language": "en",
  "proxyUrl": "http://user:pass@proxy-host:port"
}

Parameters

Parameter	Required	Default	Description
query	yes	—	Scholar search query
maxResults	no	100	Max papers to return
yearFrom	no	—	Filter: papers from this year
yearTo	no	—	Filter: papers up to this year
language	no	en	Interface language code
proxyUrl	no	—	Residential proxy URL for bulk runs

Usage Notes

Small runs (≤ 30 results): work without a proxy from a clean IP (tested).

Bulk runs (> 30 results): Google Scholar rate-limits datacenter IPs aggressively. Pass a residential proxy URL via proxyUrl to enable reliable bulk scraping. Apify residential proxies are supported.

parse_confidence: A value below 0.8 means Scholar's page structure may have changed. The actor logs a warning and continues — buyers can filter by confidence to surface data quality issues.

Use cases for Google Scholar data extraction

• Systematic literature review — pull all papers on a topic into a spreadsheet, filter by year and citation count
• Citation network mapping — follow cluster IDs and citation counts to reconstruct influence graphs
• Research gap analysis — find underexplored topics by comparing search volume vs paper volume
• Author discovery — search author:"Y LeCun" to pull a researcher's full publication list
• Topic trend analysis — compare paper counts by year to chart field growth or decline
• AI knowledge base enrichment — feed structured paper metadata to RAG pipelines and LLM agents
• Grant writing support — quickly compile references and citation statistics for a field

Frequently asked questions

Do I need a Google API key? No. This scraper parses the public Google Scholar HTML search page. No API key or login is required.

How many results can I get? Google Scholar caps results at 1,000 per query (10 results × 100 pages). For more coverage, split into narrower queries (year ranges, subtopics).

When do I need a proxy? Small runs (≤ 30 results) work without a proxy. For bulk extraction (> 30), Scholar aggressively rate-limits datacenter IPs — pass a residential proxy URL via proxyUrl.

What does parse_confidence mean? A float 0.0–1.0. Values ≥ 0.8 = all fields extracted cleanly. Below 0.8 = Scholar may have changed its structure; check warnings for codes.

Can I scrape by author? Yes — use Scholar's standard query syntax: author:"Geoffrey Hinton" or author:"Y LeCun" neural networks.

Integrations

Built for academic researchers and R&D teams building literature reviews, citation networks, and AI knowledge bases — the JSON/dataset output drops into the tools you already run, no glue code:

• n8n / Make / Zapier — trigger a run or pipe every new dataset item into 500+ apps (Google Sheets, Airtable, Slack, HubSpot, your database) with no code: n8n, Make, Zapier.
• Webhooks — fire your own endpoint the moment a run finishes, to push results straight into your pipeline (docs).
• MCP server — expose this actor as a tool to Claude, Cursor, or any MCP client so an AI agent can pull this data mid-conversation (guide).
• API & SDKs — fetch the dataset as JSON, CSV, or Excel through the Apify REST API or the Python / JS SDKs.

See all Apify integrations.