Google Scholar Scraper — Academic Papers & Citations

Scrape Google Scholar search results into structured JSON — title, authors, publication year, journal/venue, abstract snippet, PDF link, citation count, and Scholar cluster ID. This academic paper scraper returns 17 fields per record including parse_confidence, making it the most complete Google Scholar data extraction tool available. Ideal for systematic literature reviews, citation network analysis, research gap discovery, and AI knowledge base enrichment.

Not affiliated with Google or Google Scholar.

Why this Google Scholar scraper?

• 17 fields per record — more than any competing actor; includes PDF link, cluster ID, related/versions URLs
• parse_confidence on every record — machine-readable freshness signal that drops when Scholar changes its HTML structure, before silent failures hit your pipeline
• Year range filter (yearFrom / yearTo) — narrow results to specific publication windows
• Multi-language support via language parameter
• Residential proxy support for bulk scraping without Scholar rate-limiting
• Structural HTML parsing anchored to data-cid / data-rp / gs_ri / gs_a / gs_rs / gs_fl — not fragile CSS class names that break on every Scholar update

Output Schema

Field	Type	Description
title	string	Paper / book title
authors	list[string]	Author name list
publication_year	integer	Publication year
venue	string	Journal / conference / publisher
snippet	string	Abstract fragment shown by Scholar
result_url	string	Link to the paper or book page
pdf_url	string	Direct PDF link (when available)
pdf_host	string	Domain serving the PDF
citations_count	integer	"Cited by N" count
cluster_id	string	Scholar internal cluster ID (stable)
related_url	string	"Related articles" Scholar URL
versions_url	string	"All N versions" Scholar URL
result_type	string	article / book / citation / patent
query	string	Input query (for batch traceability)
rank	integer	0-based rank in result set
parse_confidence	float	0.0–1.0 parse quality signal
warnings	list[string]	Machine-readable issue codes
scraped_at	string	ISO-8601 UTC timestamp

Input

{
  "query": "deep learning image recognition",
  "maxResults": 100,
  "yearFrom": 2020,
  "yearTo": 2024,
  "language": "en",
  "proxyUrl": "http://user:pass@proxy-host:port"
}

Parameters

Parameter	Required	Default	Description
query	yes	—	Scholar search query
maxResults	no	100	Max papers to return
yearFrom	no	—	Filter: papers from this year
yearTo	no	—	Filter: papers up to this year
language	no	en	Interface language code
proxyUrl	no	—	Residential proxy URL for bulk runs

Usage Notes

Small runs (≤ 30 results): work without a proxy from a clean IP (tested).

Bulk runs (> 30 results): Google Scholar rate-limits datacenter IPs aggressively. Pass a residential proxy URL via proxyUrl to enable reliable bulk scraping. Apify residential proxies are supported.

parse_confidence: A value below 0.8 means Scholar's page structure may have changed. The actor logs a warning and continues — buyers can filter by confidence to surface data quality issues.

Use cases for Google Scholar data extraction

• Systematic literature review — pull all papers on a topic into a spreadsheet, filter by year and citation count
• Citation network mapping — follow cluster IDs and citation counts to reconstruct influence graphs
• Research gap analysis — find underexplored topics by comparing search volume vs paper volume
• Author discovery — search author:"Y LeCun" to pull a researcher's full publication list
• Topic trend analysis — compare paper counts by year to chart field growth or decline
• AI knowledge base enrichment — feed structured paper metadata to RAG pipelines and LLM agents
• Grant writing support — quickly compile references and citation statistics for a field

Frequently asked questions

Do I need a Google API key? No. This scraper parses the public Google Scholar HTML search page. No API key or login is required.

How many results can I get? Google Scholar caps results at 1,000 per query (10 results × 100 pages). For more coverage, split into narrower queries (year ranges, subtopics).

When do I need a proxy? Small runs (≤ 30 results) work without a proxy. For bulk extraction (> 30), Scholar aggressively rate-limits datacenter IPs — pass a residential proxy URL via proxyUrl.

What does parse_confidence mean? A float 0.0–1.0. Values ≥ 0.8 = all fields extracted cleanly. Below 0.8 = Scholar may have changed its structure; check warnings for codes.

Can I scrape by author? Yes — use Scholar's standard query syntax: author:"Geoffrey Hinton" or author:"Y LeCun" neural networks.

Integrations

Built for academic researchers and R&D teams building literature reviews, citation networks, and AI knowledge bases — the JSON/dataset output drops into the tools you already run, no glue code:

• n8n / Make / Zapier — trigger a run or pipe every new dataset item into 500+ apps (Google Sheets, Airtable, Slack, HubSpot, your database) with no code: n8n, Make, Zapier.
• Webhooks — fire your own endpoint the moment a run finishes, to push results straight into your pipeline (docs).
• MCP server — expose this actor as a tool to Claude, Cursor, or any MCP client so an AI agent can pull this data mid-conversation (guide).
• API & SDKs — fetch the dataset as JSON, CSV, or Excel through the Apify REST API or the Python / JS SDKs.

See all Apify integrations.

Usage statistics

This Actor creates a small, content-free summary at the end of each run. It is used only to monitor reliability and improve this Actor. A copy is saved as USAGE_STATS in your own Apify key-value store, so you can see the exact record created for your run.

Set disableUsageStats to true in the input to opt out. Nothing is sent then; your USAGE_STATS record only says that statistics were disabled.

Only these fields are recorded:

• schema version, Actor name and build number;
• UTC start and finish hour (not a precise timestamp);
• run duration, number of results and time to the first result, each as a coarse range;
• whether the result was empty, the end status, and an error type from a fixed list;
• memory setting and counts of charged events;
• names of the input fields you set, never their values;
• the selected option for input fields that offer a fixed list of choices (for example a sort order).

We do not collect input text, search terms, URLs, domains, usernames, email addresses, names, proxy credentials, tokens, scraped records, output items, raw error messages, stack traces, or hashes of any of those values. Records are kept for no longer than 13 months, used only as aggregated operational statistics, and never sold or shared.

Additional fields (Phase 2)

This Actor also records your Apify user ID, whether Apify marks the account as paying, the size range of list inputs, the selected country when the input offers a fixed list of countries, and one category from a fixed Actor taxonomy. We use these fields only for aggregate reliability, repeat-use and cross-Actor analysis; reports suppress any cell with fewer than five distinct users.

The same disableUsageStats: true input flag turns these fields off too. The user ID is removed after 13 months; we do not export, sell, share, or attempt to re-identify this data.

Run-outcome signals (v2)

To learn whether a run did what it was asked to do, the record also holds a few more coarse ranges and yes/no flags. None of them contains content:

• the result limit you asked for (a range, when the input has one) and what share of it was delivered;
• results delivered per input item you listed (a range);
• output quality as ranges: how fully the result fields were filled, the share of rows that look like errors, the share of duplicate rows, and how many different fields appeared. These are counted in memory while results are saved; no result content is kept;
• how the run was started (console, API, schedule, webhook, another Actor);
• how it ended: stopped by you, timed out, reached the requested limit, stopped by the charge limit, and how many times the platform moved the run;
• if this Actor reports it: how many items to process worked or failed (ranges) and one failure reason from a fixed list;
• a short code made from the names of the input fields you set, never their values.

Repeat-run fingerprint (v2)

When your Apify user ID is recorded (see above), the record also holds an 8-character one-way code made from your input (proxy settings left out) and this Actor's name. It only lets us see that the same account ran the same input again soon after an unsatisfying run; we never see the input itself. It is stored only in the database, never published, and reports use it in aggregate with the same five-user minimum. It is the one exception to the statement above that no hashes are collected, and disableUsageStats: true turns it off.