Google Scholar Public Profiles Scraper

Extract normalized data from public Google Scholar profile pages. Version 2 uses Scholar's server-rendered HTML and public structured publication-pagination response, so profile and batch modes normally finish in seconds without launching a browser.

Modes

• profileId: one Scholar user ID or full public profile URL.
• profileIds / profileUrls: deduplicated batch mode.
• query / queries: Scholar's public author-search form, followed by full profile extraction. Google may redirect this route to sign-in in some regions; the Actor reports that restriction and does not bypass it.
• includePublications: false: fast profile-summary mode.
• maxPublications: 0–1,000 publications per profile. Values over 100 use bounded pagination.
• publicationSort: citations or recent.
• maxConcurrency / minRequestDelayMs: request pressure controls (default: one request at a time, 1200 ms jittered pause).

At least one ID, URL, or query is required. maxItems applies across the run and maxProfilesPerQuery applies independently to every query.

Input

Field	Type	Default
profileId	string	—
profileIds	array of strings	—
profileUrls	array of strings	—
query	string	—
queries	array of strings	—
maxItems	integer	25
maxProfilesPerQuery	integer	5
hl	string	en
includePublications	boolean	true
maxPublications	integer	100
publicationSort	string	citations
maxConcurrency	integer	1
minRequestDelayMs	integer	1200
maxRequestRetries	integer	3
requestTimeoutSecs	integer	45
failOnNoResults	boolean	false
proxyConfiguration	object	Apify Residential

The complete input schema, including limits and proxy defaults, is in .actor/input_schema.json.

Output

Each row is one public profile with a stable Scholar ID, identity fields, verified domain when published, interests, homepage and image URLs, all-time and recent citation metrics, citation history, publication details, co-authors, provenance, and scrape metadata. Profiles discovered through author search also retain the public search card's query, position, citation count, affiliation, verified domain, and interests. When the profile page displays it, publicAccessByMandate records the article counts and details link under the visible “Based on funding mandates” label. Empty optional values are omitted; zero metrics and zero citation counts are preserved as valid numeric values.

The full dataset field schema is in .actor/dataset_schema.json.

Proxy and access boundaries

Direct HTTP is fastest for local development, while the cloud input schema defaults to authorized Apify Residential routing because Scholar rejects the tested Apify datacenter IPs with HTTP 403. Custom HTTP proxies are supported. GOOGLE_SERP is intentionally rejected because it supports ordinary Google Search requests, not scholar.google.com profile pages. The Actor does not solve CAPTCHAs, bypass sign-in, access private profiles, or defeat access controls.

Reliability

Responses are checked for status, expected content type, maximum size, challenges, authentication redirects, malformed pagination payloads, and selector drift. Empty optional values are omitted; zero metrics and zero citation counts are preserved as valid numeric values. A challenge verdict requires an empty parse as well as the challenge wording, so a healthy page that happens to mention it is never discarded.

Requests are serialized by default (maxConcurrency: 1) and separated by a jittered minRequestDelayMs pause (default 1200 ms), because Scholar rate-limits concurrent requests from one IP. A query costs exactly one author-search list request; the profiles are derived from that page and de-duplicated against directly supplied IDs, so no profile is fetched twice. A profile costs exactly one request: the publication pagination request is skipped when the profile page proves it already lists every publication (its "Show more" control is disabled).

The whole run shares one sticky proxy session — one egress IP, the way a browser behaves. This was measured on 2026-10-25 against scholar.google.com through Apify Residential/US: a brand-new proxy session per request hit an already rate-limited exit in 4 of 19 requests (~21 % HTTP 429), while one session reused for three sequential profile requests returned 9/9 HTTP 200 across three trials.

HTTP 429 is treated as rate limiting, not a permanent block: the Actor waits the Retry-After value when Scholar sends one, otherwise a longer jittered rate-limit backoff (4 s base, doubling, capped at 30 s), and the retry rotates the proxy session so it leaves from a fresh egress IP. HTTP 403, CAPTCHA/unusual-traffic pages and Google sign-in redirects are never retried — they fail closed and are reported. Missing public profiles return a clean empty dataset unless failOnNoResults: true. Records are written atomically after the crawl succeeds, and a partial run reports every failed request (type, error code, status) in OUTPUT.failedRequests instead of silently shrinking the dataset.