# Google Scholar Scraper — Papers & Citations (`bovi/google-scholar-scraper`) Actor

Scrape academic paper records from Google Scholar. Returns title, authors, publication year, venue, snippet, PDF link, citation count, and cluster ID. Year range and language filters. Parse-confidence on every record. More fields than any competitor.

- **URL**: https://apify.com/bovi/google-scholar-scraper.md
- **Developed by:** [Vitalii Bondarev](https://apify.com/bovi) (community)
- **Categories:** Education, Automation, AI
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.20 / 1,000 google scholar scraper — papers & citations

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Google Scholar Scraper — Academic Papers & Citations

Scrape **Google Scholar** search results into structured JSON — title, authors, publication year, journal/venue, abstract snippet, PDF link, citation count, and Scholar cluster ID. This **academic paper scraper** returns **17 fields per record** including `parse_confidence`, making it the most complete Google Scholar data extraction tool available. Ideal for **systematic literature reviews**, citation network analysis, research gap discovery, and AI knowledge base enrichment.

**Not affiliated with Google or Google Scholar.**

### Why this Google Scholar scraper?

- **17 fields per record** — more than any competing actor; includes PDF link, cluster ID, related/versions URLs
- **parse\_confidence** on every record — machine-readable freshness signal that drops when Scholar changes its HTML structure, before silent failures hit your pipeline
- Year range filter (`yearFrom` / `yearTo`) — narrow results to specific publication windows
- Multi-language support via `language` parameter
- Residential proxy support for bulk scraping without Scholar rate-limiting
- Structural HTML parsing anchored to `data-cid` / `data-rp` / `gs_ri` / `gs_a` / `gs_rs` / `gs_fl` — not fragile CSS class names that break on every Scholar update

### Output Schema

| Field | Type | Description |
|---|---|---|
| `title` | string | Paper / book title |
| `authors` | list\[string] | Author name list |
| `publication_year` | integer | Publication year |
| `venue` | string | Journal / conference / publisher |
| `snippet` | string | Abstract fragment shown by Scholar |
| `result_url` | string | Link to the paper or book page |
| `pdf_url` | string | Direct PDF link (when available) |
| `pdf_host` | string | Domain serving the PDF |
| `citations_count` | integer | "Cited by N" count |
| `cluster_id` | string | Scholar internal cluster ID (stable) |
| `related_url` | string | "Related articles" Scholar URL |
| `versions_url` | string | "All N versions" Scholar URL |
| `result_type` | string | article / book / citation / patent |
| `query` | string | Input query (for batch traceability) |
| `rank` | integer | 0-based rank in result set |
| `parse_confidence` | float | 0.0–1.0 parse quality signal |
| `warnings` | list\[string] | Machine-readable issue codes |
| `scraped_at` | string | ISO-8601 UTC timestamp |

### Input

```json
{
  "query": "deep learning image recognition",
  "maxResults": 100,
  "yearFrom": 2020,
  "yearTo": 2024,
  "language": "en",
  "proxyUrl": "http://user:pass@proxy-host:port"
}
```

#### Parameters

| Parameter | Required | Default | Description |
|---|---|---|---|
| `query` | **yes** | — | Scholar search query |
| `maxResults` | no | 100 | Max papers to return |
| `yearFrom` | no | — | Filter: papers from this year |
| `yearTo` | no | — | Filter: papers up to this year |
| `language` | no | `en` | Interface language code |
| `proxyUrl` | no | — | Residential proxy URL for bulk runs |

### Usage Notes

**Small runs (≤ 30 results):** work without a proxy from a clean IP (tested).

**Bulk runs (> 30 results):** Google Scholar rate-limits datacenter IPs
aggressively. Pass a residential proxy URL via `proxyUrl` to enable
reliable bulk scraping. Apify residential proxies are supported.

**parse\_confidence:** A value below 0.8 means Scholar's page structure
may have changed. The actor logs a warning and continues — buyers can
filter by confidence to surface data quality issues.

### Use cases for Google Scholar data extraction

- **Systematic literature review** — pull all papers on a topic into a spreadsheet, filter by year and citation count
- **Citation network mapping** — follow cluster IDs and citation counts to reconstruct influence graphs
- **Research gap analysis** — find underexplored topics by comparing search volume vs paper volume
- **Author discovery** — search `author:"Y LeCun"` to pull a researcher's full publication list
- **Topic trend analysis** — compare paper counts by year to chart field growth or decline
- **AI knowledge base enrichment** — feed structured paper metadata to RAG pipelines and LLM agents
- **Grant writing support** — quickly compile references and citation statistics for a field

### Frequently asked questions

**Do I need a Google API key?** No. This scraper parses the public Google Scholar HTML search page. No API key or login is required.

**How many results can I get?** Google Scholar caps results at 1,000 per query (10 results × 100 pages). For more coverage, split into narrower queries (year ranges, subtopics).

**When do I need a proxy?** Small runs (≤ 30 results) work without a proxy. For bulk extraction (> 30), Scholar aggressively rate-limits datacenter IPs — pass a residential proxy URL via `proxyUrl`.

**What does parse\_confidence mean?** A float 0.0–1.0. Values ≥ 0.8 = all fields extracted cleanly. Below 0.8 = Scholar may have changed its structure; check `warnings` for codes.

**Can I scrape by author?** Yes — use Scholar's standard query syntax: `author:"Geoffrey Hinton"` or `author:"Y LeCun" neural networks`.

### Integrations

Built for academic researchers and R\&D teams building literature reviews, citation networks, and AI knowledge bases — the JSON/dataset output drops into the tools you already run, no glue code:

- **n8n / Make / Zapier** — trigger a run or pipe every new dataset item into 500+ apps (Google Sheets, Airtable, Slack, HubSpot, your database) with no code: [n8n](https://docs.apify.com/platform/integrations/n8n), [Make](https://docs.apify.com/platform/integrations/make), [Zapier](https://docs.apify.com/platform/integrations/zapier).
- **Webhooks** — fire your own endpoint the moment a run finishes, to push results straight into your pipeline ([docs](https://docs.apify.com/platform/integrations/webhooks)).
- **MCP server** — expose this actor as a tool to Claude, Cursor, or any [MCP client](https://mcp.apify.com) so an AI agent can pull this data mid-conversation ([guide](https://blog.apify.com/how-to-use-mcp/)).
- **API & SDKs** — fetch the dataset as JSON, CSV, or Excel through the Apify REST API or the Python / JS SDKs.

See all [Apify integrations](https://apify.com/integrations).

# Actor input Schema

## `query` (type: `string`):

Google Scholar search query. Supports all Scholar syntax: 'author:"Y LeCun"', 'intitle:transformer', 'site:arxiv.org', exact phrase with quotes, etc. Example: 'deep learning image recognition'.

## `maxResults` (type: `integer`):

Maximum number of paper records to return. Each page has 10 results. Keep under 100 for quick lookups; use 500+ for systematic literature reviews. High values need a residential proxy to avoid rate-limiting.

## `yearFrom` (type: `integer`):

Filter: only include papers published from this year onwards. Leave blank for no lower bound. Example: 2020.

## `yearTo` (type: `integer`):

Filter: only include papers published up to and including this year. Leave blank for no upper bound. Example: 2024.

## `language` (type: `string`):

Scholar interface language (hl parameter). Affects Scholar's display language and some result ordering. Use standard language codes: 'en' (English), 'de' (German), 'fr' (French), 'zh' (Chinese), etc.

## `proxyConfiguration` (type: `object`):

Proxy used to fetch Google Scholar. Defaults to Apify Residential proxy — Scholar blocks datacenter IPs after a few pages. Billed to the run owner; no external proxy account needed.

## Actor input object example

```json
{
  "query": "deep learning",
  "maxResults": 100,
  "language": "en",
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset containing Google Scholar Scraper records (rank, title, authors, publication\_year, venue, citations\_count, result\_type, pdf\_url, result\_url, snippet, parse\_confidence, scraped\_at).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "deep learning",
    "maxResults": 100,
    "language": "en",
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "RESIDENTIAL"
        ]
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("bovi/google-scholar-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "query": "deep learning",
    "maxResults": 100,
    "language": "en",
    "proxyConfiguration": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["RESIDENTIAL"],
    },
}

# Run the Actor and wait for it to finish
run = client.actor("bovi/google-scholar-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "deep learning",
  "maxResults": 100,
  "language": "en",
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}' |
apify call bovi/google-scholar-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,bovi/google-scholar-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/bMFx6Od0sWAfnu9WM/builds/awVl8wpZ3aqjzmk1V/openapi.json