# Google Scholar Scraper (Papers, Citations, Free PDFs) (`webdatatools/google-scholar-scraper`) Actor

Scrape Google Scholar search results: paper titles, authors, venue, year, snippet, cited-by counts, version counts and free PDF or HTML links, from public Scholar result pages.

- **URL**: https://apify.com/webdatatools/google-scholar-scraper.md
- **Developed by:** [Murat Uzun](https://apify.com/webdatatools) (community)
- **Categories:** Other, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 75.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.20 / 1,000 papers

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Google Scholar Scraper (Papers, Citations, Free PDFs)

### What is Google Scholar Scraper?

Google Scholar Scraper collects Google Scholar search results for any topic. For each paper it returns the title, link, authors, venue, year, a snippet, the cited-by count, the number of versions and a free PDF or HTML link when Scholar shows one. Use it for literature reviews, citation tracking, lists of papers per topic and research monitoring. It reads public Scholar result pages. No Google login is used, and no cookies are sent.

Each query walks Scholar's result pages (10 papers per page) until your result limit is reached or Scholar has no more results. One dataset row is one paper, and you pay per row.

### What data does Google Scholar Scraper extract?

| Field | Type | Meaning |
|---|---|---|
| resultId | string | Scholar's own id for the paper (stable across runs) |
| position | number | Rank in the query's results (1 = first) |
| page | number | Scholar result page the row came from (10 per page) |
| query | string | The search query |
| title | string | Paper title, without the \[PDF] or \[BOOK] label |
| documentType | string or null | Scholar label such as BOOK, HTML, CITATION, when shown |
| url | string or null | Link to the publisher or source page |
| authors | string or null | Author list as Scholar prints it (often abbreviated, e.g. "R Van Noorden") |
| venue | string or null | Journal, conference or publisher line |
| year | number or null | Publication year |
| source | string or null | Domain or publisher shown by Scholar |
| snippet | string or null | Text excerpt with the matched terms |
| citedBy | number | Cited-by count (0 when Scholar shows none) |
| citedByUrl | string or null | Scholar page listing the papers that cite this one |
| versionsCount | number | Number of Scholar versions of the paper |
| versionsUrl | string or null | Scholar page listing all versions |
| freeAccessType | string or null | PDF or HTML, when Scholar shows a free copy |
| freeAccessUrl | string or null | The free copy link |
| scrapedAt | string | ISO timestamp of the scrape |

### How to use Google Scholar Scraper

1. Open the Actor and enter one or more queries in "Search queries", one per line, e.g. `solid-state battery`.
2. Set "Max results per query" (default 20, up to 100).
3. Optional: set "Published from" and "Published to" years, or turn on "Sort by date".
4. Leave "Connection mode" on Auto. It tries a direct request first and switches to a residential proxy only after Scholar blocks it.
5. Start the run, then download the dataset as JSON, CSV or Excel, or read it through the API.

### Example input

```json
{
    "queries": ["solid-state battery"],
    "maxResultsPerQuery": 20
}
```

### Example output

Real row from a run on 23 Oct 2026 (query `federated learning`, residential proxy):

```json
{
    "resultId": "FE0GVCg5twYJ",
    "position": 1,
    "page": 1,
    "query": "federated learning",
    "title": "A review of applications in federated learning",
    "documentType": null,
    "url": "https://www.sciencedirect.com/science/article/pii/S0360835220305532",
    "authors": "L Li, Y Fan, M Tse , KY Lin",
    "venue": "Computers & Industrial Engineering",
    "year": 2020,
    "source": "Elsevier",
    "snippet": "… This TTF consists of FL API and Federated Core (FC) API. In detail, FL API offers a set of …",
    "citedBy": 2339,
    "citedByUrl": "https://scholar.google.com/scholar?cites=483918330336791828&as_sdt=2005&sciodt=0,5&hl=en",
    "versionsCount": 6,
    "versionsUrl": "https://scholar.google.com/scholar?cluster=483918330336791828&hl=en&as_sdt=0,5",
    "freeAccessType": "PDF",
    "freeAccessUrl": "https://orca.cardiff.ac.uk/id/eprint/134968/1/CAIE_A%20review%20of%20applications%20in%20federated%20learning_deposit.pdf",
    "scrapedAt": "2026-10-23T10:02:44.909Z"
}
```

### Input parameters

| Parameter | Default | Description |
|---|---|---|
| queries | `solid-state battery` | Topics or exact phrases, one per line, up to 20 per run |
| maxResultsPerQuery | 20 | Papers per query, 1 to 100 |
| yearFrom | empty | Earliest publication year (four digits) |
| yearTo | empty | Latest publication year (four digits) |
| sortByDate | false | Newest papers first instead of Scholar relevance |
| proxyMode | auto | auto, direct (never proxy) or residential (always proxy) |

### Pricing

Pay per result: **$2.00 per 1,000 papers** on the free plan ($0.002 per paper). Volume tiers lower the price on paid Apify plans (down to $0.0008 per paper on Diamond).

Platform cost measured on 23 Oct 2026 at 256 MB: one run for 30 papers used $0.001 of platform usage, and one run for 10 papers used $0.002 (the CLI prints usage to three decimals, so these are rounded figures). Retries that wait on blocked requests add run time, and so cost.

### Google Scholar Scraper vs. alternatives

This Actor returns Google Scholar's own ranking, cited-by counts and version lists, so it fits when those are the numbers you need. For a complete bibliographic index with a documented API, use a scholarly index service instead; this Actor does not replace one.

### Using Google Scholar Scraper with AI agents / MCP

The Actor is available through the Apify MCP server and the Apify API. An agent sends `queries` and `maxResultsPerQuery` and reads the dataset rows. Keep `maxResultsPerQuery` small per query when an agent runs many queries, because the total charge is capped by the caller's `maxTotalChargeUsd`.

### FAQ

**Is this legal?** The Actor reads public Scholar result pages, the same pages a person sees in a browser. It does not log in and does not bypass a login. Google's terms restrict automated access, so check them for your use case. You are responsible for how you use the data.

**Why did a query return fewer results than I asked for?** Scholar stops after a few pages for some queries, and some requests are soft-blocked. The Actor retries on fresh proxy sessions with a pause between attempts. Measured on 23 Oct 2026, `transformer attention` (100 requested, from 2020) returned 8 papers before a block, `solid-state battery` (100 requested, from 2020) returned 50, and `federated learning` (30 requested) returned all 30. Rerun the query later, or split the topic by publication year, to reach more papers.

**Why did a query return 0 results?** Scholar sometimes answers a valid query with the "did not match any articles" page, which looks the same as a real empty result. In a test on 23 Oct 2026, the unquoted `graph neural network` returned 0 papers from both the direct connection and the residential proxy, while the quoted phrase returned results from the same network. Try an exact phrase in quotes, or rerun the query later.

**Why does the direct connection fail?** From Apify datacenter IPs, Scholar answered HTTP 403 to direct requests in both direct-only test runs. Auto mode therefore switches to the residential proxy after one blocked request.

**Can I call it from n8n, Make or Zapier?** Those tools can start any Apify Actor through their Apify integrations. This was not tested for this Actor.

**What about errors and limits?** A run fails only when no paper is collected at all. If some queries are blocked, the run succeeds and logs which queries were blocked. Each run is capped by the caller's maximum charge, and the Actor stops adding rows when the cap is reached.

### Related Actors

Other Actors from the same publisher are listed on its Apify profile.

### Support

Open an issue on the Actor's Issues tab on Apify. Include the run ID, the input you used and the log lines that mention "Blocked".

### Limitations

- Scholar's public pages return at most a few pages for some queries before blocking.
- Scholar's authors line is abbreviated ("R Van Noorden"), and it is not a DOI or ORCID. Use the `url` to find the full record.
- `freeAccessUrl` is the link Scholar shows, which can be a repository copy or a preprint. It is not checked for licence terms.
- Results are Scholar's ranking at scrape time. They change often, and two runs can differ.

# Actor input Schema

## `queries` (type: `array`):

Topics or exact phrases to search on Google Scholar, one per line, e.g. transformer attention mechanism or "solid-state battery". Up to 20 queries per run.

## `maxResultsPerQuery` (type: `integer`):

How many papers to return per query, e.g. 50. Scholar shows 10 per page; the Actor follows pages until this number is reached or Scholar runs out of results.

## `yearFrom` (type: `string`):

Optional. Earliest publication year, four digits, e.g. 2020. Leave empty for all years.

## `yearTo` (type: `string`):

Optional. Latest publication year, four digits, e.g. 2026. Leave empty for all years.

## `sortByDate` (type: `boolean`):

When on, Scholar lists the newest papers first (Scholar "sort by date"). When off, Scholar relevance ranking is used.

## `proxyMode` (type: `string`):

auto (recommended): direct first, residential proxy only after Scholar blocks a request. direct: never use a proxy. residential: always use a residential proxy.

## Actor input object example

```json
{
  "queries": [
    "solid-state battery"
  ],
  "maxResultsPerQuery": 20,
  "yearFrom": "",
  "yearTo": "",
  "sortByDate": false,
  "proxyMode": "auto"
}
```

# Actor output Schema

## `results` (type: `string`):

All scholar results from this run.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "solid-state battery"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("webdatatools/google-scholar-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "queries": ["solid-state battery"] }

# Run the Actor and wait for it to finish
run = client.actor("webdatatools/google-scholar-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "solid-state battery"
  ]
}' |
apify call webdatatools/google-scholar-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,webdatatools/google-scholar-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/dJyQgsNpFMxCfVTJS/builds/CByO58HOCYtlByhdv/openapi.json