Google Scholar Scraper

Google Scholar Scraper extracts academic paper data from Google Scholar search results. Give it a list of keywords and it returns paper titles, authors, citation counts, publication details, abstract snippets, and PDF links as structured JSON, ready for spreadsheets, databases, or research pipelines.

What it does

Google Scholar is the largest freely accessible index of academic literature, covering journal articles, conference papers, theses, books, and preprints across every discipline. This actor searches it programmatically and returns the results as clean, structured data.

Each result includes the paper title, direct URL, authors, journal or conference name, publication year, how many times the paper has been cited, an abstract snippet, a link to the citing papers page, and a PDF link when one is publicly available.

You can filter results by publication year range, choose the result language, and control how many pages to fetch per query. For bulk collection, pass a list of queries and each one runs independently up to its own item limit.

Input

Field	Type	Default	Description
queries	string[]	—	List of search queries (keywords or phrases)
language	string	en	Language code: en, de, fr, zh, etc.
startYear	integer	—	Filter: only papers published on or after this year
endYear	integer	—	Filter: only papers published on or before this year
pages	integer	10	Pages to fetch per query (max 10, each page has up to 10 results)
maxItems	integer	10	Maximum results to return per query (max 1000)
requestTimeoutSecs	integer	30	Per-request timeout in seconds

How pagination works

Google Scholar returns 10 results per page. With pages: 10 and maxItems: 100, you get up to 100 results per query. With 5 queries, that is up to 500 total results. maxItems is a per-query cap, not a global cap.

Example: single query

{
  "queries": ["machine learning"],
  "language": "en",
  "pages": 10,
  "maxItems": 100
}

Example: filter by year range

{
  "queries": ["climate change carbon capture"],
  "startYear": 2020,
  "endYear": 2024,
  "pages": 5,
  "maxItems": 50
}

Example: batch queries

{
  "queries": [
    "deep learning image recognition",
    "reinforcement learning robotics",
    "transformer models NLP"
  ],
  "startYear": 2022,
  "pages": 10,
  "maxItems": 100
}

This fetches up to 100 results per query (300 total across 3 queries).

Example: multilingual search

{
  "queries": ["maschinelles Lernen"],
  "language": "de",
  "pages": 3,
  "maxItems": 30
}

Output fields

Each result is one row in the dataset:

Field	Type	Description
query	string	The search query that produced this result
page	integer	Page number (0-indexed)
position	integer	Position on the page (1–10)
paperTitle	string	Title of the paper
url	string	Direct link to the paper or its Google Scholar entry
authors	string	Author names as listed on Google Scholar
publicationInfo	string	Journal, conference, or publisher and year
year	integer	Publication year (null if not found)
citedBy	integer	Number of times this paper has been cited
citedByUrl	string	Link to the Google Scholar page listing papers that cite this one
paperAbstract	string	Abstract snippet shown on the search result
relatedArticlesUrl	string	Link to related articles on Google Scholar
pdfUrl	string	Direct PDF link (null if not publicly available)
scrapedAt	string	ISO 8601 UTC timestamp of when this result was collected
error	string	Error message if this result failed (null on success)

Example output record

{
  "query": "machine learning",
  "page": 0,
  "position": 1,
  "paperTitle": "Machine learning",
  "url": "https://books.google.com/books?id=ctM-EAAAQBAJ",
  "authors": "ZH Zhou",
  "publicationInfo": "2021 - books.google.com",
  "year": 2021,
  "citedBy": 3759,
  "citedByUrl": "https://scholar.google.com/scholar?cites=3387547533016043281",
  "paperAbstract": "Machine learning addresses the question of how to build computers that improve automatically through experience...",
  "relatedArticlesUrl": "https://scholar.google.com/scholar?q=related:EQ8shYj8Ai8J:scholar.google.com/",
  "pdfUrl": null,
  "scrapedAt": "2026-05-14T10:23:45.123456+00:00",
  "error": null
}

Use cases

Academic research and literature reviews

Search a topic and collect the top-cited papers. Sort by citedBy to find the most influential work. Use startYear/endYear to scope a review to a specific period.

{
  "queries": ["CRISPR gene editing cancer therapy"],
  "startYear": 2018,
  "endYear": 2024,
  "pages": 10,
  "maxItems": 100
}

Citation tracking

Monitor how many times a specific paper or author's work has been cited over time. Run the same query on a schedule and compare citedBy counts between runs.

SEO and content research

Find the most-cited academic sources for a topic before writing a long-form article or white paper. Use paperAbstract and pdfUrl to quickly assess relevance without opening each link.

Competitive intelligence

Track publication output in a specific subject area or from a specific institution. Feed the results into a database and query it over time to spot trends.

Dataset building for NLP and ML

Collect paper metadata at scale for training or evaluation datasets. The paperAbstract field gives you short English-language text samples across many topics.

Grant and funding research

Search for papers related to a funding call topic. Use authors and publicationInfo to identify active researchers and institutions in the space.

Tips for better results

• Use specific multi-word queries for more precise results: "deep learning drug discovery" returns more relevant papers than just "deep learning".
• Set startYear to filter out older work. Google Scholar's index goes back decades, and many queries surface results from the 1990s without a year filter.
• Set maxItems to 100 and pages to 10 to collect the full first 100 results for each query. Google Scholar caps public access at around 100 results per search.
• Use language: "en" to restrict results to English-language papers even when searching non-English topics.
• For multi-author or multi-topic tracking, use the queries list with one query per author name, paper topic, or keyword cluster.

Notes

• Google Scholar returns up to 10 results per page and limits public access to roughly the first 10 pages (100 results) per query.
• Year filtering uses Google Scholar's built-in as_ylo and as_yhi parameters and applies server-side.
• Some results are book entries, patents, or citation-only records with no direct URL. The url field will be null for those.
• PDF links appear when Google Scholar indexes a freely available version. Not all papers have one.
• If a page returns no results, pagination stops early for that query and moves to the next one.
• Results with scraping errors are pushed with an error field set so the rest of the run continues uninterrupted.