# Semantic Scholar Scraper - Papers, Citations & Authors (`jungle_synthesizer/semantic-scholar-scraper`) Actor

Scrape Semantic Scholar for academic papers with abstracts, citation counts, authors, venues, open-access PDFs and DOIs. Search by keyword, filter by year, field and citations, look up papers by DOI or arXiv ID, or pull citations, references and author profiles.

- **URL**: https://apify.com/jungle_synthesizer/semantic-scholar-scraper.md
- **Developed by:** [BowTiedRaccoon](https://apify.com/jungle_synthesizer) (community)
- **Categories:** Education, Developer tools, Other
- **Stats:** 2 total users, 1 monthly users, 85.7% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.80 / 1,000 record scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Semantic Scholar Scraper - Papers, Citations, References & Authors

Pull structured academic data from [Semantic Scholar](https://www.semanticscholar.org): paper titles, abstracts, citation counts, authors, venues, DOIs, open-access PDF links, plus the citation graph and author profiles. Search by keyword, filter by year, field of study, publication type, venue and minimum citations, or look up exact papers by DOI, arXiv ID or Semantic Scholar link.

### Features

- Keyword search across the Semantic Scholar corpus, sorted by citations or by publication date
- Filters for publication year range, minimum citations, fields of study, publication types, venue and open-access PDF availability
- Look up papers by DOI, arXiv ID, PubMed ID, CorpusId or a semanticscholar.org paper link, any mix in one run
- Citations of a paper (who cites it) and references of a paper (what it cites), each row linked back to the source paper
- Author profile search with h-index, paper count, citation count, affiliations and homepage
- Up to a thousand papers per page, so long result lists come back quickly
- Flat, typed records ready for CSV, Excel or a database

### Who uses this

- **Literature reviewers** who need the most-cited papers on a topic with abstracts in one export
- **Research-intelligence and AI teams** building citation datasets, retrieval corpora and recommenders
- **Bibliometric analysts** tracking a field, venue or author over time
- **Lab and library teams** assembling reading lists and open-access collections

### How it works

1. Choose what to collect: a keyword search, specific papers, a paper's citations, a paper's references, or authors.
2. Add your query or paper IDs and any filters.
3. Set `maxItems` and run. Rows stream into the dataset as they are collected.

### Input

#### Most-cited papers on a topic

```json
{
    "mode": "search",
    "query": "\"graph neural network\"",
    "sortBy": "citationCount:desc",
    "maxItems": 100
}
```

#### Recent open-access medical papers

```json
{
    "mode": "search",
    "query": "immunotherapy",
    "fieldsOfStudy": ["Medicine"],
    "yearFrom": 2023,
    "openAccessOnly": true,
    "sortBy": "publicationDate:desc",
    "maxItems": 200
}
```

#### Specific papers

```json
{
    "mode": "papers",
    "paperIds": ["ARXIV:1706.03762", "DOI:10.1109/5.726791"],
    "maxItems": 50
}
```

#### Who cites a paper

```json
{
    "mode": "citations",
    "paperIds": ["ARXIV:1706.03762"],
    "maxItems": 500
}
```

#### Input parameters

| Parameter             | Type    | Description |
|-----------------------|---------|------------------------------------------------------------------------------------------------------------------------|
| `mode`                | string  | `search`, `papers`, `citations`, `references` or `authors`. Default `search`. |
| `query`               | string  | Keywords for paper search or a name for author search. Supports quoted phrases, `+` (and), `\|` (or), `-` (exclude), `*` (prefix). |
| `paperIds`            | array   | For `papers`, `citations` and `references`: Semantic Scholar IDs, `DOI:...`, `ARXIV:...`, `PMID:...`, `CorpusId:...` or paper links. |
| `sortBy`              | string  | Search order: most cited, newest, oldest or paper ID. |
| `yearFrom` / `yearTo` | integer | Publication year bounds for keyword search. |
| `minCitations`        | integer | Only papers with at least this many citations. |
| `fieldsOfStudy`       | array   | One or more fields, for example Computer Science or Medicine. Empty means all. |
| `publicationTypes`    | array   | Review, journal article, conference paper, dataset and more. Empty means all. |
| `openAccessOnly`      | boolean | Only papers with a free PDF link. |
| `venue`               | string  | Journal or conference name. |
| `maxItems`            | integer | Maximum rows to return. |

Keyword search matches words in titles and abstracts rather than ranking by relevance, so wrap multi-word topics in quotes (`"graph neural network"`) for a phrase match. Citation and reference lists for a single paper are available up to 10,000 entries.

### Output

Paper row (abstract omitted here for length):

```json
{
    "record_type": "paper",
    "paper_id": "162d958ff885f1462aeda91cd72582323fd6a1f4",
    "corpus_id": 14542261,
    "title": "Gradient-based learning applied to document recognition",
    "year": 1998,
    "venue": "Proceedings of the IEEE",
    "journal_volume": "86",
    "journal_pages": "2278-2324",
    "publication_types": ["JournalArticle", "Review"],
    "fields_of_study": ["Computer Science"],
    "citation_count": 63440,
    "influential_citation_count": 6512,
    "reference_count": 147,
    "is_open_access": false,
    "doi": "10.1109/5.726791",
    "authors": ["Yann LeCun", "L. Bottou", "Yoshua Bengio", "P. Haffner"],
    "author_ids": ["1688882", "52184096", "1751762", "1721248"],
    "url": "https://www.semanticscholar.org/paper/162d958ff885f1462aeda91cd72582323fd6a1f4"
}
```

| Field                                                                                                     | Description |
|-----------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------|
| `paper_id`, `corpus_id`, `url`                                                                            | Semantic Scholar identifiers and page |
| `title`, `abstract`, `year`, `publication_date`                                                           | Core bibliographic data |
| `venue`, `journal_volume`, `journal_pages`                                                                | Where it was published |
| `publication_types`, `fields_of_study`                                                                    | Classification |
| `citation_count`, `influential_citation_count`, `reference_count`                                         | Impact counts |
| `is_open_access`, `open_access_pdf_url`                                                                   | Free PDF availability |
| `doi`, `arxiv_id`, `pubmed_id`                                                                            | External identifiers |
| `authors`, `author_ids`                                                                                   | Author names and IDs, in matching order |
| `relation`, `source_paper_id`, `is_influential`, `citation_intents`                                       | On citation and reference rows: link back to the paper you asked about |
| `author_id`, `author_name`, `affiliations`, `homepage`, `paper_count`, `author_citation_count`, `h_index` | Author rows |

Fields that Semantic Scholar does not hold for a paper (many records have no abstract or DOI) come back empty.

### FAQ

**How many results can one run return?** Keyword search pages through very large result sets; set `maxItems` to what you need. Citations and references of a single paper stop at 10,000 entries.

**Why does a search return papers that only loosely match?** Search matches words anywhere in the title and abstract. Use quotes for exact phrases and `-` to exclude terms.

**Can I get the full text of papers?** No. You get metadata, abstracts and links to free PDFs where Semantic Scholar knows of one.

**Is every paper included?** Rows reflect what Semantic Scholar indexes at run time. Recent papers can appear with zero citations or without an abstract.

**What format can I export to?** JSON, CSV, Excel, XML or HTML from the dataset tab, or through the Apify integrations.

# Actor input Schema

## `sp_intended_usage` (type: `string`):

What will this data feed? E.g. lead lists, KYB checks, price tracking.

## `sp_improvement_suggestions` (type: `string`):

Provide any feedback or suggestions for improvements.

## `sp_contact` (type: `string`):

We'll personally help with your use case. No spam.

## `mode` (type: `string`):

Papers matching a search, specific papers by ID, the papers that cite a paper, the papers a paper references, or author profiles matching a name.

## `query` (type: `string`):

Keywords for paper search, or an author name for author search. Paper search supports quoted phrases ("graph neural network"), + (and), | (or), - (exclude) and \* (prefix).

## `paperIds` (type: `array`):

For lookup, citations and references. Accepts Semantic Scholar IDs, DOI:10.xxxx/..., ARXIV:1706.03762, PMID:..., CorpusId:..., or a semanticscholar.org paper URL.

## `sortBy` (type: `string`):

Ordering for keyword search.

## `yearFrom` (type: `integer`):

Earliest publication year for keyword search. Leave empty for no lower bound.

## `yearTo` (type: `integer`):

Latest publication year for keyword search. Leave empty for no upper bound.

## `minCitations` (type: `integer`):

Only papers with at least this many citations (keyword search).

## `fieldsOfStudy` (type: `array`):

Restrict keyword search to one or more fields. Leave empty for all fields.

## `publicationTypes` (type: `array`):

Restrict keyword search to specific publication types. Leave empty for all types.

## `openAccessOnly` (type: `boolean`):

Only papers that have a free PDF available (keyword search).

## `venue` (type: `string`):

Restrict keyword search to a journal or conference name (for example "Nature" or "NeurIPS").

## `maxItems` (type: `integer`):

Maximum number of records to return.

## Actor input object example

```json
{
  "sp_contact": "Share your email here...",
  "mode": "search",
  "query": "\"graph neural network\"",
  "paperIds": [
    "ARXIV:1706.03762"
  ],
  "sortBy": "citationCount:desc",
  "fieldsOfStudy": [],
  "publicationTypes": [],
  "openAccessOnly": false,
  "maxItems": 10
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "sp_contact": "Share your email here...",
    "mode": "search",
    "query": "\"graph neural network\"",
    "paperIds": [
        "ARXIV:1706.03762"
    ],
    "sortBy": "citationCount:desc",
    "fieldsOfStudy": [],
    "publicationTypes": [],
    "openAccessOnly": false,
    "maxItems": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("jungle_synthesizer/semantic-scholar-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "sp_contact": "Share your email here...",
    "mode": "search",
    "query": "\"graph neural network\"",
    "paperIds": ["ARXIV:1706.03762"],
    "sortBy": "citationCount:desc",
    "fieldsOfStudy": [],
    "publicationTypes": [],
    "openAccessOnly": False,
    "maxItems": 10,
}

# Run the Actor and wait for it to finish
run = client.actor("jungle_synthesizer/semantic-scholar-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "sp_contact": "Share your email here...",
  "mode": "search",
  "query": "\\"graph neural network\\"",
  "paperIds": [
    "ARXIV:1706.03762"
  ],
  "sortBy": "citationCount:desc",
  "fieldsOfStudy": [],
  "publicationTypes": [],
  "openAccessOnly": false,
  "maxItems": 10
}' |
apify call jungle_synthesizer/semantic-scholar-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,jungle_synthesizer/semantic-scholar-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/6zlsXZygeXvgcFvhX/builds/Cz0a1N70cLhMLRmAm/openapi.json