# Semantic Scholar Scraper — Papers, Authors & Citations (`du7chmaniac/semantic-scholar-scraper`) Actor

Search academic papers, authors, and citation data from the Semantic Scholar API. No API key required.

- **URL**: https://apify.com/du7chmaniac/semantic-scholar-scraper.md
- **Developed by:** [Joren Maurissen](https://apify.com/du7chmaniac) (community)
- **Categories:** Developer tools, Automation, Other
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## Semantic Scholar Scraper — Papers, Authors & Citations

Scrape academic papers, authors, and citation data from the **Semantic Scholar Academic Graph API** — a free, open-access API covering 200M+ papers across all scientific domains. No API key required.

### Features

- 🔍 **Paper Search** — Search 200M+ papers by keyword query with pagination
- 📄 **Paper Details** — Get full paper metadata including references and citing papers
- 👤 **Author Search** — Find authors by name across the entire academic graph
- 📚 **Author Papers** — Retrieve an author's complete publication list with citation counts
- 📊 **Flexible Fields** — Choose which fields to return (title, abstract, authors, year, venue, openAccessPdf, and more)
- 🚀 **Rate-Limit Aware** — Automatic retry with backoff on HTTP 429, pagination up to 1000 results
- 💡 **No API Key Needed** — Works with the free tier (~100 requests per 5 minutes)

### Use Cases

- **Literature reviews** — Gather all papers on a topic with abstracts and citation counts
- **Citation analysis** — Find the most-cited papers in a field or by a specific author
- **Author profiling** — Get an author's full publication list with metrics
- **Open access discovery** — Find open-access PDF URLs for papers
- **Research monitoring** — Track new papers by query or author on a schedule
- **Academic data pipelines** — Feed structured paper data into downstream analytics

### Input

| Field      | Type   | Required | Default                                                             | Description                                              |
| ---------- | ------ | -------- | ------------------------------------------------------------------- | -------------------------------------------------------- |
| `mode`     | select | ✅       | `search-papers`                                                     | One of: `search-papers`, `paper-details`, `author-search`, `author-papers` |
| `query`    | string | ❌       | `"attention is all you need"`                                        | Search query (for `search-papers` and `author-search`)   |
| `paperId`  | string | ❌       | —                                                                   | Semantic Scholar paper ID (for `paper-details`)          |
| `authorId` | string | ❌       | —                                                                   | Semantic Scholar author ID (for `author-papers`)         |
| `fields`   | string | ❌       | `title,abstract,authors,year,citationCount,venue,openAccessPdf,url` | Comma-separated fields to return                         |
| `limit`    | int    | ❌       | `50`                                                                | Max results (1–1000, paginated in batches of 100)        |
| `offset`   | int    | ❌       | `0`                                                                 | Number of results to skip                               |

### Output

Each dataset item is a JSON object. Fields depend on the `fields` input parameter and the mode.

#### Example: Paper Search

**Input:**
```json
{
    "mode": "search-papers",
    "query": "attention is all you need",
    "limit": 3
}
````

**Output (dataset items):**

```json
[
    {
        "paperId": "204e3073870fae3d05bcbc2f6a8e263d9b72e776",
        "title": "Attention Is All You Need",
        "abstract": "The dominant sequence transduction models are based on complex recurrent or convolutional neural networks...",
        "authors": [
            { "authorId": "1741101", "name": "Ashish Vaswani" },
            { "authorId": "1726699", "name": "Noam Shazeer" }
        ],
        "year": 2017,
        "citationCount": 138523,
        "venue": "Advances in Neural Information Processing Systems",
        "openAccessPdf": { "url": "https://..." },
        "url": "https://www.semanticscholar.org/paper/204e3073870fae3d05bcbc2f6a8e263d9b72e776",
        "_mode": "search-papers"
    }
]
```

#### Example: Paper Details

**Input:**

```json
{
    "mode": "paper-details",
    "paperId": "204e3073870fae3d05bcbc2f6a8e263d9b72e776",
    "fields": "title,abstract,authors,year,citationCount,references.title,citations.title"
}
```

#### Example: Author Search

**Input:**

```json
{
    "mode": "author-search",
    "query": "Yann LeCun",
    "limit": 5
}
```

#### Example: Author Papers

**Input:**

```json
{
    "mode": "author-papers",
    "authorId": "1726699",
    "limit": 10
}
```

### How It Works

1. Reads input from the Apify input schema (mode, query, paperId, authorId, fields, limit, offset)
2. Constructs requests to the Semantic Scholar Academic Graph API (`https://api.semanticscholar.org/graph/v1/`)
3. Paginates results in batches of 100 (API max per page) up to the `limit`
4. Retries on HTTP 429 (rate limit) with exponential backoff
5. Pushes all results to the Apify dataset with a `_mode` tag

### API Reference

This actor uses the [Semantic Scholar Academic Graph API](https://api.semanticscholar.org/api-docs/graph):

- `GET /paper/search?query=<q>&limit=<n>&fields=<f>` — Paper search
- `GET /paper/{paperId}?fields=<f>` — Paper details
- `GET /author/search?query=<q>&limit=<n>` — Author search
- `GET /author/{authorId}/papers?fields=<f>&limit=<n>` — Author's papers

**Rate limit**: ~100 requests per 5 minutes without an API key (sufficient for most use cases).

### Pricing

**$0.50 per 1,000 results** — pay only for what you get.

At the default `limit=50`, a single run costs ~$0.025. The marginal cost per request is negligible (the API is free); the fee covers infrastructure and maintenance.

### Resources

- [Semantic Scholar API Documentation](https://api.semanticscholar.org/api-docs/graph)
- [Semantic Scholar Website](https://www.semanticscholar.org/)
- [Apify Platform Documentation](https://docs.apify.com/platform)
- [Apify Python SDK](https://docs.apify.com/sdk/python)
- [Integration with Make, Zapier, Google Drive, and more](https://apify.com/integrations)

# Actor input Schema

## `mode` (type: `string`):

What to retrieve from Semantic Scholar Academic Graph API.

## `query` (type: `string`):

Search query for paper search or author search modes.

## `paperId` (type: `string`):

Semantic Scholar paper ID (e.g. 204e3073870fae3d05bcbc2f6a8e263d9b72e776) for paper-details mode.

## `authorId` (type: `string`):

Semantic Scholar author ID (e.g. 1741101) for author-papers mode.

## `fields` (type: `string`):

Comma-separated list of fields to return for each paper (e.g. title,abstract,authors,year,citationCount,venue,openAccessPdf,url).

## `limit` (type: `integer`):

Maximum number of results to return (per mode). API max 100 per page; we paginate.

## `offset` (type: `integer`):

Number of results to skip (for pagination).

## `proxyConfiguration` (type: `object`):

Recommended — Semantic Scholar API rate limits aggressively

## Actor input object example

```json
{
  "mode": "search-papers",
  "query": "attention is all you need",
  "fields": "title,abstract,authors,year,citationCount,venue,openAccessPdf,url",
  "limit": 50,
  "offset": 0,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "attention is all you need"
};

// Run the Actor and wait for it to finish
const run = await client.actor("du7chmaniac/semantic-scholar-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "query": "attention is all you need" }

# Run the Actor and wait for it to finish
run = client.actor("du7chmaniac/semantic-scholar-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "attention is all you need"
}' |
apify call du7chmaniac/semantic-scholar-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=du7chmaniac/semantic-scholar-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "Semantic Scholar Scraper — Papers, Authors & Citations",
        "description": "Search academic papers, authors, and citation data from the Semantic Scholar API. No API key required.",
        "version": "0.1",
        "x-build-id": "LJa725OPTrFTDAbkl"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/du7chmaniac~semantic-scholar-scraper/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-du7chmaniac-semantic-scholar-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/du7chmaniac~semantic-scholar-scraper/runs": {
            "post": {
                "operationId": "runs-sync-du7chmaniac-semantic-scholar-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/du7chmaniac~semantic-scholar-scraper/run-sync": {
            "post": {
                "operationId": "run-sync-du7chmaniac-semantic-scholar-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "required": [
                    "mode"
                ],
                "properties": {
                    "mode": {
                        "title": "Mode",
                        "enum": [
                            "search-papers",
                            "paper-details",
                            "author-search",
                            "author-papers"
                        ],
                        "type": "string",
                        "description": "What to retrieve from Semantic Scholar Academic Graph API.",
                        "default": "search-papers"
                    },
                    "query": {
                        "title": "Search query",
                        "type": "string",
                        "description": "Search query for paper search or author search modes."
                    },
                    "paperId": {
                        "title": "Paper ID",
                        "type": "string",
                        "description": "Semantic Scholar paper ID (e.g. 204e3073870fae3d05bcbc2f6a8e263d9b72e776) for paper-details mode."
                    },
                    "authorId": {
                        "title": "Author ID",
                        "type": "string",
                        "description": "Semantic Scholar author ID (e.g. 1741101) for author-papers mode."
                    },
                    "fields": {
                        "title": "Fields to return",
                        "type": "string",
                        "description": "Comma-separated list of fields to return for each paper (e.g. title,abstract,authors,year,citationCount,venue,openAccessPdf,url).",
                        "default": "title,abstract,authors,year,citationCount,venue,openAccessPdf,url"
                    },
                    "limit": {
                        "title": "Max results",
                        "minimum": 1,
                        "maximum": 1000,
                        "type": "integer",
                        "description": "Maximum number of results to return (per mode). API max 100 per page; we paginate.",
                        "default": 50
                    },
                    "offset": {
                        "title": "Offset",
                        "minimum": 0,
                        "type": "integer",
                        "description": "Number of results to skip (for pagination).",
                        "default": 0
                    },
                    "proxyConfiguration": {
                        "title": "Proxy configuration",
                        "type": "object",
                        "description": "Recommended — Semantic Scholar API rate limits aggressively",
                        "default": {
                            "useApifyProxy": true,
                            "apifyProxyGroups": [
                                "RESIDENTIAL"
                            ]
                        }
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```