# Semantic Scholar Scraper — Papers, Authors & Citations (`muhammadafzal/semantic-scholar-scraper`) Actor

Search Semantic Scholar papers and authors, retrieve abstracts, TLDRs, citation counts, references, citing papers, external IDs, and open-access PDF metadata for literature reviews and AI-agent research workflows. Uses the public Academic Graph API. Charged $0.001 per returned record.

- **URL**: https://apify.com/muhammadafzal/semantic-scholar-scraper.md
- **Developed by:** [Muhammad Afzal](https://apify.com/muhammadafzal) (community)
- **Categories:** AI, Developer tools, MCP servers
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 returned records

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## Semantic Scholar Scraper — Papers, Authors & Citations

Search Semantic Scholar and retrieve structured scholarly metadata for literature reviews, research discovery, citation mapping, bibliometrics, and AI-agent research workflows. The actor uses the public Semantic Scholar Academic Graph API rather than browser HTML, so results are compact, typed, and easy to page through.

### What it returns

Each dataset row has one stable shape and a `recordType` of `paper`, `author`, `citation`, or `reference`.

| Field | Meaning |
|---|---|
| `paperId`, `authorId` | Semantic Scholar identifiers |
| `title`, `abstract`, `tldr` | Paper title/abstract/short summary, or author name |
| `authors` | Compact author ID/name objects |
| `citationCount`, `referenceCount`, `influentialCitationCount` | Graph metrics when supplied |
| `externalIds`, `url`, `openAccessPdf` | DOI, arXiv, canonical URL, and OA PDF metadata |
| `venue`, `year`, `publicationDate` | Publication metadata |
| `hIndex`, `paperCount` | Author metrics when supplied |
| `relation` | `cites` or `references` for graph traversal rows |

### When to use

Use `searchPapers` to discover papers by topic, `getPaper` for one known work, `searchAuthors` or `getAuthor` for researcher metadata, and `getCitations`/`getReferences` to traverse a paper’s graph. This actor does not download PDFs, extract full text, or bypass private publisher access; use a full-text or document actor for those jobs.

### Input examples

Paper search:

```json
{ "operation": "searchPapers", "query": "retrieval augmented generation", "year": "2020-2024", "limit": 25, "openAccessPdfOnly": true }
````

Paper details:

```json
{ "operation": "getPaper", "paperId": "10.1038/s41586-020-2649-2", "fields": ["paperId", "title", "authors", "abstract", "citationCount", "openAccessPdf"] }
```

Citation graph:

```json
{ "operation": "getCitations", "paperId": "649def34f8be52c8b66281af98ae884c09aef38b", "limit": 50, "offset": 0 }
```

For AI agents, the operation description is the routing boundary: use paper search for discovery, one-paper lookup for known identifiers, and graph operations for citation relationships. Keep `limit` tight and paginate with `offset` to control context and cost.

### Pricing

| Event | Price |
|---|---:|
| Returned record | $0.001 per dataset row |

The worst-case charge is `limit × $0.001` for list operations. A one-paper lookup costs one record event. Semantic Scholar API traffic is not a browser scrape; no residential proxy is required.

### Reliability and limits

The actor retries transient API errors and HTTP 429 responses with exponential backoff. A valid query with no matches is a successful run with zero rows. Invalid input, missing identifiers, API rate limiting, and upstream errors are reported in the `OUTPUT` key-value record with actionable warnings instead of being presented as a misleading empty search.

Semantic Scholar documents introductory API-key rate limits of approximately one request per second. Set the `SEMANTIC_SCHOLAR_API_KEY` environment variable in Apify for higher limits when available. The API can omit abstracts, TLDRs, PDFs, dates, and external IDs for individual records; those fields are returned as `null` or empty arrays consistently.

### Legal and responsible use

Use the Semantic Scholar API according to its current terms, attribution requirements, rate limits, and applicable law. Respect publisher copyright and do not represent API metadata as a license to redistribute full papers.

# Actor input Schema

## `operation` (type: `string`):

Use this to choose the Academic Graph API action. Accepted values are searchPapers, getPaper, searchAuthors, getAuthor, getCitations, and getReferences. Defaults to searchPapers.

## `query` (type: `string`):

Use this for paper or author search terms such as "retrieval augmented generation" or "Geoffrey Hinton". Defaults to "machine learning". This is not a paper ID; use paperId for one-paper and graph operations.

## `paperId` (type: `string`):

Use this for getPaper, getCitations, or getReferences with a Semantic Scholar paperId, DOI, arXiv ID, or CorpusID. Defaults to a known paper ID for a working prefill. This is not an author ID.

## `authorId` (type: `string`):

Use this for getAuthor with a Semantic Scholar author ID. Defaults to an empty value. This is not a paper ID or a free-text author name; use searchAuthors for names.

## `fields` (type: `array`):

Use this to select paper fields returned by the API. Defaults to core metadata plus abstract, TLDR, citations, and open-access PDF. Use fewer fields for faster, smaller agent responses.

## `limit` (type: `integer`):

Use this to cap returned dataset rows and cost. Enter 1–100 for search, citation, or reference operations. Defaults to 10 and is not a guarantee that matching records exist.

## `offset` (type: `integer`):

Use this to page through search, citation, or reference results. Enter a non-negative offset; for example, 100 starts after the first 100 records. Defaults to 0.

## `year` (type: `string`):

Use this to filter paper search by year or range, such as "2023" or "2020-2024". Defaults to an empty value, meaning no year filter. This does not filter author search.

## `minCitationCount` (type: `integer`):

Use this to keep only papers with at least this many citations. Defaults to 0. This applies to paper search only and can reduce broad result sets.

## `openAccessPdfOnly` (type: `boolean`):

Use this to keep only paper search results with an open-access PDF. Defaults to false. This applies to paper search only and is not a full-text download option.

## Actor input object example

```json
{
  "operation": "searchPapers",
  "query": "retrieval augmented generation",
  "paperId": "10.1038/s41586-020-2649-2",
  "authorId": "1741101",
  "fields": [
    "paperId",
    "title",
    "authors",
    "citationCount"
  ],
  "limit": 25,
  "offset": 100,
  "year": "2020-2024",
  "minCitationCount": 100,
  "openAccessPdfOnly": true
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset rows for papers, authors, citations, or references.

## `summary` (type: `string`):

Operation, counts, pagination, warnings, and API source.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "operation": "searchPapers",
    "query": "machine learning",
    "paperId": "649def34f8be52c8b66281af98ae884c09aef38b",
    "limit": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("muhammadafzal/semantic-scholar-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "operation": "searchPapers",
    "query": "machine learning",
    "paperId": "649def34f8be52c8b66281af98ae884c09aef38b",
    "limit": 10,
}

# Run the Actor and wait for it to finish
run = client.actor("muhammadafzal/semantic-scholar-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "operation": "searchPapers",
  "query": "machine learning",
  "paperId": "649def34f8be52c8b66281af98ae884c09aef38b",
  "limit": 10
}' |
apify call muhammadafzal/semantic-scholar-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=muhammadafzal/semantic-scholar-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "Semantic Scholar Scraper — Papers, Authors & Citations",
        "description": "Search Semantic Scholar papers and authors, retrieve abstracts, TLDRs, citation counts, references, citing papers, external IDs, and open-access PDF metadata for literature reviews and AI-agent research workflows. Uses the public Academic Graph API. Charged $0.001 per returned record.",
        "version": "1.0",
        "x-build-id": "J9hVat6oaqb3JPekz"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/muhammadafzal~semantic-scholar-scraper/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-muhammadafzal-semantic-scholar-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/muhammadafzal~semantic-scholar-scraper/runs": {
            "post": {
                "operationId": "runs-sync-muhammadafzal-semantic-scholar-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/muhammadafzal~semantic-scholar-scraper/run-sync": {
            "post": {
                "operationId": "run-sync-muhammadafzal-semantic-scholar-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "properties": {
                    "operation": {
                        "title": "Operation",
                        "enum": [
                            "searchPapers",
                            "getPaper",
                            "searchAuthors",
                            "getAuthor",
                            "getCitations",
                            "getReferences"
                        ],
                        "type": "string",
                        "description": "Use this to choose the Academic Graph API action. Accepted values are searchPapers, getPaper, searchAuthors, getAuthor, getCitations, and getReferences. Defaults to searchPapers.",
                        "default": "searchPapers"
                    },
                    "query": {
                        "title": "Search query",
                        "type": "string",
                        "description": "Use this for paper or author search terms such as \"retrieval augmented generation\" or \"Geoffrey Hinton\". Defaults to \"machine learning\". This is not a paper ID; use paperId for one-paper and graph operations.",
                        "default": "machine learning"
                    },
                    "paperId": {
                        "title": "Paper identifier",
                        "type": "string",
                        "description": "Use this for getPaper, getCitations, or getReferences with a Semantic Scholar paperId, DOI, arXiv ID, or CorpusID. Defaults to a known paper ID for a working prefill. This is not an author ID.",
                        "default": "649def34f8be52c8b66281af98ae884c09aef38b"
                    },
                    "authorId": {
                        "title": "Author identifier",
                        "type": "string",
                        "description": "Use this for getAuthor with a Semantic Scholar author ID. Defaults to an empty value. This is not a paper ID or a free-text author name; use searchAuthors for names.",
                        "default": ""
                    },
                    "fields": {
                        "title": "Paper fields",
                        "type": "array",
                        "description": "Use this to select paper fields returned by the API. Defaults to core metadata plus abstract, TLDR, citations, and open-access PDF. Use fewer fields for faster, smaller agent responses.",
                        "items": {
                            "type": "string"
                        },
                        "default": [
                            "paperId",
                            "externalIds",
                            "url",
                            "title",
                            "abstract",
                            "venue",
                            "year",
                            "publicationDate",
                            "authors",
                            "citationCount",
                            "referenceCount",
                            "openAccessPdf",
                            "fieldsOfStudy",
                            "publicationTypes",
                            "tldr"
                        ]
                    },
                    "limit": {
                        "title": "Maximum records",
                        "minimum": 1,
                        "maximum": 100,
                        "type": "integer",
                        "description": "Use this to cap returned dataset rows and cost. Enter 1–100 for search, citation, or reference operations. Defaults to 10 and is not a guarantee that matching records exist.",
                        "default": 10
                    },
                    "offset": {
                        "title": "Result offset",
                        "minimum": 0,
                        "maximum": 10000,
                        "type": "integer",
                        "description": "Use this to page through search, citation, or reference results. Enter a non-negative offset; for example, 100 starts after the first 100 records. Defaults to 0.",
                        "default": 0
                    },
                    "year": {
                        "title": "Publication year filter",
                        "type": "string",
                        "description": "Use this to filter paper search by year or range, such as \"2023\" or \"2020-2024\". Defaults to an empty value, meaning no year filter. This does not filter author search.",
                        "default": ""
                    },
                    "minCitationCount": {
                        "title": "Minimum citation count",
                        "minimum": 0,
                        "maximum": 1000000,
                        "type": "integer",
                        "description": "Use this to keep only papers with at least this many citations. Defaults to 0. This applies to paper search only and can reduce broad result sets.",
                        "default": 0
                    },
                    "openAccessPdfOnly": {
                        "title": "Open-access PDFs only",
                        "type": "boolean",
                        "description": "Use this to keep only paper search results with an open-access PDF. Defaults to false. This applies to paper search only and is not a full-text download option.",
                        "default": false
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```