Papers with Code Scraper

Scrape Papers with Code — the free, open resource for machine learning papers with code implementations. Search papers, browse ML tasks and leaderboards, find datasets, and look up ML methods. Uses the official Papers with Code public REST API — no authentication required.

What you can scrape

• Papers: title, abstract, authors, arXiv ID, published date, PDF/abstract URLs, top GitHub repo stars, linked tasks and datasets
• Tasks: ML benchmark tasks with category hierarchy, paper/dataset/method counts
• Leaderboard results: ranked model performance on benchmark datasets
• Datasets: ML datasets with modality info and paper count
• Methods: ML methods/architectures with paper links and collections

Supported modes

Mode	Description
searchPapers	Search ML papers by keyword (e.g. "large language models", "BERT")
byPaperId	Fetch a single paper with full details, repositories, and results
searchTasks	Search ML tasks/benchmarks (e.g. "image classification")
leaderboard	Ranked results for a specific task (e.g. "image-classification")
searchDatasets	Search ML datasets (e.g. "ImageNet", "CIFAR")
searchMethods	Search ML methods/architectures (e.g. "transformer", "attention")

Input

Field	Type	Description
mode	select	Scrape mode (default: searchPapers)
query	string	Search keyword (modes: searchPapers, searchTasks, searchDatasets, searchMethods)
paperId	string	Paper slug ID (mode: byPaperId, e.g. attention-is-all-you-need)
taskId	string	Task slug ID (mode: leaderboard, e.g. image-classification)
ordering	select	Sort order for paper search (newest published, most stars, alphabetical, etc.)
minStars	integer	Only include papers with at least N GitHub stars on top repo
hasCode	boolean	Only include papers that have a linked GitHub repository
maxItems	integer	Maximum records to emit (default: 50)

Example input

{
  "mode": "searchPapers",
  "query": "large language models",
  "ordering": "-github_stars",
  "minStars": 1000,
  "maxItems": 20
}

Output

Each record includes recordType, siteName = "Papers with Code", and scrapedAt timestamp.

Paper record

{
  "paperId": "attention-is-all-you-need",
  "arxivId": "1706.03762",
  "title": "Attention Is All You Need",
  "abstract": "We propose a new simple network architecture...",
  "authors": ["Ashish Vaswani", "Noam Shazeer", "Niki Parmar"],
  "primaryAuthor": "Ashish Vaswani",
  "publishedDate": "2017-06-12",
  "urlAbstract": "https://arxiv.org/abs/1706.03762",
  "urlPdf": "https://arxiv.org/pdf/1706.03762.pdf",
  "paperswithcodeUrl": "https://paperswithcode.com/paper/attention-is-all-you-need",
  "stars": 14500,
  "repoUrl": "https://github.com/tensorflow/tensor2tensor",
  "taskNames": ["Machine Translation", "Language Modelling"],
  "datasetNames": ["WMT 2014 English-German"],
  "numResults": 12,
  "hasCode": true,
  "recordType": "paper",
  "siteName": "Papers with Code",
  "scrapedAt": "2026-05-10T12:00:00+00:00"
}

Leaderboard result record

{
  "taskName": "Image Classification",
  "datasetName": "ImageNet",
  "metricName": "Top-1 Accuracy",
  "rank": 1,
  "paperTitle": "Deep Residual Learning for Image Recognition",
  "modelName": "ResNet-152",
  "score": 78.57,
  "repoUrl": "https://github.com/KaimingHe/deep-residual-networks",
  "paperUrl": "https://paperswithcode.com/paper/resnet",
  "recordType": "leaderboardResult",
  "siteName": "Papers with Code",
  "scrapedAt": "2026-05-10T12:00:00+00:00"
}

Task record

{
  "taskId": "image-classification",
  "name": "Image Classification",
  "description": "Assigning an input image one or more labels from a fixed set.",
  "categories": ["Computer Vision"],
  "numPapers": 3500,
  "numDatasets": 280,
  "numMethods": 1200,
  "taskUrl": "https://paperswithcode.com/task/image-classification",
  "recordType": "task",
  "siteName": "Papers with Code",
  "scrapedAt": "2026-05-10T12:00:00+00:00"
}

FAQs

Does this require an API key or login? No. The Papers with Code public API is completely free and requires no authentication.

How do I find a paper's ID for byPaperId mode? The paper ID is the slug in the URL on paperswithcode.com. For example, https://paperswithcode.com/paper/attention-is-all-you-need → ID is attention-is-all-you-need.

How do I find a task ID for the leaderboard mode? Look at the URL on the Papers with Code task page. For example, https://paperswithcode.com/task/image-classification → task ID is image-classification.

What does ordering do for paper search? Controls sort order: newest published first, most GitHub stars first, alphabetical by title, or their reverses.

Can I filter to only papers with code? Yes — set hasCode: true. This filters to papers that have at least one linked GitHub repository.

What is the stars field? The number of GitHub stars on the paper's top-ranked repository. Only present when the paper has linked code.