# Hugging Face Papers Scraper (`automation-lab/huggingface-papers-scraper`) Actor Extract ML research papers from Hugging Face. Get titles, authors, abstracts, AI summaries, GitHub repos, star counts, upvotes, and keywords. Search by topic or get daily trending papers. Pure API — fast and cheap. - **URL**: https://apify.com/automation-lab/huggingface-papers-scraper.md - **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community) - **Categories:** Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Hugging Face Papers Scraper ### What does Hugging Face Papers Scraper do? **Hugging Face Papers Scraper** extracts machine learning research papers from [Hugging Face](https://huggingface.co/papers). Get **titles, authors, abstracts, AI-generated summaries, GitHub repositories, star counts, upvotes, and keywords** — all via a clean API, no browser needed. Two scraping modes: **search by keyword** to find papers on any ML topic, or **daily trending** to get the hottest papers the community is upvoting right now. Pure HTTP — fast, cheap, and reliable. This is the only Apify actor for Hugging Face Papers. Papers With Code (paperswithcode.com) now redirects to Hugging Face, making this the single source for ML research paper data. ### Who is Hugging Face Papers Scraper for? - 🔬 **ML researchers and PhD students** — Track state-of-the-art progress in your subfield. Monitor new papers on transformers, reinforcement learning, or computer vision. Get GitHub repos automatically so you can reproduce results. - 📊 **AI startup founders and VCs** — Monitor research trends to spot emerging techniques. Track which labs are publishing on what topics. Use upvote counts as a proxy for community interest. - 🏗️ **MLOps and AI engineers** — Find papers with open-source implementations (GitHub links + star counts). Build automated pipelines to discover new models and techniques relevant to your stack. - 📰 **AI newsletter writers and content creators** — Get daily trending papers with AI summaries already written. Build automated newsletters tracking the hottest ML research. ### Why use Hugging Face Papers Scraper? - 📡 **Pure API-based** — No browser needed. Fast, lightweight (256 MB), and cost-effective - 🤖 **AI summaries included** — Hugging Face provides AI-generated summaries and keywords for each paper - 🔗 **GitHub links and star counts** — Know immediately which papers have code implementations - 📅 **Daily trending mode** — Get the papers the ML community is upvoting right now - 🔍 **Search mode** — Find papers on any ML topic by keyword - 🔄 **Apify platform** — API access, scheduling, webhooks, export to JSON/CSV/Excel, integrate with 5,000+ apps - 💰 **Pay per result** — Only pay for the papers you extract, no flat fees ### What data can you extract? | Field | Description | Example | |-------|-------------|---------| | 📄 `paperId` | ArXiv paper identifier | `2604.15145` | | 📝 `title` | Full paper title | `Attention Is All You Need` | | 👥 `authors` | List of author names | `["Ashish Vaswani", "Noam Shazeer"]` | | 📖 `abstract` | Full paper abstract | `We propose a new simple network...` | | 📅 `publishedAt` | Publication date (ISO 8601) | `2026-04-16T15:19:58.000Z` | | 👍 `upvotes` | Community upvote count | `81` | | 💬 `numComments` | Number of discussion comments | `12` | | 🔗 `paperUrl` | Hugging Face paper page URL | `https://huggingface.co/papers/1706.03762` | | 📚 `arxivUrl` | ArXiv abstract URL | `https://arxiv.org/abs/1706.03762` | | 🐙 `githubUrl` | GitHub repository URL | `https://github.com/tensorflow/tensor2tensor` | | ⭐ `githubStars` | GitHub repository star count | `15432` | | 🤖 `aiSummary` | AI-generated paper summary | `The Transformer architecture...` | | 🏷️ `aiKeywords` | AI-extracted keywords | `["attention mechanism", "Transformer"]` | | 🖼️ `thumbnailUrl` | Paper thumbnail image URL | `https://cdn-thumbnails.huggingface.co/...` | | 🕐 `scrapedAt` | Timestamp of extraction | `2026-04-19T08:15:04.254Z` | ### How much does it cost to scrape Hugging Face papers? Hugging Face Papers Scraper uses **pay-per-event** pricing: | Event | Cost | |-------|------| | 🚀 Run started | $0.001 per run | | 📄 Paper extracted | $0.002 per paper | **Example costs:** - 20 trending papers (1 day) → $0.001 + 20 × $0.002 = **$0.041** - 50 papers from search → $0.001 + 50 × $0.002 = **$0.101** - 200 papers (multi-query) → $0.001 + 200 × $0.002 = **$0.401** With the **free Apify plan ($5/month credits)**, you can extract approximately **2,400 papers per month**. ### How to scrape Hugging Face papers 1. Go to [Hugging Face Papers Scraper](https://apify.com/automation-lab/huggingface-papers-scraper) on Apify Store 2. Click **Try for free** to open the actor in Apify Console 3. Choose your scraping mode: - **Search by keyword** — Enter keywords like "transformer", "reinforcement learning" - **Daily trending** — Enter dates to get that day's trending papers 4. Set the maximum number of papers per query 5. Click **Start** and wait for results 6. Download your data as JSON, CSV, or Excel **Search mode example input:** ```json { "mode": "search", "searchQueries": ["transformer", "large language model"], "maxPapersPerQuery": 50, "includeDetails": true } ```` **Daily trending example input:** ```json { "mode": "daily", "dates": ["2026-04-17", "2026-04-18"], "maxPapersPerQuery": 30 } ``` ### Input parameters | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `mode` | string | `search` | Scraping mode: `search` (keyword search) or `daily` (trending papers) | | `searchQueries` | string\[] | `["transformer"]` | Keywords to search (search mode only) | | `dates` | string\[] | `[]` (today) | Dates in YYYY-MM-DD format (daily mode only) | | `maxPapersPerQuery` | integer | `50` | Maximum papers per keyword or date (1-200) | | `includeDetails` | boolean | `true` | Fetch full paper details (AI summary, GitHub, keywords) | | `maxRequestRetries` | integer | `3` | Retry attempts for failed requests | ### Output example ```json { "paperId": "2604.14268", "title": "HY-World 2.0: A Multi-Modal World Model", "authors": ["Team HY-World", "Chenjie Cao", "Xuhui Zuo"], "abstract": "We present HY-World 2.0, a multi-modal world model...", "publishedAt": "2026-04-15T00:00:00.000Z", "upvotes": 81, "numComments": 5, "paperUrl": "https://huggingface.co/papers/2604.14268", "arxivUrl": "https://arxiv.org/abs/2604.14268", "githubUrl": "https://github.com/Tencent-Hunyuan/HY-World-2.0", "githubStars": 1174, "aiSummary": "HY-World 2.0 presents a multi-modal world model...", "aiKeywords": ["multi-modal world model", "3D Gaussian Splatting"], "thumbnailUrl": "https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2604.14268.png", "scrapedAt": "2026-04-19T08:15:04.254Z" } ``` ### Tips for best results - 🎯 **Start small** — Use 5-10 papers per query for your first run. Scale up once you're happy with the output - 📅 **Daily mode for monitoring** — Schedule daily runs to track trending ML papers automatically - 🔍 **Specific keywords** — "vision transformer" gives more relevant results than just "AI" - ⚡ **Skip details for speed** — Set `includeDetails: false` for faster runs when you only need titles and abstracts - 💾 **Multiple queries** — Search for multiple topics in one run to save on start event costs - 📆 **Date ranges** — In daily mode, pass multiple dates to get trending papers across a date range ### Integrations - 📊 **Hugging Face Papers → Google Sheets** — Track trending papers daily in a spreadsheet. Schedule the actor to run every morning and push results to Sheets via Apify integration - 🔔 **Hugging Face Papers → Slack/Discord** — Get notified when papers matching your keywords appear. Set up a webhook to post new high-upvote papers to your team channel - 📧 **Hugging Face Papers → Email newsletter** — Build an automated weekly ML research digest by scheduling the actor and connecting to Mailchimp via Make/Zapier - 🗃️ **Hugging Face Papers → Airtable/Notion** — Build a searchable research paper database that updates automatically - ⚙️ **Webhooks** — Trigger downstream processing when new papers are found. Parse AI summaries to categorize papers automatically ### Using the Apify API #### Node.js ```javascript import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' }); const run = await client.actor('automation-lab/huggingface-papers-scraper').call({ mode: 'search', searchQueries: ['transformer'], maxPapersPerQuery: 50, includeDetails: true, }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items); ``` #### Python ```python from apify_client import ApifyClient client = ApifyClient("YOUR_APIFY_TOKEN") run = client.actor("automation-lab/huggingface-papers-scraper").call(run_input={ "mode": "search", "searchQueries": ["transformer"], "maxPapersPerQuery": 50, "includeDetails": True, }) items = client.dataset(run["defaultDatasetId"]).list_items().items print(items) ``` #### cURL ```bash curl "https://api.apify.com/v2/acts/automation-lab~huggingface-papers-scraper/runs" \ -X POST \ -H "Content-Type: application/json" \ -H "Authorization: Bearer YOUR_APIFY_TOKEN" \ -d '{ "mode": "search", "searchQueries": ["transformer"], "maxPapersPerQuery": 50, "includeDetails": true }' ``` ### Use with AI agents via MCP Hugging Face Papers Scraper is available as a tool for AI assistants that support the [Model Context Protocol (MCP)](https://docs.apify.com/platform/integrations/mcp). Add the Apify MCP server to your AI client — this gives you access to all Apify actors, including this one: #### Setup for Claude Code ```bash claude mcp add --transport http apify "https://mcp.apify.com?tools=automation-lab/huggingface-papers-scraper" ``` #### Setup for Claude Desktop, Cursor, or VS Code Add this to your MCP config file: ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com?tools=automation-lab/huggingface-papers-scraper" } } } ``` #### Example prompts - "Find the 20 most recent papers about vision transformers on Hugging Face" - "Get today's trending ML papers and summarize the top 5 by upvotes" - "Search for papers about reinforcement learning from human feedback and list their GitHub repos" ### Is it legal to scrape Hugging Face? Hugging Face Papers Scraper accesses publicly available research paper metadata through Hugging Face's public API. All extracted data (paper titles, abstracts, author names) is publicly available information originally published on arXiv and other open-access repositories. We follow ethical scraping practices — no authentication bypass, no rate limit abuse, and full compliance with robots.txt. The actor only reads publicly accessible API endpoints. For GDPR compliance, the actor only extracts public research metadata (author names, paper titles, abstracts). No private user data is collected. If you have concerns about specific data, consult your legal team. ### FAQ **How fast is Hugging Face Papers Scraper?** Very fast. Since it uses Hugging Face's API directly (no browser), a typical run of 50 papers completes in under 30 seconds. **How much does it cost to monitor daily papers?** A daily run fetching ~30 trending papers costs about $0.06. Monthly monitoring (30 days) costs approximately $1.80. **Why do some papers have empty GitHub URLs?** Not all papers have linked GitHub repositories. The `githubUrl` field is only populated when the paper's authors or community members have linked a code repository on Hugging Face. **Why are search results limited?** Hugging Face's search API returns a maximum of ~100 results per query. For broader coverage, use multiple specific search queries instead of one generic term. **Can I get papers from Papers With Code?** Yes! Papers With Code (paperswithcode.com) now redirects to Hugging Face. All data previously available on Papers With Code is now accessible through this actor. **What happened to paperswithcode.com?** Papers With Code was acquired by Meta and has been fully integrated into Hugging Face. All paper URLs now redirect to huggingface.co/papers. ### Other research and data scrapers - 📚 [ArXiv Scraper](https://apify.com/automation-lab/arxiv-scraper) — Extract research papers directly from ArXiv with full abstracts and subjects - 🔬 [Google Scholar Scraper](https://apify.com/automation-lab/google-scholar-scraper) — Search academic papers, citations, and author profiles - 📊 [Google Trends Scraper](https://apify.com/automation-lab/google-trends-scraper) — Track search interest trends over time # Actor input Schema ## `mode` (type: `string`): Choose how to find papers. 'search' finds papers matching keywords. 'daily' gets trending papers for specific dates. ## `searchQueries` (type: `array`): Keywords to search for papers. Each keyword runs a separate search. Used only in 'search' mode. ## `dates` (type: `array`): Dates to fetch daily trending papers (YYYY-MM-DD). Used only in 'daily' mode. Leave empty for today. ## `maxPapersPerQuery` (type: `integer`): Maximum number of papers to extract per search keyword or date. ## `includeDetails` (type: `boolean`): Fetch full details for each paper (AI summary, keywords, GitHub info). Slower but provides richer data. ## `maxRequestRetries` (type: `integer`): Number of retry attempts for failed API requests. ## Actor input object example ```json { "mode": "search", "searchQueries": [ "transformer" ], "dates": [], "maxPapersPerQuery": 20, "includeDetails": true, "maxRequestRetries": 3 } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "search", "searchQueries": [ "transformer" ], "dates": [], "maxPapersPerQuery": 20, "includeDetails": true, "maxRequestRetries": 3 }; // Run the Actor and wait for it to finish const run = await client.actor("automation-lab/huggingface-papers-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "search", "searchQueries": ["transformer"], "dates": [], "maxPapersPerQuery": 20, "includeDetails": True, "maxRequestRetries": 3, } # Run the Actor and wait for it to finish run = client.actor("automation-lab/huggingface-papers-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "search", "searchQueries": [ "transformer" ], "dates": [], "maxPapersPerQuery": 20, "includeDetails": true, "maxRequestRetries": 3 }' | apify call automation-lab/huggingface-papers-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=automation-lab/huggingface-papers-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Hugging Face Papers Scraper", "description": "Extract ML research papers from Hugging Face. Get titles, authors, abstracts, AI summaries, GitHub repos, star counts, upvotes, and keywords. Search by topic or get daily trending papers. Pure API — fast and cheap.", "version": "0.1", "x-build-id": "wXZxvGpaR35HlCoOb" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/automation-lab~huggingface-papers-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-automation-lab-huggingface-papers-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/automation-lab~huggingface-papers-scraper/runs": { "post": { "operationId": "runs-sync-automation-lab-huggingface-papers-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/automation-lab~huggingface-papers-scraper/run-sync": { "post": { "operationId": "run-sync-automation-lab-huggingface-papers-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "mode": { "title": "Scraping mode", "enum": [ "search", "daily" ], "type": "string", "description": "Choose how to find papers. 'search' finds papers matching keywords. 'daily' gets trending papers for specific dates.", "default": "search" }, "searchQueries": { "title": "Search keywords", "type": "array", "description": "Keywords to search for papers. Each keyword runs a separate search. Used only in 'search' mode.", "items": { "type": "string" } }, "dates": { "title": "Dates", "type": "array", "description": "Dates to fetch daily trending papers (YYYY-MM-DD). Used only in 'daily' mode. Leave empty for today.", "items": { "type": "string" } }, "maxPapersPerQuery": { "title": "Max papers per query/date", "minimum": 1, "maximum": 200, "type": "integer", "description": "Maximum number of papers to extract per search keyword or date.", "default": 50 }, "includeDetails": { "title": "Include full paper details", "type": "boolean", "description": "Fetch full details for each paper (AI summary, keywords, GitHub info). Slower but provides richer data.", "default": true }, "maxRequestRetries": { "title": "Max request retries", "minimum": 1, "maximum": 10, "type": "integer", "description": "Number of retry attempts for failed API requests.", "default": 3 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```