# Hugging Face Datasets Scraper (`fetch_cat/hugging-face-datasets-scraper`) Actor Scrape public Hugging Face dataset metadata, tags, access flags, downloads, likes, and recency signals. - **URL**: https://apify.com/fetch\_cat/hugging-face-datasets-scraper.md - **Developed by:** [Hanna Nosova](https://apify.com/fetch_cat) (community) - **Categories:** AI, Developer tools, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.30 / 1,000 dataset metadata records This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Hugging Face Datasets Scraper Export public Hugging Face dataset catalog metadata to structured datasets. Use it to find training/RAG datasets, compare licenses and tags, monitor updates, or enrich a list of known dataset IDs. ### At a glance - **Primary job**: Search Hugging Face datasets and save clean metadata rows. - **Input**: Search keywords and/or exact dataset IDs, plus result limits and filters. - **Output**: One row per dataset with IDs, URL, author, downloads, likes, tags, license/task/format fields, access flags, timestamps, and diagnostics. - **Best for**: AI teams, data scientists, compliance reviewers, research analysts, RAG builders, and fine-tuning dataset discovery. ### Common workflows - **Dataset discovery**: Search topics such as `finance`, `medical imaging`, or `instruction tuning` and sort by downloads, likes, recency, or trending score. - **License review prep**: Export raw Hugging Face tags plus parsed license, task, language, size, format, and library fields for spreadsheet review. - **Known-ID enrichment**: Provide exact dataset IDs such as `squad` or `HuggingFaceH4/ultrachat_200k` and receive current public metadata. - **Monitoring**: Schedule repeat runs with `sort: "lastModified"` and compare output datasets over time. ### Input recipes **Popular finance datasets** ```json { "searchQueries": ["finance"], "maxItems": 20, "sort": "downloads", "dedupe": true } ```` **Exact dataset lookup** ```json { "datasetIds": ["squad", "HuggingFaceH4/ultrachat_200k"], "maxItems": 5, "dedupe": true } ``` **Recently modified medical imaging datasets** ```json { "searchQueries": ["medical imaging"], "maxItems": 20, "sort": "lastModified", "dedupe": true } ``` ### What data can you extract? | Field | Description | | --- | --- | | `datasetId` | Stable Hugging Face dataset ID, for example `squad` or `org/name`. | | `url` | Dataset page URL on Hugging Face. | | `author`, `name` | Parsed owner and repository name when available. | | `description` | Public API description, truncated when very long. | | `downloads`, `likes`, `trendingScore` | Popularity and trending metrics exposed by Hugging Face. | | `createdAt`, `lastModified`, `sha` | Repository recency and revision metadata. | | `gated`, `private`, `disabled`, `accessStatus` | Access flags; the Actor does not bypass gated/private access. | | `tags` | Raw Hugging Face tags preserved for auditability. | | `license`, `languages`, `tasks`, `sizeCategory`, `formats`, `libraries`, `modalities`, `region` | Parsed convenience fields from raw tags. | | `cardData` | Optional public README dataset card text, truncated and off by default. | | `sourceQuery`, `fetchedAt`, `schemaVersion` | Run provenance fields. | ### Input configuration | Setting | JSON key | Use it for | Example | | --- | --- | --- | --- | | Search queries | `searchQueries` | Find datasets by public catalog keyword. | `["finance"]` | | Dataset IDs | `datasetIds` | Fetch exact dataset metadata. | `["squad"]` | | Maximum records | `maxItems` | Cap output rows and cost. | `50` | | Sort order | `sort` | Rank search results. | `"downloads"` | | Minimum downloads | `minDownloads` | Skip low-download records. | `1000` | | Minimum likes | `minLikes` | Skip low-like records. | `10` | | Include dataset card | `includeCardData` | Add bounded README text for saved records. | `false` | | De-duplicate | `dedupe` | Skip repeated dataset IDs across inputs. | `true` | ### Example input ```json { "searchQueries": ["finance"], "datasetIds": ["squad"], "maxItems": 25, "sort": "downloads", "dedupe": true } ``` ### Example output ```json { "datasetId": "squad", "url": "https://huggingface.co/datasets/squad", "author": null, "name": "squad", "description": "Stanford Question Answering Dataset...", "descriptionTruncated": false, "downloads": 12345, "likes": 100, "trendingScore": 0, "createdAt": "2022-03-02T00:00:00.000Z", "lastModified": "2024-01-01T00:00:00.000Z", "sha": "...", "gated": false, "private": false, "disabled": false, "accessStatus": "public", "tags": ["task_categories:question-answering", "language:en"], "license": null, "languages": ["en"], "tasks": ["question-answering"], "formats": ["parquet"], "sourceQuery": "dataset:squad", "fetchedAt": "2026-07-17T00:00:00.000Z", "schemaVersion": "1.0.0" } ``` ### Pricing | Event | Price | Charged when | | --- | --- | --- | | Run start | `$0.005` per run | Once when the Actor starts. | | Dataset metadata record | Tiered from `$0.000575` down to `$0.00014` per saved dataset row | Each dataset metadata item is saved. | Store discount tiers for dataset metadata records: Free `$0.000575`, Bronze `$0.0005`, Silver `$0.00039`, Gold `$0.0003`, Platinum `$0.0002`, Diamond `$0.00014`. ### Tips for best results - **Start small**: Use `maxItems: 10-50` while tuning queries. - **Use focused terms**: Domain phrases such as `legal documents` or `medical imaging` are usually better than one-word broad searches. - **Leave card data off by default**: Enable `includeCardData` only when you need README text because it adds one request per saved item. - **Respect access flags**: Gated datasets are marked; this Actor does not fetch gated contents. ### API usage Node.js: ```js import { ApifyClient } from "apify-client"; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor("fetch_cat/hugging-face-datasets-scraper").call({ searchQueries: ["finance"], maxItems: 20, sort: "downloads" }); console.log(run.defaultDatasetId); ``` Python: ```python from apify_client import ApifyClient client = ApifyClient("YOUR_APIFY_TOKEN") run = client.actor("fetch_cat/hugging-face-datasets-scraper").call(run_input={ "searchQueries": ["finance"], "maxItems": 20, "sort": "downloads", }) print(run["defaultDatasetId"]) ``` cURL: ```bash curl -X POST "https://api.apify.com/v2/acts/fetch_cat~hugging-face-datasets-scraper/runs?token=YOUR_APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{"searchQueries":["finance"],"maxItems":20,"sort":"downloads"}' ``` ### MCP and AI agents Use this Actor from MCP-compatible tools through the official Apify MCP Server. ```bash claude mcp add --transport http apify "https://mcp.apify.com?tools=fetch_cat/hugging-face-datasets-scraper" ``` ```json { "mcpServers": { "apify": { "url": "https://mcp.apify.com?tools=fetch_cat/hugging-face-datasets-scraper" } } } ``` Example prompt: "Find 20 popular finance datasets on Hugging Face and summarize their licenses, tasks, and access flags." ### Limits and caveats - **Public metadata only**: The Actor uses public Hugging Face catalog endpoints and public README files only. - **No gated-content bypass**: Gated/private/disabled flags are exported as metadata; protected dataset contents are not downloaded. - **Source changes**: Hugging Face fields and tags can change over time. - **Diagnostics**: Per-source run diagnostics are stored in the `DIAGNOSTICS` key-value record. ### Legality and responsible use Process only data you are allowed to access. Follow Hugging Face terms, dataset licenses, Apify's terms, and applicable laws. ### Related actors - [Hugging Face Models Scraper](https://apify.com/fetch_cat/hugging-face-models-scraper) - [Google Scholar Researcher Profile Scraper](https://apify.com/fetch_cat/google-scholar-researcher-profile-scraper) ### FAQ #### Can it download dataset rows? No. This Actor is for public metadata discovery and monitoring, not dataset content extraction. #### Why are some fields empty? Hugging Face does not expose every tag or metric for every dataset. Missing fields are left empty instead of guessed. #### Can I export results? Yes. Apify datasets support JSON, CSV, Excel, XML, RSS, and API access. ### Support Open an issue from the Actor page if a run fails or output looks wrong. Include the run ID or URL, input JSON, expected output, actual output, and one reproducible public URL such as `https://huggingface.co/datasets/squad`. # Actor input Schema ## `searchQueries` (type: `array`): Keywords to search in the public Hugging Face dataset catalog. Each query is processed independently and results are de-duplicated by dataset ID. ## `datasetIds` (type: `array`): Optional exact dataset IDs such as squad or HuggingFaceH4/ultrachat\_200k. Use this for direct metadata lookups. ## `maxItems` (type: `integer`): Maximum number of dataset metadata records to save across all queries and IDs. ## `sort` (type: `string`): Sort used for keyword searches. Exact dataset ID lookups ignore this setting. ## `minDownloads` (type: `integer`): Only save datasets with at least this many downloads. Leave empty to disable. ## `minLikes` (type: `integer`): Only save datasets with at least this many likes. Leave empty to disable. ## `includeCardData` (type: `boolean`): Fetch the public README.md dataset card for each saved dataset, truncated to a safe length. This adds one extra request per saved dataset and is off by default. ## `dedupe` (type: `boolean`): Skip duplicate dataset IDs across multiple queries and direct lookups. ## Actor input object example ```json { "searchQueries": [ "finance", "medical imaging" ], "datasetIds": [ "squad", "HuggingFaceH4/ultrachat_200k" ], "maxItems": 20, "sort": "downloads", "includeCardData": false, "dedupe": true } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "searchQueries": [ "finance", "medical imaging" ], "datasetIds": [ "squad", "HuggingFaceH4/ultrachat_200k" ] }; // Run the Actor and wait for it to finish const run = await client.actor("fetch_cat/hugging-face-datasets-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "searchQueries": [ "finance", "medical imaging", ], "datasetIds": [ "squad", "HuggingFaceH4/ultrachat_200k", ], } # Run the Actor and wait for it to finish run = client.actor("fetch_cat/hugging-face-datasets-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "searchQueries": [ "finance", "medical imaging" ], "datasetIds": [ "squad", "HuggingFaceH4/ultrachat_200k" ] }' | apify call fetch_cat/hugging-face-datasets-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=fetch_cat/hugging-face-datasets-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Hugging Face Datasets Scraper", "description": "Scrape public Hugging Face dataset metadata, tags, access flags, downloads, likes, and recency signals.", "version": "0.1", "x-build-id": "fmIuLiznB9m6b2w2O" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/fetch_cat~hugging-face-datasets-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-fetch_cat-hugging-face-datasets-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/fetch_cat~hugging-face-datasets-scraper/runs": { "post": { "operationId": "runs-sync-fetch_cat-hugging-face-datasets-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/fetch_cat~hugging-face-datasets-scraper/run-sync": { "post": { "operationId": "run-sync-fetch_cat-hugging-face-datasets-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "searchQueries": { "title": "Search queries", "type": "array", "description": "Keywords to search in the public Hugging Face dataset catalog. Each query is processed independently and results are de-duplicated by dataset ID.", "items": { "type": "string" } }, "datasetIds": { "title": "Dataset IDs", "type": "array", "description": "Optional exact dataset IDs such as squad or HuggingFaceH4/ultrachat_200k. Use this for direct metadata lookups.", "items": { "type": "string" } }, "maxItems": { "title": "Maximum dataset records", "minimum": 1, "maximum": 10000, "type": "integer", "description": "Maximum number of dataset metadata records to save across all queries and IDs.", "default": 20 }, "sort": { "title": "Sort order", "enum": [ "downloads", "likes", "lastModified", "createdAt", "trendingScore" ], "type": "string", "description": "Sort used for keyword searches. Exact dataset ID lookups ignore this setting.", "default": "downloads" }, "minDownloads": { "title": "Minimum downloads", "minimum": 0, "type": "integer", "description": "Only save datasets with at least this many downloads. Leave empty to disable." }, "minLikes": { "title": "Minimum likes", "minimum": 0, "type": "integer", "description": "Only save datasets with at least this many likes. Leave empty to disable." }, "includeCardData": { "title": "Include dataset card README", "type": "boolean", "description": "Fetch the public README.md dataset card for each saved dataset, truncated to a safe length. This adds one extra request per saved dataset and is off by default.", "default": false }, "dedupe": { "title": "De-duplicate results", "type": "boolean", "description": "Skip duplicate dataset IDs across multiple queries and direct lookups.", "default": true } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```