Hugging Face Datasets Scraper

Export public Hugging Face dataset catalog metadata to structured datasets. Use it to find training/RAG datasets, compare licenses and tags, monitor updates, or enrich a list of known dataset IDs.

At a glance

• Primary job: Search Hugging Face datasets and save clean metadata rows.
• Input: Search keywords and/or exact dataset IDs, plus result limits and filters.
• Output: One row per dataset with IDs, URL, author, downloads, likes, tags, license/task/format fields, access flags, timestamps, and diagnostics.
• Best for: AI teams, data scientists, compliance reviewers, research analysts, RAG builders, and fine-tuning dataset discovery.

Common workflows

• Dataset discovery: Search topics such as finance, medical imaging, or instruction tuning and sort by downloads, likes, recency, or trending score.
• License review prep: Export raw Hugging Face tags plus parsed license, task, language, size, format, and library fields for spreadsheet review.
• Known-ID enrichment: Provide exact dataset IDs such as squad or HuggingFaceH4/ultrachat_200k and receive current public metadata.
• Monitoring: Schedule repeat runs with sort: "lastModified" and compare output datasets over time.

Input recipes

Popular finance datasets

{
  "searchQueries": ["finance"],
  "maxItems": 20,
  "sort": "downloads",
  "dedupe": true
}

Exact dataset lookup

{
  "datasetIds": ["squad", "HuggingFaceH4/ultrachat_200k"],
  "maxItems": 5,
  "dedupe": true
}

Recently modified medical imaging datasets

{
  "searchQueries": ["medical imaging"],
  "maxItems": 20,
  "sort": "lastModified",
  "dedupe": true
}

What data can you extract?

Field	Description
datasetId	Stable Hugging Face dataset ID, for example squad or org/name.
url	Dataset page URL on Hugging Face.
author, name	Parsed owner and repository name when available.
description	Public API description, truncated when very long.
downloads, likes, trendingScore	Popularity and trending metrics exposed by Hugging Face.
createdAt, lastModified, sha	Repository recency and revision metadata.
gated, private, disabled, accessStatus	Access flags; the Actor does not bypass gated/private access.
tags	Raw Hugging Face tags preserved for auditability.
license, languages, tasks, sizeCategory, formats, libraries, modalities, region	Parsed convenience fields from raw tags.
cardData	Optional public README dataset card text, truncated and off by default.
sourceQuery, fetchedAt, schemaVersion	Run provenance fields.

Input configuration

Setting	JSON key	Use it for	Example
Search queries	searchQueries	Find datasets by public catalog keyword.	["finance"]
Dataset IDs	datasetIds	Fetch exact dataset metadata.	["squad"]
Maximum records	maxItems	Cap output rows and cost.	50
Sort order	sort	Rank search results.	"downloads"
Minimum downloads	minDownloads	Skip low-download records.	1000
Minimum likes	minLikes	Skip low-like records.	10
Include dataset card	includeCardData	Add bounded README text for saved records.	false
De-duplicate	dedupe	Skip repeated dataset IDs across inputs.	true

Example input

{
  "searchQueries": ["finance"],
  "datasetIds": ["squad"],
  "maxItems": 25,
  "sort": "downloads",
  "dedupe": true
}

Example output

{
  "datasetId": "squad",
  "url": "https://huggingface.co/datasets/squad",
  "author": null,
  "name": "squad",
  "description": "Stanford Question Answering Dataset...",
  "descriptionTruncated": false,
  "downloads": 12345,
  "likes": 100,
  "trendingScore": 0,
  "createdAt": "2022-03-02T00:00:00.000Z",
  "lastModified": "2024-01-01T00:00:00.000Z",
  "sha": "...",
  "gated": false,
  "private": false,
  "disabled": false,
  "accessStatus": "public",
  "tags": ["task_categories:question-answering", "language:en"],
  "license": null,
  "languages": ["en"],
  "tasks": ["question-answering"],
  "formats": ["parquet"],
  "sourceQuery": "dataset:squad",
  "fetchedAt": "2026-07-17T00:00:00.000Z",
  "schemaVersion": "1.0.0"
}

Pricing

Event	Price	Charged when
Run start	$0.005 per run	Once when the Actor starts.
Dataset metadata record	Tiered from $0.000575 down to $0.00014 per saved dataset row	Each dataset metadata item is saved.

Store discount tiers for dataset metadata records: Free $0.000575, Bronze $0.0005, Silver $0.00039, Gold $0.0003, Platinum $0.0002, Diamond $0.00014.

Tips for best results

• Start small: Use maxItems: 10-50 while tuning queries.
• Use focused terms: Domain phrases such as legal documents or medical imaging are usually better than one-word broad searches.
• Leave card data off by default: Enable includeCardData only when you need README text because it adds one request per saved item.
• Respect access flags: Gated datasets are marked; this Actor does not fetch gated contents.

API usage

Node.js:

import { ApifyClient } from "apify-client";

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor("fetch_cat/hugging-face-datasets-scraper").call({
  searchQueries: ["finance"],
  maxItems: 20,
  sort: "downloads"
});
console.log(run.defaultDatasetId);

Python:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("fetch_cat/hugging-face-datasets-scraper").call(run_input={
    "searchQueries": ["finance"],
    "maxItems": 20,
    "sort": "downloads",
})
print(run["defaultDatasetId"])

cURL:

curl -X POST "https://api.apify.com/v2/acts/fetch_cat~hugging-face-datasets-scraper/runs?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchQueries":["finance"],"maxItems":20,"sort":"downloads"}'

MCP and AI agents

Use this Actor from MCP-compatible tools through the official Apify MCP Server.

$claude mcp add --transport http apify "https://mcp.apify.com?tools=fetch_cat/hugging-face-datasets-scraper"

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=fetch_cat/hugging-face-datasets-scraper"
    }
  }
}

Example prompt: "Find 20 popular finance datasets on Hugging Face and summarize their licenses, tasks, and access flags."

Limits and caveats

• Public metadata only: The Actor uses public Hugging Face catalog endpoints and public README files only.
• No gated-content bypass: Gated/private/disabled flags are exported as metadata; protected dataset contents are not downloaded.
• Source changes: Hugging Face fields and tags can change over time.
• Diagnostics: Per-source run diagnostics are stored in the DIAGNOSTICS key-value record.

Legality and responsible use

Process only data you are allowed to access. Follow Hugging Face terms, dataset licenses, Apify's terms, and applicable laws.

Related actors

• Hugging Face Models Scraper
• Google Scholar Researcher Profile Scraper

FAQ

Can it download dataset rows?

No. This Actor is for public metadata discovery and monitoring, not dataset content extraction.

Why are some fields empty?

Hugging Face does not expose every tag or metric for every dataset. Missing fields are left empty instead of guessed.

Can I export results?

Yes. Apify datasets support JSON, CSV, Excel, XML, RSS, and API access.

Support

Open an issue from the Actor page if a run fails or output looks wrong. Include the run ID or URL, input JSON, expected output, actual output, and one reproducible public URL such as https://huggingface.co/datasets/squad.