Hugging Face Scraper

List and rank models, datasets and spaces on the Hugging Face Hub by downloads, likes or trending score. Filter by task, author or search term, or look up specific repos by name.

Reads the official Hub API. No token needed for public repos.

What you can actually ask it

"What are the most downloaded embedding models right now?" Sort by downloads with a task filter and you get a ranked table: sentence-transformers/all-MiniLM-L6-v2 at 250 million downloads in 30 days, then cross-encoder/ms-marco-MiniLM-L6-v2 at 85 million, then BAAI/bge-small-en-v1.5 at 66 million.

"Which models can we legally ship?" Turn on full detail and filter on license. Apache-2.0 and MIT are shippable; a good number of popular models are not, and the licence is the field people discover too late.

"What is everyone fine-tuning from?" base_model names the parent, so you can see which foundation models the ecosystem actually builds on.

"What changed this month?" Sort by createdAt or lastModified and get new and freshly updated repos.

Fields

• Identity: id, repo_type, author, url
• Popularity: downloads_30d, likes, trending_score
• Classification: pipeline_tag (the task), library_name, tags
• Provenance: base_model, datasets_used, arxiv_ids
• Legal: license
• Reach: languages
• Access: gated, private, disabled
• Dates: created_at, last_modified
• Contents: file_count, files (first 50)
• Spaces only: sdk (gradio, streamlit, docker)

Three things about this data that catch people out

downloads_30d is a 30-day figure, not a lifetime total. The Hub reports downloads over a rolling month. Reading it as all-time understates every new model and overstates nothing; it simply is not the number most people assume it is. The field is named for what it holds.

Licence and file list only exist on the detail call. A listing gives you downloads, likes, tags and task. It does not carry cardData, which is where the licence lives, nor siblings, which is the file list. Turn on Fetch full detail and the Actor makes one extra call per repo to fill those in. When you leave it off and licences come back partly empty, the run summary says so instead of letting you conclude those models have no licence.

Looking up specific repo_ids always fetches detail, so that path is complete.

gated is not private. A gated repo is public and listed, but its files need you to accept terms first. Llama models are the well-known case. You can read all the metadata; you cannot download the weights without agreeing.

A missing repo and a private one look identical

The Hub answers HTTP 401 "Invalid username or password" for a repo that does not exist, exactly as it does for one that is private. That is deliberate: it stops the API leaking whether a private repo exists.

So this Actor does not pretend to know which it hit. The error names both possibilities. Anything else would be a guess dressed as a fact.

The rate limit is published, so it is respected

Every Hub response carries its own limit:

ratelimit-policy: "fixed window";"api";q=500;w=300
ratelimit: "api";r=499;t=272

500 requests per 300 seconds, with the remaining count and seconds-to-reset on every call. This Actor reads both from the response rather than assuming a figure, and pauses until the window resets when fewer than five requests remain, instead of running into a 429. The remaining allowance is reported in the run summary.

One listing request returns 100 repos, so the window is generous: roughly 50,000 repos per five minutes without detail. With Fetch full detail on, every repo costs an extra request, so a 500-repo run with detail uses about 505 of the 500 in the window and will pause once. That is expected, not a fault.

Input reference

Field	Type	Default
repo_type	models, datasets, spaces	models
search	free text over repo names	—
task	task tag, e.g. text-generation	—
author	user or organisation	—
repo_ids	list of exact repo ids	—
sort	downloads, likes, lastModified, createdAt, trendingScore	downloads
direction	desc, asc	desc
detail	fetch licence, languages, files	false
token	only for private or gated repos	—
limit	1-3000	100
retries	1-6	3

Typical uses

• Model selection. Rank by downloads within a task, then filter by licence and language to get a shortlist you can actually use.
• Licence audit. Feed your deployed model ids into repo_ids and read the licences back.
• Ecosystem tracking. Sort by createdAt weekly on a task and watch what is new. base_model shows the lineage.
• Dataset discovery. Switch repo_type to datasets and the same sorting and filtering applies.
• Competitive research. Filter by author to see an organisation's whole published output, with adoption numbers attached.

Notes

Public repos need no token. Supply one only if you need private or gated repos, and it will be sent as a bearer token; the rate limit is the same either way.

Tags on the Hub are freeform and inconsistent. languages prefers the model card's own language field and falls back to language:-prefixed tags, because most repos declare a bare en tag that carries no reliable meaning on its own.