# Hugging Face Model Rankings API & Monitor (`automa-flow/huggingface-model-rankings-monitor`) Actor See who ranks in a Hugging Face Hub task top-N or on a named model watchlist. Export the public catalog, then monitor downloads, likes, license and rank changes through the official Hub API. No HTML scrape, model weights or login. Schedule the same query to catch rank moves without a full re-export. - **URL**: https://apify.com/automa-flow/huggingface-model-rankings-monitor.md - **Developed by:** [Vadim Bezrukov](https://apify.com/automa-flow) (community) - **Categories:** AI, Developer tools, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.60 / 1,000 model results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Hugging Face Model Rankings API & Monitor See who sits in a Hugging Face Hub task top-N, or on a named model watchlist, and whether rank, presence, license or gating moved since your last successful run, with downloads and likes deltas on every row. The Actor calls the official Hugging Face Hub HTTP API. It does not scrape huggingface.co HTML, download weights, or open private/gated files. Paste the prefill below, run once, and you get the current public `text-generation` catalog plus one `query_status` row. Tomorrow, switch the same query to `monitor` + `changesOnly` and add three competitor ids as a watchlist. Build a daily shortlist or change alert without diffing two full exports. Each row is an observation at `scraped_at`; the Actor does not reconstruct history from before your first monitor run. ### Choose your workflow | Goal | Settings | What you get | | --- | --- | --- | | Export today's top-N for a task | `snapshot` + `all` | One `model_observation` per public model, all `BASELINE` | | Re-check the same segment tomorrow | `monitor` + `changesOnly` | `NEW`, `UPDATED`, `RANK_UP`, `RANK_DOWN`, `EXITED`, `REAPPEARED`; a quiet day is one `query_status` row | | Track download / like momentum | `monitor` + `all` | Every row, including `UNCHANGED`, carries `downloads_delta` and `likes_delta` against the last successful run | | Watch named competitors | `kind: watchlist` | One row per id: the model, or a `NOT_FOUND` / `FAILED` row for that id; a missing repo never fails the query | `BASELINE` means "this is the first successful catalog we stored for this query". It does not mean the model was created today. ### Hugging Face model ranking workflows These are the three jobs the Actor is built for. Ready-made examples you can copy and run: - [Export Hugging Face Text-Generation Top Models](https://apify.com/automa-flow/huggingface-model-rankings-monitor/examples/export-huggingface-text-generation-top-models) - [Monitor Hugging Face Text-Generation Rank Changes](https://apify.com/automa-flow/huggingface-model-rankings-monitor/examples/monitor-huggingface-text-generation-rank-changes) - [Watch Hugging Face Model Competitors by ID](https://apify.com/automa-flow/huggingface-model-rankings-monitor/examples/watch-named-huggingface-model-competitors) ### Quick start ```json { "mode": "snapshot", "queries": [ { "kind": "segment", "pipelineTag": "text-generation", "sort": "downloads", "topN": 20 } ] } ``` Then schedule the same query: ```json { "mode": "monitor", "outputMode": "changesOnly", "stateNamespace": "textgen-top20", "queries": [ { "kind": "segment", "pipelineTag": "text-generation", "sort": "downloads", "topN": 20 } ] } ``` Keep `stateNamespace` stable for a given watch. Snapshot mode does not read or write history. `snapshot` + `changesOnly` is rejected before any Hub request. A watchlist accepts `namespace/name` or `https://huggingface.co/namespace/name` (including `www.huggingface.co`). Other hosts, and Hub URLs for datasets, Spaces, collections or papers, are `INVALID_INPUT` for that query, without a fetch. Rank is input order after de-duplication; the same ids in a different order are a different `query_key`. ### Hub API vs this Actor API - **Hub API:** `GET https://huggingface.co/api/models` and `GET https://huggingface.co/api/models/{namespace}/{name}`. Public JSON, no token in this Actor. - **Actor API:** Apify input, Dataset rows (`model_observation`, `query_status`, `run_summary`), and optional last-good state in this Actor's key-value store. Pagination follows Hub `Link: rel="next"` with `cursor=`. The Actor does not invent `page=` parameters. A short live probe on 2026-09-11 confirmed that header on `limit=2`. List pagination stops after 100 pages or a repeated next URL; that is `PARTIAL` / `SOURCE_LIMIT`, not an empty catalog. Last-good state is kept. ### What changed, and what did not Monitor diffs **last successful** state against **current successful** state. - First successful monitor, including `changesOnly`, emits `BASELINE`. - A new id after that is `NEW`. Rank-only moves are `RANK_UP` / `RANK_DOWN`. A change in `license`, `gated`, `pipeline_tag`, `library_name` or `last_modified` is `UPDATED`, with the old values in `previous_values`. - `downloads` (a rolling 30-day counter on the Hub), `likes` and `trending_score` drift on almost every popular model every day, and even within hours. They do **not** classify a row by default: a row whose only movement is metric drift is `UNCHANGED`, and every monitor row carries `downloads_previous`, `downloads_delta`, `likes_previous`, `likes_delta`, `trending_score_previous` and `trending_score_delta` instead. A rank move is still `RANK_UP` / `RANK_DOWN` even when downloads moved with it. Set `metricChangeThresholdPct` (0-1000) to also get `UPDATED` when a metric moves by at least that share of its previous value; `0` means any metric change. The row `fingerprint` still covers the metrics, so history stays exact. - An id missing from **two different sequential successful complete** observations becomes `EXITED`. The first miss is `pending_exits` only. A transport error, incomplete watchlist, or `OUTPUT_LIMIT` does not advance that counter. A watchlist pass where every remaining id is delivered or explicitly `NOT_FOUND` is complete, even if the query status is `PARTIAL`. - `EXITED` means "not in this query's successful top-N / watchlist response". It does **not** mean the model was deleted from the Hub. - If a successful segment keeps under 50% of the previous ids at the same `topN`, the query is `PARTIAL` (mass-removal guard). Pending exits are not applied. High-churn reshuffles can look like this on purpose. The guard does not fire for `topN=1` or a previous set of one id: replacing the leader is a real change. After independently checking the Hub ranking, rerun **only that segment** with `mode: monitor`, the same namespace and `confirmMassRemoval: true`. This accepts a complete, valid new composition without deleting history. Missing ids still require two successful checks before `EXITED`. The status code is `MASS_REMOVAL_CONFIRMED` when used. Remove the confirmation flag afterwards; never enable it in a schedule. Source failures, malformed rows and pagination limits cannot be overridden. - Broken stored state is `STATE_INCOMPATIBLE` for that query, not a silent new baseline. Private `true` rows are dropped before Dataset and state. `gated` may be `false`, `true`, or Hub's `auto` / `manual` string. Files are never downloaded. Segment requests explicitly expand the ranking and comparison fields, including `gated`, `lastModified` and `trendingScore`. `observed_fields` lists the normalized comparison fields actually supplied in this response. A null with its name in that list is an explicit empty value; a name absent from the list is unknown. Unknown fields do not trigger changes or overwrite their last verified state. The next comparison uses the last available value, which may predate the last run. Fields first becoming available establish a field baseline, not `UPDATED`. Legacy snapshots remain readable; legacy nulls are treated as unknown. Monitor `event_id` values now use the opaque format `v2:::::`. Independent namespaces no longer collide. Existing historical ids are not rewritten: consumers must accept both old and v2 ids during migration and must not parse them to obtain entity identity (use `source_id`). Snapshot runs do not advance a generation; retain `scraped_at` when storing snapshot history. Event ids are scoped to an account's state store, not globally across accounts. Each canonical public watchlist model gets one `SUCCESS` observation (unless `changesOnly` omits an unchanged model). Aliases resolving to the same model share that observation and one billable event; `requested_model_ids` lists the input IDs and rank uses their first input position. A missing repo (Hub 404, or 401 `Invalid username or password.`) and a `private: true` repo are a `NOT_FOUND` row with `error.code` `NOT_FOUND` / `PRIVATE`. A transport error or an unusable detail payload is a `FAILED` row with the transport code. These per-id rows have `change_type: null`, `matches_query: false`, are never billed and do not count against `maxModelRows`, so your webhook can alert on a specific id going missing without diffing two exports. Free per-id statuses already fetched are delivered even when a post-fetch spend limit prevents the paid observations from being delivered. Queries skipped before fetching have a query-level `NOT_PROCESSED` status. A watchlist id is resolved when Hub returns the model, `private: true`, or an explicit missing-repo 404 / 401. If every id is resolved and some are missing, the query is `PARTIAL` (no `query-check`), but last-good state and pending/`EXITED` **are** updated. If every id is missing, the query is `NOT_FOUND` (a verified observation, charged as one `query-check` in monitor mode) and the previous composition moves toward `EXITED`. A transport error on any id still delivers the models that were found, does not move pending, and does not charge `model-result` for that incomplete pass. A 200 detail without a usable `id` / `modelId` is unverified: other resolved ids still complete the pass (`PARTIAL_UNVERIFIED`), last-good state for the bad id is kept, and explicit `NOT_FOUND` ids can still pending/`EXITED`. ### Failure semantics `NO_RESULTS` is a verified Hub `[]` (or no public models after filtering). It is never a transport failure. HTML, truncated JSON, HTTP 5xx/429-after-retry, or 401 on the **list** endpoint are `FAILED`. Unknown Hub repos often return **401** `Invalid username or password.` on the detail endpoint; this Actor maps that message (and HTTP 404) to `NOT_FOUND` for that id. Any other detail 401 is a failed id, not a missing repo. The Actor does not send credentials. One bad query does not kill the batch. `maxModelRows` never truncates a query: if the query's model rows do not fit, you get `PARTIAL` / `OUTPUT_LIMIT` and the previous state is kept. Pagination counts unique model IDs, including when pages overlap. Any invalid model row makes the segment `FAILED` / `INVALID_LIST_ITEM`: its ranks and composition cannot be verified, so it emits no paid events and keeps state. Model IDs reject URL control characters such as `?`, `#` and `%`. The run itself ends as **FAILED** (`SOURCE_FAILED` in the status message) only when every query that reached the Hub came back as a source-level failure (5xx, 429 after retries, network errors, HTML or malformed JSON, a redirect off huggingface.co). All Dataset rows, the `run_summary` and the `RUN_SUMMARY` record are still written first, so a scheduled monitor gets a real failure to alert on instead of a quiet `SUCCEEDED` with an empty catalog. `INVALID_INPUT`, `DUPLICATE`, `NO_RESULTS`, `NOT_FOUND`, `STATE_INCOMPATIBLE` and a mixed batch keep the run `SUCCEEDED`. A page safety limit with usable model observations remains `PARTIAL` and does not by itself fail the run. Completed delivery checkpoints retain the final run outcome, so resurrection preserves a source failure without replaying Dataset rows or charges. ### Pricing The Store **Pricing** tab is authoritative. Current pay-per-event rates: | Event | When it is charged | FREE / BRONZE | | --- | --- | ---: | | `apify-actor-start` | Platform start; one event at the default 512 MB. The Actor never emits it. | $0.00005 | | `model-result` | Each delivered `SUCCESS` model row in snapshot. In monitor, only `NEW` / `UPDATED` / `RANK_*` / `EXITED` / `REAPPEARED`. Eligible verified rows in `PARTIAL_NOT_FOUND` and `PARTIAL_UNVERIFIED` watchlists are charged; unresolved ids are free. Metric-only drift without `metricChangeThresholdPct` is `UNCHANGED` and free. | $0.001 | | `query-check` | Monitor mode, unique query whose requested result was fully delivered, including `NO_RESULTS`, a fully `NOT_FOUND` watchlist and zero-change. Not charged for `PARTIAL`, `FAILED`, duplicates or incomplete watchlists. | $0.01 | `model-result` is $0.0008 on SILVER and $0.0006 on GOLD+. Platform usage is included, not passed through. `apify-default-dataset-item` is off: enabling it would double-bill every Dataset row. Private cloud measurement on 2026-09-11 (builds 0.1.2 and 0.1.3, 512 MB): snapshot of 20 models → 1 × $0.00005 start + 20 × $0.001 = $0.02005; quiet or first monitor of the same top-20 → start + $0.01 `query-check` = $0.01005 and $0 change rows. Builds before 0.1.6 counted daily downloads/likes drift as `UPDATED`, so a real day-apart monitor of a downloads-sorted top-20 billed about 20 change rows; from 0.1.6 that drift is `UNCHANGED` with deltas and a day-apart pass with no rank or presence move is the $0.01005 quiet check. A run cap must cover start plus the events: $0.02 is not enough for the sample top-20, and $0.01 is not enough for a quiet monitor. The minimum run cap is $0.021 so the Store prefill can finish. On 0.1.3 a top-100 snapshot at that minimum cap and an `OUTPUT_LIMIT` that cannot fit `topN` skip the Hub request (`httpRequestCount` 0). Retries, duplicates, `FAILED`, invalid hosts, `PARTIAL_WATCHLIST`, `SOURCE_LIMIT`, `MASS_REMOVAL` and output/spend-limited queries are not charged as `model-result` or `query-check` (the platform still charges start). `PARTIAL_NOT_FOUND` and `PARTIAL_UNVERIFIED` have no query-check fee, but their eligible verified model rows do incur model-result charges. A query is charged as whole `query-check` then `model-result` batches after its Dataset rows are written. `query-check` is charged first so a later refused `model-result` batch cannot leave models billed while state stays unmoved. If any batch charge is refused, state is not advanced and a later run may emit the same observations. Snapshot spend and `maxModelRows` limits are checked before the Hub request when the row count is known. Monitor still fetches when `query-check` fits; a later burst of change rows can still stop after the request. The $0.021 minimum covers a quiet/first monitor, not twenty billed changes plus `query-check` ($0.03005). Use the default $0.50 run cap or the monitor Task cap ($0.22) for that. ### Limitations - Even expanded Hub rows can omit optional metadata. Use `observed_fields` to distinguish missing fields from explicit nulls; null never means zero. - There is no native Hub history comparable to third-party daily dumps. - EXITED tombstones are kept 180 days, at most 20 000 per query. After prune, a very old return may classify as `NEW`. - MVP assumes one writer per `stateNamespace` and sequential runs. There is no distributed lock. - Dataset views cannot filter by `record_type`, so the *Changes and deltas* view also lists `query_status` and `run_summary` rows with empty model columns. Filter on `record_type=model_observation` in your consumer. ### Use from AI agents with MCP Connect the public Actor directly at `https://mcp.apify.com?tools=automa-flow/huggingface-model-rankings-monitor`. Ask the agent to keep `mode: monitor` and `outputMode: changesOnly` on repeat runs, and to treat `NO_RESULTS` as a verified empty catalog, not a source failure. This Actor is **not** targeted for MCP payment; do not treat that URL as agentic billing readiness. ### Automation example (n8n / Make) This is a copy-paste illustration, not a hosted integration promise. 1. Schedule this Actor daily with the monitor JSON above. 2. Webhook the Dataset URL into n8n/Make. 3. Keep rows where `record_type=model_observation` and `change_type` is one of `NEW`, `UPDATED`, `RANK_UP`, `RANK_DOWN`, `EXITED`, `REAPPEARED`, plus watchlist rows with `status` `NOT_FOUND` or `FAILED`. 4. Open `source_url` for the human check. Ignore `EXITED` as "deleted from Hub". 5. For momentum, run `monitor` + `all` and sort by `downloads_delta`. The saved monitor examples use `metricChangeThresholdPct: 5`: a metric must change by at least 5% from its last verified value to emit `UPDATED`. Remove the threshold for structural/rank-only alerts, or use `all` for every metric observation. The threshold is per comparison, not cumulative since the last alert. A missing watchlist id produces a free status on each check; deduplicate those notifications by `source_id` and status transition in your consumer. ### Protocol notes User-Agent: `automa-flow-huggingface-model-rankings-monitor/0.1`. Origin is fixed to `https://huggingface.co` (www is accepted on the same host). Redirects off that origin fail the query. Legal review on 2026-09-11, re-read again the same day immediately before Store publication: `https://huggingface.co/robots.txt` is `User-agent: *` / `Allow: /`. Terms of Service effective 2022-09-15 contain no scrape, crawler or monitoring ban for this path; a public repository grants Users a license to use and reproduce its Content. This Actor only reads public Hub model metadata. Re-read both before Store publication; a later ban is a stop, not something to bypass. # Actor input Schema ## `queries` (type: `array`): One to 20 Hugging Face Hub ranking or watchlist queries. A ranking is a Hub list (task, author and/or search, sorted, top N). A watchlist is explicit model ids. One bad query does not fail the others. ## `mode` (type: `string`): Get current catalog returns every matching public model and writes no history. Monitor changes compares this successful run with the last successful run of the same query in the same state namespace. ## `outputMode` (type: `string`): All models returns every current row, including UNCHANGED rows with metric deltas. Changes only omits UNCHANGED rows. Snapshot plus changesOnly is rejected before any Hub request. ## `stateNamespace` (type: `string`): Isolates monitor history. Use a stable name per watch, for example textgen-top20. Letters, numbers, '\_' and '-', 1-64 characters. Snapshot mode ignores it. ## `maxModelRows` (type: `integer`): Cap on delivered SUCCESS model\_observation rows for the whole run. Per-id NOT\_FOUND / FAILED rows, query\_status and run\_summary rows are free and not counted. A query that would exceed the remaining cap is skipped as a whole (PARTIAL/OUTPUT\_LIMIT), not truncated. ## `metricChangeThresholdPct` (type: `integer`): Monitor mode only. Hub downloads (rolling 30 days), likes and trendingScore drift on almost every model every day, so by default they never classify a row: you get them as downloads\_delta / likes\_delta on every row and a quiet day stays quiet. Set a percentage (0-1000) to also emit UPDATED when a metric moves by at least that share of its previous value; 0 means any metric change is UPDATED. Rank, presence, license, gated, pipeline\_tag, library\_name and lastModified always classify. ## `confirmMassRemoval` (type: `boolean`): One-off recovery only: accept a complete ranking that retained less than 50% of the previous models. Requires monitor mode and exactly one segment query. Review the Hub ranking first. Keeps history and the two-check EXITED rule. Leave false for scheduled runs; cannot override source failures or incomplete pages. ## Actor input object example ```json { "queries": [ { "kind": "segment", "pipelineTag": "text-generation", "sort": "downloads", "topN": 20 } ], "mode": "snapshot", "outputMode": "all", "stateNamespace": "default", "maxModelRows": 200, "confirmMassRemoval": false } ``` # Actor output Schema ## `results` (type: `string`): No description ## `runSummary` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "queries": [ { "kind": "segment", "pipelineTag": "text-generation", "sort": "downloads", "topN": 20 } ] }; // Run the Actor and wait for it to finish const run = await client.actor("automa-flow/huggingface-model-rankings-monitor").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "queries": [{ "kind": "segment", "pipelineTag": "text-generation", "sort": "downloads", "topN": 20, }] } # Run the Actor and wait for it to finish run = client.actor("automa-flow/huggingface-model-rankings-monitor").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "queries": [ { "kind": "segment", "pipelineTag": "text-generation", "sort": "downloads", "topN": 20 } ] }' | apify call automa-flow/huggingface-model-rankings-monitor --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,automa-flow/huggingface-model-rankings-monitor" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/TgBIRs4raRtWw4tRW/builds/GJ0BxRCI58gucgb9p/openapi.json