🤗 Hugging Face AI Models, Datasets & Trends Scraper

A high-performance Apify Actor built in Python to extract trending AI models, open-source LLMs, datasets, download velocity, upvote counts, task pipeline tags, and architecture metadata directly from Hugging Face's REST APIs.

---

🌟 Key Features

• ⚡ Direct REST API: Hits official Hugging Face API endpoints (huggingface.co/api/models and huggingface.co/api/datasets) directly. No headless browser overhead (~128MB RAM footprint).
• 🤖 Models & Datasets Support: Seamlessly switch between extracting machine learning models or open-source datasets.
• 🏷️ Task & Pipeline Filtering: Filter AI models by task pipeline (e.g., text-generation, text-to-image, automatic-speech-recognition, zero-shot-classification).
• 📈 Popularity & Recency Metrics: Sort records by downloads, likes, lastModified, or createdAt.
• 📊 Store-Ready Output: Automatically pushes Pydantic-validated dataset records with built-in tabular overview support.

---

📥 Input Parameters

The Actor accepts the following input settings in JSON format:

Parameter	Type	Default	Description
resourceType	string	"models"	Resource type to scrape: "models" or "datasets".
pipelineTag	string	"text-generation"	Filter models by task tag (e.g. text-generation, text-to-image). Leave blank for all.
searchQuery	string	""	Optional keyword or architecture filter (e.g. "qwen", "llama", "whisper").
sort	string	"downloads"	Metric to sort by: downloads, likes, lastModified, or createdAt.
maxItems	integer	100	Maximum number of records to retrieve.

Example Input JSON

{
  "resourceType": "models",
  "pipelineTag": "text-generation",
  "searchQuery": "llama",
  "sort": "downloads",
  "maxItems": 50
}

---

📤 Output Format

Each item pushed to the output dataset follows a structured schema:

{
  "id": "meta-llama/Llama-3.1-8B-Instruct",
  "item_type": "model",
  "author": "meta-llama",
  "repo_name": "Llama-3.1-8B-Instruct",
  "hf_url": "https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct",
  "pipeline_tag": "text-generation",
  "downloads_count": 8421090,
  "likes_count": 4820,
  "library_name": "transformers",
  "tags": ["transformers", "llama", "text-generation", "en", "license:llama3.1"],
  "created_at": "2024-07-23T12:00:00.000Z",
  "last_modified": "2024-08-20T14:30:00.000Z",
  "is_private": false,
  "is_gated": true
}

---

🚀 Running Locally

Step 1: Install Dependencies

$pip install -r requirements.txt

Step 2: Run the Actor

$python -m src.main

---

🐳 Docker Support

To build and run containerized:

docker build -t huggingface-scraper .
docker run -it huggingface-scraper