GitHub Trending Repos Tracker

Scrape GitHub Trending and get structured, dataset-ready records for every trending repository — ranks, star counts, fork counts, stars gained today, contributor avatars, programming language, and description — all filtered exactly the way you need.

---

What It Does

GitHub Trending is a server-rendered HTML page; there is no official API. This actor:

1. Builds the correct GitHub Trending URL from your input parameters (language, date range, spoken language).
2. Fetches the page with a polite User-Agent header, retrying up to 3 times with exponential backoff on transient failures.
3. Parses the HTML using BeautifulSoup4 to extract structured data from each repository card.
4. Pushes each repository as a structured record to the Apify default dataset.

No authentication or API keys are required.

---

Input Parameters

Parameter	Type	Default	Description
language	string	""	Programming language slug (e.g. python, javascript, rust). Empty = all languages.
date_range	enum	daily	Time window: daily, weekly, or monthly.
spoken_language	string	""	ISO 639-1 code for the human language of the repo (e.g. en, zh). Empty = any.
max_repos	integer	25	Max repositories to return. GitHub shows at most 25 per page.

Example Input

Trending Python repos this week, English README only:

{
  "language": "python",
  "date_range": "weekly",
  "spoken_language": "en",
  "max_repos": 25
}

All-languages daily trending (default):

{
  "language": "",
  "date_range": "daily",
  "spoken_language": "",
  "max_repos": 25
}

---

Output Format

Each record in the dataset represents one trending repository:

{
  "rank": 1,
  "full_name": "owner/repository-name",
  "url": "https://github.com/owner/repository-name",
  "description": "A short description of what the repository does.",
  "language": "Python",
  "stars": 48312,
  "forks": 3201,
  "stars_today": 872,
  "built_by": [
    {
      "username": "alice",
      "avatar_url": "https://avatars.githubusercontent.com/u/12345?s=40&v=4"
    },
    {
      "username": "bob",
      "avatar_url": "https://avatars.githubusercontent.com/u/67890?s=40&v=4"
    }
  ],
  "scraped_language": "python",
  "scraped_date_range": "weekly",
  "scraped_spoken_language": "en",
  "source_url": "https://github.com/trending/python?since=weekly&spoken_language_code=en"
}

Field Reference

Field	Type	Description
rank	integer	Position on the trending page (1-based).
full_name	string	owner/repo slug, unique identifier on GitHub.
url	string	Full GitHub URL to the repository.
description	string | null	Repository description from the GitHub card.
language	string | null	Primary programming language as detected by GitHub.
stars	integer | null	Total cumulative star count.
forks	integer | null	Total cumulative fork count.
stars_today	integer | null	Stars gained during the selected date_range period.
built_by	array	List of contributor objects with username and avatar_url.
scraped_language	string | null	The language input used for this scrape run.
scraped_date_range	string	The date_range input used for this scrape run.
scraped_spoken_language	string | null	The spoken_language input used for this scrape run.
source_url	string	The exact GitHub Trending URL that was scraped.

---

Language Slugs

Use GitHub's own slug format, which is the language name lowercased with spaces replaced by hyphens. Common examples:

Language	Slug
Python	python
JavaScript	javascript
TypeScript	typescript
Rust	rust
Go	go
C++	c++
C#	c%23
Ruby	ruby
Java	java
Swift	swift
Kotlin	kotlin
Shell	shell
Jupyter Notebook	jupyter-notebook

You can find any language's slug by browsing to github.com/trending, selecting the language from the dropdown, and reading the URL path segment.

---

Spoken Language Codes

Common ISO 639-1 codes supported by GitHub Trending:

Language	Code
English	en
Chinese (Simplified)	zh
Japanese	ja
Korean	ko
Spanish	es
French	fr
German	de
Portuguese	pt
Russian	ru

---

Use Cases

Technology Radar

Run the actor daily across multiple languages to track which projects are gaining traction fastest. Combine stars_today with total stars to separate genuinely new entrants from established projects getting a burst of attention.

Newsletter & Content Curation

Schedule a daily or weekly run to automatically populate a "what's trending on GitHub" section in a developer newsletter. Filter by language to stay on-topic for a specific audience.

Competitive Intelligence

Monitor trending repos in a domain (e.g. language=python, date_range=weekly) to catch newly open-sourced tools that compete with or complement your product before they go viral.

Recruiting & Ecosystem Mapping

The built_by contributor list surfaces active GitHub users working on high-momentum projects. Cross-reference with LinkedIn or other sources to identify talent in a specific language ecosystem.

Data Analysis & Trend Forecasting

Export daily snapshots to a time-series dataset (BigQuery, DuckDB, pandas) and analyze how quickly repositories rise and fall on the trending page, or how trending topics shift over months.

---

Error Handling

• Transient network errors (timeouts, connection resets): retried up to 3 times with 2-second exponential backoff before the actor fails.
• HTTP 4xx errors (except 429 Too Many Requests): logged and treated as permanent failures — the actor will call Actor.fail() with a descriptive message.
• GitHub markup changes: if the HTML parser finds no article.Box-row elements, it logs a warning and exits cleanly rather than pushing empty or malformed data.
• Invalid inputs: date_range values outside daily/weekly/monthly cause an immediate graceful failure with a descriptive error message.

---

Rate Limiting & Politeness

GitHub does not publish a rate limit for the public trending page. The actor:

• Sends a single HTTP request per run (the trending page is one URL).
• Uses a descriptive User-Agent string that identifies the actor.
• Does not paginate or crawl linked repositories.

The total runtime for a single run is typically under 5 seconds.

---

Technical Details

• Language: Python 3.12
• HTTP client: httpx (async)
• HTML parser: BeautifulSoup4 with Python's built-in html.parser
• Apify SDK: apify >= 2.0.0
• Docker base image: apify/actor-python:3.12
• Concurrency: single async request (GitHub Trending is one page)

Project Structure

github-trending/
├── .actor/
│   ├── actor.json          # Actor metadata and dataset view configuration
│   └── INPUT_SCHEMA.json   # Input parameter schema for the Apify UI
├── src/
│   ├── __init__.py
│   └── main.py             # Actor entry point, HTTP fetch, HTML parsing
├── Dockerfile
├── requirements.txt
└── README.md