Kaggle ML Competition Explorer

Explore Kaggle competitions by topic, status, date range, task type and keywords — and get back rows classified into normalized ML categories.

Ask for "time-series competitions from 2024 to 2026" and the Actor issues 3 requests and downloads 39 competitions, not the full 750-competition corpus.

What it does

Kaggle has no way to ask for "the time-series competitions" or "NLP competitions that closed in 2023". This Actor:

1. Translates your filters into Kaggle's own server-side query language — its taxonomy tag ids, its list filter, and its full-text search — so only relevant competitions are ever downloaded.
2. Applies the rest locally, because Kaggle genuinely supports no date filtering and no sorting (both verified by probing the live endpoint).
3. Classifies every result deterministically into topics, task types, domain and modality. No LLM: same input, same output, always.

Supported topics: computer_vision · nlp · time_series · tabular · audio · recommendation · reinforcement_learning · multimodal · other

What Kaggle actually supports

Every field below was probed against the live endpoint. The Actor only ever sends fields verified to have an effect — sending an ignored field would falsely imply the filter was applied.

Capability	Supported?	How the Actor uses it
selector.listOption	✅ DEFAULT (750) / ACTIVE (21) / COMPLETED (729)	Status filter, pushed server-side
selector.tagIds	✅ but ANDed, not ORed	Topic filter — one request per tag, results merged
selector.searchQuery	✅ full-text, also matches tags	Keyword filter, pushed server-side
pageSize / pageToken	✅ max 100; token is an offset string	Pagination
Sorting	❌ every sortBy/sortOption placement and enum left the order unchanged	Applied locally
Date ranges	❌ dateFrom, startDate, deadlineAfter, minDeadline all silently ignored	Applied locally
categoryIds, top-level tagIds	❌ silently ignored	Not sent

Two behaviours worth knowing:

• tagIds is an intersection. tagIds: [image, tabular] returns the 3 competitions with both tags, not their union. A topic mapping to 8 tags therefore needs 8 requests, merged and deduplicated — which is why src/constants.ts stores minimal covering sets computed by greedy set-cover over the whole corpus.
• A query matching nothing returns {} — HTTP 200 with no keys at all. That is a legitimate empty result, not a broken response, and the Actor treats it as such.

The query planner

src/queryPlanner.ts turns your filters into the smallest set of requests that can still return every match:

Your input	Plan
No topics, no keywords	corpus — nothing to narrow with
Topic other	corpus — Kaggle has no tag for "no recognised topic"
Topics given	tag_narrowed — one query per tag × status
Keywords only	search_narrowed — one query per keyword × status
Topics + keywords	tag_narrowed with searchQuery folded into each query

Statuses collapse to the fewest requests: ["active","completed"] becomes one DEFAULT query, because DEFAULT (750) is exactly ACTIVE (21) + COMPLETED (729).

The corpus probe. Before running a narrowed plan, the Actor spends one 1-item request asking how big the corpus is. If it fits in a single page, fetching it whole beats any multi-query plan — the ACTIVE corpus is 21 competitions, so computer_vision + active costs 2 requests instead of 9. The probe asks for one item precisely so it does not download a page the narrowed plan is about to supersede. Every decision is logged:

Corpus probe: Kaggle reports 21 competitions for this status filter, which fits in
one page. Fetching it whole is cheaper than the 8-query tag_narrowed plan, so the
narrowed queries are skipped.

The full corpus is only ever fetched when the planner says so and logs why.

Measured efficiency

Real numbers from live runs (the previous CV-only version always fetched all 750 in 8 requests):

Query	Requests	Competitions downloaded	Rows
computer_vision + active	2	22	4
time_series + active/completed	3	39	10
nlp + completed	7	93	10
computer_vision + completed + 2020–2024	10	168	20
keyword satellite + completed	2	73	10
audio + active/completed	3	11	10

Input

Field	Type	Default	Description
topics	string[]	[]	Topics to search for. Pushed server-side as Kaggle tag ids. Empty (the default) means no topic restriction.
statuses	string[]	["active"]	active / completed / unknown. Pass [] for all.
taskTypes	string[]	[]	Optional normalized task types; applied locally.
dateField	string	deadline	deadline or launched.
dateFrom	string | null	null	Inclusive lower bound, e.g. "2024-01-01".
dateTo	string | null	null	Inclusive upper bound.
keywords	string[]	[]	Free-text terms, pushed server-side then re-checked locally.
sortBy	string	relevance	relevance, deadline_asc, deadline_desc, newest, oldest, prize_desc.
maxItems	integer | null	50	Row limit; null for no limit.
proxyConfiguration	object	—	Optional Apify Proxy settings.

topics defaults to [] — no topic restriction. Omitting it returns competitions of every topic, each still classified. A default run stays cheap because statuses defaults to ["active"]: that is the 21-competition active list in one request, not a 750-competition scan. Widen it deliberately with "statuses": [].

Example input

Everything currently open, no topic restriction — this is what the defaults do:

{
  "topics": [],
  "statuses": ["active"],
  "maxItems": 50
}

Narrowed to one topic and a date range:

{
  "topics": ["time_series"],
  "statuses": ["active", "completed"],
  "dateField": "deadline",
  "dateFrom": "2024-01-01",
  "dateTo": "2026-12-31",
  "sortBy": "deadline_desc",
  "maxItems": 25
}

Output

{
  "title": "Biohub - Cell Tracking During Development",
  "url": "https://www.kaggle.com/competitions/biohub-cell-tracking-during-development",
  "description": "Detect and track zebrafish cells through 3D space and time",
  "host": "Biohub",
  "deadline": "2026-09-29T23:59:00.000Z",
  "launchedAt": "2026-06-29T19:01:13.613Z",
  "prize": "$60,000",
  "tags": [
    "object detection",
    "video",
    "image",
    "computer vision",
    "biology"
  ],
  "status": "active",
  "topics": [
    "computer_vision"
  ],
  "taskTypes": [
    "object_detection",
    "object_tracking",
    "video_understanding",
    "medical_imaging"
  ],
  "domain": "medical",
  "modality": [
    "image",
    "video",
    "3d"
  ],
  "relevanceKeywords": [
    "3d",
    "cell",
    "cells",
    "data type > image",
    "data type > video data",
    "detect",
    "subject > earth and nature > biology",
    "task > object-detection",
    "technique > computer vision",
    "through 3d space and time",
    "track",
    "tracking",
    "zebrafish"
  ],
  "source": "kaggle"
}

Field	Notes
title, url	Always present.
description	Kaggle's one-line brief. The full overview is not available unauthenticated.
host, prize	Prize is "$50,000" or a label like "Knowledge" / "Swag".
deadline, launchedAt	ISO-8601 UTC. launchedAt comes from Kaggle's dateEnabled.
tags	Kaggle's own taxonomy tags; empty for the 182 untagged competitions.
status	active / completed / unknown.
topics	Normalized ML topics. Empty means "no recognised topic".
taskTypes	One or more; falls back to ["other_ml"].
domain	Single best-supported domain, or "other".
modality	May be empty when there is no evidence.
relevanceKeywords	The terms and tags that drove the decision — use these to audit a classification.

Classification logic

src/classification.ts combines two evidence sources:

1. Kaggle's taxonomy — high precision, but sparse: 182 of 750 competitions have no tags, and Global Wheat Detection carries only data type > image with no object-detection tag.
2. Title and description text rules — these supply the recall the tags lack.

The rule that prevents cross-domain false positives: generic task words only assign a topic when that topic's modality anchor is also present. "classification" means nothing on its own; "image classification", "text classification" and "tabular classification" are three different topics.

Task rules are scoped to their topic, so an image competition can never pick up sentiment_analysis from a stray word.

Guards added after real misclassifications found in live runs:

• visual / vision need corroboration — otherwise "Harvard Business Review 'Vision Statement'" and "predict visual stimuli from MEG recordings" both read as computer vision.
• A bare recording is not acoustic — MEG recordings are not audio.
• video game, in-depth and brand image are stripped before matching.
• tabular is dropped when a richer modality is present and nothing independently says "tabular". Kaggle tags MNIST data type > tabular because the pixels ship as CSV; reporting it as a tabular competition would mislead. Time series is exempt — a forecasting competition over a table genuinely is both.

When evidence is thin the Actor does not guess: taskTypes falls back to ["other_ml"], domain to "other", modality to an empty list.

Migrating from v1 (the CV-only scraper)

Old inputs keep working; the Actor logs every translation.

v1	v2
onlyActive: true	statuses: ["active"] — translated automatically
onlyActive: false	statuses: null (all) — translated automatically
taskTypes: ["other_cv"]	taskTypes: ["other_ml"] — translated automatically
keywords	unchanged
output isComputerVision: true	removed — use topics (contains computer_vision)
—	new output field launchedAt

An explicit statuses always wins over a legacy onlyActive.

Blocking behaviour

The Actor does not bypass CAPTCHAs, login walls or access controls. It detects and reports them: 403/429/5xx and challenge/login pages are retried with backoff (rotating proxy IP if configured); 401 and a changed response shape fail immediately with an actionable message. If every request fails, the Actor fails loudly rather than finishing with an empty dataset.

Limitations

• description is Kaggle's one-line brief. The full overview is not exposed on any unauthenticated endpoint, and competition pages are client-rendered with an empty <body>.
• 182 of 750 competitions carry no tags, so those rely on a title plus one sentence — and a tag-narrowed plan cannot reach them at all. Topic queries are limited to what Kaggle has tagged; leaving topics empty (the default), optionally with keywords, searches more broadly.
• keywords reach a larger index than topics do. Kaggle's search covers community competitions outside the curated 750-competition list, so keyword results can include lower-quality competitions that a tag query would never return.
• Known classification false positives, found in full-corpus audits: Photo Quality Prediction (tabular metadata about photos, reads as CV) and Harvard Business Review 'Vision Statement' (reads as NLP via "Review").
• domain is single-valued; a satellite-based agriculture competition gets whichever rule scores higher.
• The discovery endpoint is Kaggle-internal. Public and unauthenticated today, but undocumented and unversioned. Everything Kaggle-specific is centralized in src/constants.ts and src/discovery/competitions.ts.
• English only.

Local development

npm install
npm run build          # tsc -> dist/
npm test               # 371 deterministic tests, no network
npm run test:live      # live scenarios against real Kaggle
npm start              # run locally via tsx
npm run typecheck

Local runs read storage/key_value_stores/default/INPUT.json:

mkdir -p storage/key_value_stores/default
cp examples/input.json storage/key_value_stores/default/INPUT.json
npm start

Production container (same base image Apify Cloud uses):

docker build -t kaggle-ml-explorer .
docker run --rm \
  -e CRAWLEE_STORAGE_DIR=/tmp/storage \
  -v "$(pwd)/my-storage:/tmp/storage" \
  kaggle-ml-explorer

Put INPUT.json in my-storage/key_value_stores/default/. On Git Bash prefix the command with MSYS_NO_PATHCONV=1, otherwise the shell rewrites /tmp/storage into a Windows path and the Actor silently falls back to defaults.

Deploying to Apify

npm install -g apify-cli
apify login
apify push <actorId>                          # builds on Apify Cloud
apify call <actorId> --input-file examples/input.json