# Kaggle AI, ML & CV Competition Explorer (`siromer/kaggle-ai-ml-cv-competition-explorer`) Actor

Explore Kaggle competitions by topic, status, date range, task type, and keywords across Computer Vision, NLP, Time Series, Tabular, Audio, and more.

- **URL**: https://apify.com/siromer/kaggle-ai-ml-cv-competition-explorer.md
- **Developed by:** [Ömer Günaydın](https://apify.com/siromer) (community)
- **Categories:**
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Kaggle ML Competition Explorer

Explore Kaggle competitions by **topic, status, date range, task type and keywords** — and get back rows classified into normalized ML categories.

Ask for *"time-series competitions from 2024 to 2026"* and the Actor issues **3 requests and downloads 39 competitions**, not the full 750-competition corpus.

### What it does

Kaggle has no way to ask for "the time-series competitions" or "NLP competitions that closed in 2023". This Actor:

1. **Translates your filters into Kaggle's own server-side query language** — its taxonomy tag ids, its list filter, and its full-text search — so only relevant competitions are ever downloaded.
2. **Applies the rest locally**, because Kaggle genuinely supports no date filtering and no sorting (both verified by probing the live endpoint).
3. **Classifies every result deterministically** into topics, task types, domain and modality. No LLM: same input, same output, always.

Supported topics: `computer_vision` · `nlp` · `time_series` · `tabular` · `audio` · `recommendation` · `reinforcement_learning` · `multimodal` · `other`

### What Kaggle actually supports

Every field below was probed against the live endpoint. The Actor **only ever sends fields verified to have an effect** — sending an ignored field would falsely imply the filter was applied.

| Capability | Supported? | How the Actor uses it |
|---|---|---|
| `selector.listOption` | ✅ `DEFAULT` (750) / `ACTIVE` (21) / `COMPLETED` (729) | Status filter, pushed server-side |
| `selector.tagIds` | ✅ but **ANDed**, not ORed | Topic filter — one request per tag, results merged |
| `selector.searchQuery` | ✅ full-text, also matches tags | Keyword filter, pushed server-side |
| `pageSize` / `pageToken` | ✅ max 100; token is an offset string | Pagination |
| **Sorting** | ❌ every `sortBy`/`sortOption` placement and enum left the order unchanged | Applied **locally** |
| **Date ranges** | ❌ `dateFrom`, `startDate`, `deadlineAfter`, `minDeadline` all silently ignored | Applied **locally** |
| `categoryIds`, top-level `tagIds` | ❌ silently ignored | Not sent |

Two behaviours worth knowing:

- **`tagIds` is an intersection.** `tagIds: [image, tabular]` returns the 3 competitions with *both* tags, not their union. A topic mapping to 8 tags therefore needs 8 requests, merged and deduplicated — which is why [`constants.ts`](src/constants.ts) stores *minimal covering sets* computed by greedy set-cover over the whole corpus.
- **A query matching nothing returns `{}`** — HTTP 200 with no keys at all. That is a legitimate empty result, not a broken response, and the Actor treats it as such.

### The query planner

[`src/queryPlanner.ts`](src/queryPlanner.ts) turns your filters into the smallest set of requests that can still return every match:

| Your input | Plan |
|---|---|
| No topics, no keywords | `corpus` — nothing to narrow with |
| Topic `other` | `corpus` — Kaggle has no tag for "no recognised topic" |
| Topics given | `tag_narrowed` — one query per tag × status |
| Keywords only | `search_narrowed` — one query per keyword × status |
| Topics + keywords | `tag_narrowed` with `searchQuery` folded into each query |

Statuses collapse to the fewest requests: `["active","completed"]` becomes **one** `DEFAULT` query, because `DEFAULT` (750) is exactly `ACTIVE` (21) + `COMPLETED` (729).

**The corpus probe.** Before running a narrowed plan, the Actor spends one 1-item request asking how big the corpus is. If it fits in a single page, fetching it whole beats any multi-query plan — the ACTIVE corpus is 21 competitions, so `computer_vision + active` costs **2 requests instead of 9**. The probe asks for one item precisely so it does not download a page the narrowed plan is about to supersede. Every decision is logged:

```
Corpus probe: Kaggle reports 21 competitions for this status filter, which fits in
one page. Fetching it whole is cheaper than the 8-query tag_narrowed plan, so the
narrowed queries are skipped.
```

**The full corpus is only ever fetched when the planner says so and logs why.**

### Measured efficiency

Real numbers from live runs (the previous CV-only version always fetched all 750 in 8 requests):

| Query | Requests | Competitions downloaded | Rows |
|---|---|---|---|
| `computer_vision` + active | 2 | 22 | 4 |
| `time_series` + active/completed | 3 | 39 | 10 |
| `nlp` + completed | 7 | 93 | 10 |
| `computer_vision` + completed + 2020–2024 | 10 | 168 | 20 |
| keyword `satellite` + completed | 2 | 73 | 10 |
| `audio` + active/completed | 3 | **11** | 10 |

### Input

| Field | Type | Default | Description |
|---|---|---|---|
| `topics` | `string[]` | `[]` | Topics to search for. Pushed server-side as Kaggle tag ids. **Empty (the default) means no topic restriction.** |
| `statuses` | `string[]` | `["active"]` | `active` / `completed` / `unknown`. Pass `[]` for all. |
| `taskTypes` | `string[]` | `[]` | Optional normalized task types; applied locally. |
| `dateField` | `string` | `deadline` | `deadline` or `launched`. |
| `dateFrom` | `string \| null` | `null` | Inclusive lower bound, e.g. `"2024-01-01"`. |
| `dateTo` | `string \| null` | `null` | Inclusive upper bound. |
| `keywords` | `string[]` | `[]` | Free-text terms, pushed server-side then re-checked locally. |
| `sortBy` | `string` | `relevance` | `relevance`, `deadline_asc`, `deadline_desc`, `newest`, `oldest`, `prize_desc`. |
| `maxItems` | `integer \| null` | `50` | Row limit; `null` for no limit. |
| `proxyConfiguration` | `object` | — | Optional Apify Proxy settings. |

> **`topics` defaults to `[]` — no topic restriction.** Omitting it returns competitions of every topic, each still classified. A default run stays cheap because `statuses` defaults to `["active"]`: that is the 21-competition active list in one request, not a 750-competition scan. Widen it deliberately with `"statuses": []`.

#### Example input

Everything currently open, no topic restriction — this is what the defaults do:

```json
{
  "topics": [],
  "statuses": ["active"],
  "maxItems": 50
}
```

Narrowed to one topic and a date range:

```json
{
  "topics": ["time_series"],
  "statuses": ["active", "completed"],
  "dateField": "deadline",
  "dateFrom": "2024-01-01",
  "dateTo": "2026-12-31",
  "sortBy": "deadline_desc",
  "maxItems": 25
}
```

### Output

```json
{
  "title": "Biohub - Cell Tracking During Development",
  "url": "https://www.kaggle.com/competitions/biohub-cell-tracking-during-development",
  "description": "Detect and track zebrafish cells through 3D space and time",
  "host": "Biohub",
  "deadline": "2026-09-29T23:59:00.000Z",
  "launchedAt": "2026-06-29T19:01:13.613Z",
  "prize": "$60,000",
  "tags": [
    "object detection",
    "video",
    "image",
    "computer vision",
    "biology"
  ],
  "status": "active",
  "topics": [
    "computer_vision"
  ],
  "taskTypes": [
    "object_detection",
    "object_tracking",
    "video_understanding",
    "medical_imaging"
  ],
  "domain": "medical",
  "modality": [
    "image",
    "video",
    "3d"
  ],
  "relevanceKeywords": [
    "3d",
    "cell",
    "cells",
    "data type > image",
    "data type > video data",
    "detect",
    "subject > earth and nature > biology",
    "task > object-detection",
    "technique > computer vision",
    "through 3d space and time",
    "track",
    "tracking",
    "zebrafish"
  ],
  "source": "kaggle"
}
```

| Field | Notes |
|---|---|
| `title`, `url` | Always present. |
| `description` | Kaggle's one-line brief. The full overview is not available unauthenticated. |
| `host`, `prize` | Prize is `"$50,000"` or a label like `"Knowledge"` / `"Swag"`. |
| `deadline`, `launchedAt` | ISO-8601 UTC. `launchedAt` comes from Kaggle's `dateEnabled`. |
| `tags` | Kaggle's own taxonomy tags; empty for the 182 untagged competitions. |
| `status` | `active` / `completed` / `unknown`. |
| `topics` | Normalized ML topics. Empty means "no recognised topic". |
| `taskTypes` | One or more; falls back to `["other_ml"]`. |
| `domain` | Single best-supported domain, or `"other"`. |
| `modality` | May be empty when there is no evidence. |
| `relevanceKeywords` | The terms and tags that drove the decision — use these to audit a classification. |

### Classification logic

[`src/classification.ts`](src/classification.ts) combines two evidence sources:

1. **Kaggle's taxonomy** — high precision, but sparse: 182 of 750 competitions have no tags, and *Global Wheat Detection* carries only `data type > image` with no object-detection tag.
2. **Title and description text rules** — these supply the recall the tags lack.

The rule that prevents cross-domain false positives: **generic task words only assign a topic when that topic's modality anchor is also present.** "classification" means nothing on its own; "image classification", "text classification" and "tabular classification" are three different topics.

Task rules are **scoped to their topic**, so an image competition can never pick up `sentiment_analysis` from a stray word.

Guards added after real misclassifications found in live runs:

- `visual` / `vision` need corroboration — otherwise *"Harvard Business Review 'Vision Statement'"* and *"predict visual stimuli from MEG recordings"* both read as computer vision.
- A bare `recording` is not acoustic — *MEG recordings* are not audio.
- `video game`, `in-depth` and `brand image` are stripped before matching.
- **`tabular` is dropped when a richer modality is present** and nothing independently says "tabular". Kaggle tags MNIST `data type > tabular` because the pixels ship as CSV; reporting it as a tabular competition would mislead. Time series is exempt — a forecasting competition over a table genuinely is both.

When evidence is thin the Actor does not guess: `taskTypes` falls back to `["other_ml"]`, `domain` to `"other"`, `modality` to an empty list.

### Migrating from v1 (the CV-only scraper)

Old inputs keep working; the Actor logs every translation.

| v1 | v2 |
|---|---|
| `onlyActive: true` | `statuses: ["active"]` — translated automatically |
| `onlyActive: false` | `statuses: null` (all) — translated automatically |
| `taskTypes: ["other_cv"]` | `taskTypes: ["other_ml"]` — translated automatically |
| `keywords` | unchanged |
| **output** `isComputerVision: true` | **removed** — use `topics` (contains `computer_vision`) |
| — | **new** output field `launchedAt` |

An explicit `statuses` always wins over a legacy `onlyActive`.

### Blocking behaviour

The Actor **does not bypass CAPTCHAs, login walls or access controls.** It detects and reports them: 403/429/5xx and challenge/login pages are retried with backoff (rotating proxy IP if configured); 401 and a changed response shape fail immediately with an actionable message. If every request fails, the Actor fails loudly rather than finishing with an empty dataset.

### Limitations

- **`description` is Kaggle's one-line brief.** The full overview is not exposed on any unauthenticated endpoint, and competition pages are client-rendered with an empty `<body>`.
- **182 of 750 competitions carry no tags**, so those rely on a title plus one sentence — and a tag-narrowed plan cannot reach them at all. Topic queries are limited to what Kaggle has tagged; leaving `topics` empty (the default), optionally with `keywords`, searches more broadly.
- **`keywords` reach a larger index than topics do.** Kaggle's search covers community competitions outside the curated 750-competition list, so keyword results can include lower-quality competitions that a tag query would never return.
- **Known classification false positives**, found in full-corpus audits: *Photo Quality Prediction* (tabular metadata about photos, reads as CV) and *Harvard Business Review 'Vision Statement'* (reads as NLP via "Review").
- **`domain` is single-valued**; a satellite-based agriculture competition gets whichever rule scores higher.
- **The discovery endpoint is Kaggle-internal.** Public and unauthenticated today, but undocumented and unversioned. Everything Kaggle-specific is centralized in [`constants.ts`](src/constants.ts) and [`discovery/competitions.ts`](src/discovery/competitions.ts).
- **English only.**

### Local development

```bash
npm install
npm run build          # tsc -> dist/
npm test               # 371 deterministic tests, no network
npm run test:live      # live scenarios against real Kaggle
npm start              # run locally via tsx
npm run typecheck
```

Local runs read `storage/key_value_stores/default/INPUT.json`:

```bash
mkdir -p storage/key_value_stores/default
cp examples/input.json storage/key_value_stores/default/INPUT.json
npm start
```

Production container (same base image Apify Cloud uses):

```bash
docker build -t kaggle-ml-explorer .
docker run --rm \
  -e CRAWLEE_STORAGE_DIR=/tmp/storage \
  -v "$(pwd)/my-storage:/tmp/storage" \
  kaggle-ml-explorer
```

Put `INPUT.json` in `my-storage/key_value_stores/default/`. On Git Bash prefix the command with `MSYS_NO_PATHCONV=1`, otherwise the shell rewrites `/tmp/storage` into a Windows path and the Actor silently falls back to defaults.

### Deploying to Apify

```bash
npm install -g apify-cli
apify login
apify push <actorId>                          # builds on Apify Cloud
apify call <actorId> --input-file examples/input.json
```

# Actor input Schema

## `topics` (type: `array`):

Machine-learning topics to search for. Each topic is translated into Kaggle's own taxonomy tag ids and pushed server-side, so only relevant competitions are downloaded. Leave empty (the default) for no topic restriction - every competition is returned and classified.

## `statuses` (type: `array`):

Which competition statuses to return. "active" and "completed" are pushed server-side via Kaggle's list filter. "unknown" (no reliable deadline) can only be found locally and forces a wider scan.

## `taskTypes` (type: `array`):

Optional normalized task types. A competition is returned when it matches at least one. Applied locally, after Kaggle-side narrowing. Leave empty to include all task types.

## `dateField` (type: `string`):

Which date the range below applies to. "deadline" is the competition deadline; "launched" is Kaggle's launch timestamp (its `dateEnabled` field).

## `dateFrom` (type: `string`):

Inclusive lower bound, e.g. "2024-01-01". Kaggle supports no server-side date filtering, so this is applied locally after narrowing. Leave empty for no lower bound.

## `dateTo` (type: `string`):

Inclusive upper bound, e.g. "2026-12-31". Applied locally. Leave empty for no upper bound.

## `keywords` (type: `array`):

Optional free-text terms, e.g. "satellite", "medical". Pushed server-side via Kaggle's search, then re-checked locally. Note that Kaggle's search index also covers community competitions outside the main public list.

## `sortBy` (type: `string`):

Kaggle supports no server-side sorting, so ordering is applied locally after retrieval. "relevance" keeps Kaggle's own ordering and allows the run to stop early once maxItems is reached.

## `maxItems` (type: `integer`):

Maximum number of dataset rows to produce. Leave empty for no artificial limit.

## `proxyConfiguration` (type: `object`):

Optional Apify Proxy settings. Kaggle's public endpoint currently answers direct requests, but a proxy helps if you hit rate limits or a regional block.

## Actor input object example

```json
{
  "topics": [],
  "statuses": [
    "active"
  ],
  "taskTypes": [],
  "dateField": "deadline",
  "keywords": [],
  "sortBy": "relevance",
  "maxItems": 50,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `competitions` (type: `string`):

The default dataset produced by this run: one row per matching Kaggle competition, with topics, taskTypes, domain, modality, status, deadline, prize and the source URL. Field definitions and the Console table view come from .actor/dataset\_schema.json.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "statuses": [
        "active"
    ],
    "maxItems": 50
};

// Run the Actor and wait for it to finish
const run = await client.actor("siromer/kaggle-ai-ml-cv-competition-explorer").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "statuses": ["active"],
    "maxItems": 50,
}

# Run the Actor and wait for it to finish
run = client.actor("siromer/kaggle-ai-ml-cv-competition-explorer").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "statuses": [
    "active"
  ],
  "maxItems": 50
}' |
apify call siromer/kaggle-ai-ml-cv-competition-explorer --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,siromer/kaggle-ai-ml-cv-competition-explorer"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/wrrLqa1HH5NsJyQLU/builds/g4fPkJvxmnvGr7yVz/openapi.json