# Kaggle Datasets Scraper (`automation-lab/kaggle-dataset-catalog-export`) Actor

Search public Kaggle datasets and export normalized licenses, sizes, usability, download signals, versions, file manifests, and canonical URLs.

- **URL**: https://apify.com/automation-lab/kaggle-dataset-catalog-export.md
- **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community)
- **Categories:** Education
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.84 / 1,000 item extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Kaggle Datasets Scraper

Search public **Kaggle datasets** and export a normalized catalog for dataset discovery, governance, research sourcing, and recurring inventory refreshes. Each result can include identity, owner, license, byte size, popularity and usability signals, current version, version history, tags, file manifest, and a canonical Kaggle URL.

This Actor returns metadata only. It does not download dataset files, scrape models, competitions, notebooks, courses, or private resources.

### What can you do with this Kaggle datasets scraper?

- Find datasets by one or more keywords and deduplicate overlaps.
- Enrich known Kaggle dataset URLs or `owner/slug` references.
- Filter discovery and explicit URLs by exact owner or public tag.
- Rank discovery by hotness, votes, update time, activity, or publication time.
- Save file names and sizes without downloading potentially large data files.
- Capture version numbers, dates, status, and public version notes.
- Schedule the same input and compare datasets downstream to detect catalog changes.

### Who is it for?

**Data governance teams** can maintain source-linked inventories with licenses, versions, sizes, and file manifests.

**Data scientists and analysts** can shortlist relevant public datasets using download, vote, view, usability, and recency signals.

**Research and ML teams** can feed normalized records into notebooks, spreadsheets, warehouses, or review queues without manually copying Kaggle pages.

**Automation developers** can use the Apify API, webhooks, schedules, and MCP to connect catalog discovery to a larger workflow.

### Why use this Actor?

Kaggle exposes rich metadata across catalog, detail, and file endpoints. This Actor joins those surfaces into one stable record per dataset. It also:

1. accepts both discovery queries and canonical URLs;
2. deduplicates the same dataset across multiple queries;
3. applies owner and tag filters consistently;
4. validates response status, content type, and shape;
5. retries only temporary network, rate-limit, and server failures;
6. fails closed instead of returning a misleading empty success after an upstream error.

### What data does it extract?

| Field | Meaning |
| --- | --- |
| `datasetId`, `ref`, `url` | Stable Kaggle identity, `owner/slug`, and canonical source URL |
| `title`, `subtitle`, `description` | Public catalog text; optional text may be `null` |
| `ownerName`, `ownerRef`, `creatorName` | Dataset owner and creator identity exposed by Kaggle |
| `licenseName` | Public license label, when supplied |
| `totalBytes` | Dataset size in bytes, when supplied |
| `downloadCount`, `viewCount`, `voteCount` | Public discovery and popularity signals |
| `kernelCount`, `topicCount` | Public usage/community signals |
| `usabilityRating` | Kaggle usability score, when supplied |
| `currentVersionNumber`, `lastUpdated` | Current version and source update timestamp |
| `tags` | Public Kaggle dataset tags |
| `files` | File names, byte sizes, and creation dates; no file content |
| `versions` | Public version number, date, status, and notes |
| `matchedQueries` | Search terms or explicit-URL route that found the dataset |
| `scrapedAt` | UTC timestamp when the record was normalized |

### Getting started

1. Open the Actor input page.
2. Keep the prefilled `climate change` query or enter your own search terms.
3. Optionally add known Kaggle dataset URLs, an owner, or a tag.
4. Choose the sort order and enrichment options.
5. Set `maxItems` to the number of records you need.
6. Start the Actor and open the **Kaggle datasets** dataset view.
7. Export JSON, CSV, Excel, XML, RSS, or connect the dataset API.

### Input parameters

#### `searchQueries`

An array of keyword searches. The Actor processes each query and deduplicates records by Kaggle ref. Up to 20 queries are accepted.

#### `datasetUrls`

Known public dataset URLs or refs such as:

```text
https://www.kaggle.com/datasets/uciml/iris
uciml/iris
```

Only `kaggle.com/datasets/owner/slug` URLs are accepted.

#### `owner`

An exact Kaggle owner username. It can be used alone for owner catalog discovery or combined with searches and explicit URLs. Explicit URLs owned by another account are filtered out.

#### `tag`

A public dataset tag name such as `classification`. It can be used alone or as an additional filter. Because the tag is verified against detailed records, keep `includeDetails` enabled when using it.

#### `sortBy`

Choose `hottest`, `votes`, `updated`, `active`, or `published` for discovered records.

#### Detail switches

- `includeDetails` fetches descriptions, full tags, and version metadata.
- `includeFiles` fetches the manifest but never downloads file content.
- `includeVersions` includes version history when details are enabled.

All three default to `true` for complete governance records.

#### Limits and retries

`maxItems` accepts 1–1,000 normalized records. `maxRequestRetries` accepts 0–5 and applies only to temporary network failures, HTTP 429, and 5xx responses.

### Example: search for climate datasets

```json
{
  "searchQueries": ["climate change"],
  "sortBy": "votes",
  "maxItems": 5
}
```

This is useful for a one-time source shortlist with licenses, file manifests, versions, and popularity signals.

### Example: enrich a known dataset

```json
{
  "datasetUrls": ["https://www.kaggle.com/datasets/uciml/iris"],
  "includeDetails": true,
  "includeFiles": true,
  "includeVersions": true,
  "maxItems": 1
}
```

The result includes `Iris.csv`, `database.sqlite`, the CC0 license label, current version, version history, and the canonical source URL as currently exposed by Kaggle.

### Example output

```json
{
  "type": "dataset",
  "datasetId": 19,
  "ref": "uciml/iris",
  "url": "https://www.kaggle.com/datasets/uciml/iris",
  "title": "Iris Species",
  "ownerRef": "uciml",
  "licenseName": "CC0: Public Domain",
  "totalBytes": 15347,
  "downloadCount": 931707,
  "voteCount": 4910,
  "usabilityRating": 0.7941176,
  "currentVersionNumber": 2,
  "tags": ["biology"],
  "files": [
    { "name": "Iris.csv", "totalBytes": 5107, "creationDate": "2019-09-19T03:05:19.853Z" },
    { "name": "database.sqlite", "totalBytes": 10240, "creationDate": "2019-09-19T03:05:19.901Z" }
  ],
  "versions": [
    { "versionNumber": 2, "creationDate": "2016-09-27T07:38:05.44Z", "status": "Ready", "notes": "Republishing files so they're formally in our system" }
  ],
  "matchedQueries": ["explicit URL"],
  "scrapedAt": "2026-09-10T06:10:00.000Z"
}
```

Counts and timestamps change as Kaggle updates its catalog.

### How much does it cost to export Kaggle datasets?

The Actor uses pay-per-event pricing: a **$0.005 start fee** per run plus one tiered `item` charge for each dataset record saved. Files and version entries are included in the dataset item and have no separate event charge.

The **BRONZE item rate is $0.0064**. The six plan tiers are FREE $0.00736, BRONZE $0.0064, SILVER $0.004992, and GOLD/PLATINUM/DIAMOND $0.00384 per saved dataset. At BRONZE, the total is the start fee plus the item rate multiplied by saved datasets: 1, 5, 10, 25, and 100 results cost USD 0.0114, 0.037, 0.069, 0.165, and 0.645 respectively. Apify Console shows your applicable plan tier before a run. You are never charged an item event for a rejected, duplicate, filtered, or failed record.

### Recurring catalog inventory workflow

1. Configure a query, owner, or tag and save it as an Apify Task.
2. Schedule the Task daily, weekly, or monthly.
3. Send the finished dataset to a webhook, Make, Zapier, Google Sheets, or your warehouse.
4. Compare records by `ref`, `currentVersionNumber`, `lastUpdated`, `licenseName`, or file names.
5. Review newly found or changed datasets in your governance process.

The Actor produces snapshots. Change detection, alerts, and historical retention are performed by your downstream workflow.

### Export to a spreadsheet or pipeline

Use the dataset export menu for CSV or Excel. Nested `files` and `versions` are best preserved in JSON; spreadsheet tools may serialize these arrays into cells. For a warehouse, request dataset items through the API after the run succeeds and upsert on `ref`.

### API usage with cURL

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~kaggle-dataset-catalog-export/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchQueries":["renewable energy"],"sortBy":"updated","maxItems":10}'
```

To wait for completion and return dataset items, use the `run-sync-get-dataset-items` endpoint where suitable for your timeout and item volume.

### API usage with JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/kaggle-dataset-catalog-export').call({
  searchQueries: ['renewable energy'],
  sortBy: 'updated',
  maxItems: 10,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

### API usage with Python

```python
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/kaggle-dataset-catalog-export').call(run_input={
    'searchQueries': ['renewable energy'],
    'sortBy': 'updated',
    'maxItems': 10,
})
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)
```

### Use through MCP

Add the Apify MCP server to Claude Code:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/kaggle-dataset-catalog-export"
```

#### Claude Desktop

Use this MCP server entry in Claude Desktop's configuration:

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/kaggle-dataset-catalog-export"
    }
  }
}
```

#### Cursor

Add the same `mcpServers.apify.url` entry in Cursor's MCP settings.

#### VS Code

Add the same HTTP MCP server URL in your VS Code MCP configuration.

Example prompts:

- “Search Kaggle for five highly voted climate-change datasets and summarize their licenses and sizes.”
- “Export the metadata, version history, and file manifest for `uciml/iris`.”
- “Refresh a five-item classification dataset inventory ordered by update time.”

### Reliability and failure behavior

The Actor uses anonymous public Kaggle JSON endpoints. It checks for JSON responses and recognizable shapes. Temporary transport failures, rate limits, and server errors receive bounded exponential retries. Stable client errors, malformed inputs, non-JSON challenge pages, and unexpected response shapes fail the run with a diagnostic log.

There is no automatic residential proxy or browser fallback, so a retry cannot silently create expensive transfer or browser charges.

### Limits

- Public datasets only; private datasets and account-only metadata are unavailable.
- Dataset files are listed but not downloaded or parsed.
- Counts, usability ratings, tags, descriptions, and versions reflect Kaggle's current public response and may change.
- `maxItems` is a cap, not a promise that every query has that many matching datasets.
- A restrictive owner/tag combination can naturally return no records.
- Search relevance and sort order are controlled by Kaggle.
- Very large inventories make multiple detail and file requests and take longer.

### Responsible use and legality

Use only public information you are authorized to collect. Review Kaggle's terms, dataset-specific licenses, and any restrictions stated by dataset owners. A catalog license label is metadata, not legal advice and not permission to reuse dataset contents. Do not use this Actor to bypass access controls or to process personal data unlawfully. Apply your organization's retention, attribution, security, and governance requirements.

### Troubleshooting

#### Why did my run return no records?

Check spelling and remove overly restrictive owner or tag filters. Try the same query directly in Kaggle. A valid no-result search succeeds with an empty dataset.

#### Why did the run fail instead of returning an empty dataset?

The Actor distinguishes a real empty catalog result from an upstream error. Inspect logs for an HTTP status, content-type mismatch, timeout, or unexpected response shape. Retry later for a temporary Kaggle outage; do not raise retries indefinitely.

#### Why is a known URL filtered out?

The same owner and tag filters apply to explicit URLs. Remove the conflicting filter or provide a matching URL.

#### Why are `files` or `versions` empty?

Confirm `includeFiles`, `includeDetails`, and `includeVersions` are enabled. Kaggle may also omit metadata for some records.

### FAQ

#### Does this download Kaggle datasets as CSV?

No. It exports the **catalog metadata** itself as JSON, CSV, Excel, and other Apify dataset formats. It lists dataset files but does not download their contents. This keeps recurring discovery runs small and avoids republishing source data.

#### Does it include Kaggle models?

No. This Actor intentionally covers datasets only. Models, competitions, notebooks, courses, and profiles are outside its contract.

#### Can I combine queries and URLs?

Yes. The Actor deduplicates by Kaggle `owner/slug` ref and records which queries matched. The final result count still respects `maxItems`.

#### Can I schedule it?

Yes. Save the input as an Apify Task, add a schedule, and connect the resulting dataset to your preferred storage or automation tool.

#### Is an API credential required?

No Kaggle credential is required for the supported public catalog metadata. Your Apify token is needed only when invoking the Actor through Apify's API or client.

### Related Automation Lab Actors

- [arXiv Paper Search & Export](https://apify.com/automation-lab/arxiv-paper-search-export) for recurring public research-paper metadata searches.
- [Hugging Face Scraper](https://apify.com/automation-lab/huggingface-scraper) when your workflow genuinely needs Hugging Face catalog records rather than Kaggle datasets.

Choose this Actor when the source of record is Kaggle and you need normalized, source-linked dataset catalog metadata rather than file content.

# Actor input Schema

## `searchQueries` (type: `array`):

Keywords to search in the Kaggle datasets catalog. Results from multiple queries are deduplicated.

## `datasetUrls` (type: `array`):

Specific public datasets to enrich, as Kaggle URLs or owner/slug refs.

## `owner` (type: `string`):

Return datasets owned by this exact Kaggle username. Can be used alone or with search queries and URLs.

## `tag` (type: `string`):

Return datasets whose public tag name matches this value, for example classification.

## `sortBy` (type: `string`):

Kaggle catalog ranking used for discovered datasets.

## `includeDetails` (type: `boolean`):

Fetch each dataset detail record for full descriptions, tags, and version metadata.

## `includeFiles` (type: `boolean`):

Fetch file names, sizes, and creation dates. Dataset files are not downloaded.

## `includeVersions` (type: `boolean`):

Include public version numbers, dates, statuses, and notes when details are enabled.

## `maxItems` (type: `integer`):

Maximum number of normalized dataset records saved after filtering.

## `maxRequestRetries` (type: `integer`):

Retries for temporary network, rate-limit, and Kaggle server errors.

## Actor input object example

```json
{
  "searchQueries": [
    "climate change"
  ],
  "sortBy": "hottest",
  "includeDetails": true,
  "includeFiles": true,
  "includeVersions": true,
  "maxItems": 10,
  "maxRequestRetries": 2
}
```

# Actor output Schema

## `overview` (type: `string`):

Open normalized dataset records in the overview table.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQueries": [
        "climate change"
    ],
    "maxItems": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/kaggle-dataset-catalog-export").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "searchQueries": ["climate change"],
    "maxItems": 10,
}

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/kaggle-dataset-catalog-export").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQueries": [
    "climate change"
  ],
  "maxItems": 10
}' |
apify call automation-lab/kaggle-dataset-catalog-export --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,automation-lab/kaggle-dataset-catalog-export"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/mLYYtIW3xQyrSTvVW/builds/RMsHbhJtfTGGm8tVj/openapi.json