Kaggle Datasets Scraper

Search public Kaggle datasets and export a normalized catalog for dataset discovery, governance, research sourcing, and recurring inventory refreshes. Each result can include identity, owner, license, byte size, popularity and usability signals, current version, version history, tags, file manifest, and a canonical Kaggle URL.

This Actor returns metadata only. It does not download dataset files, scrape models, competitions, notebooks, courses, or private resources.

What can you do with this Kaggle datasets scraper?

• Find datasets by one or more keywords and deduplicate overlaps.
• Enrich known Kaggle dataset URLs or owner/slug references.
• Filter discovery and explicit URLs by exact owner or public tag.
• Rank discovery by hotness, votes, update time, activity, or publication time.
• Save file names and sizes without downloading potentially large data files.
• Capture version numbers, dates, status, and public version notes.
• Schedule the same input and compare datasets downstream to detect catalog changes.

Who is it for?

Data governance teams can maintain source-linked inventories with licenses, versions, sizes, and file manifests.

Data scientists and analysts can shortlist relevant public datasets using download, vote, view, usability, and recency signals.

Research and ML teams can feed normalized records into notebooks, spreadsheets, warehouses, or review queues without manually copying Kaggle pages.

Automation developers can use the Apify API, webhooks, schedules, and MCP to connect catalog discovery to a larger workflow.

Why use this Actor?

Kaggle exposes rich metadata across catalog, detail, and file endpoints. This Actor joins those surfaces into one stable record per dataset. It also:

1. accepts both discovery queries and canonical URLs;
2. deduplicates the same dataset across multiple queries;
3. applies owner and tag filters consistently;
4. validates response status, content type, and shape;
5. retries only temporary network, rate-limit, and server failures;
6. fails closed instead of returning a misleading empty success after an upstream error.

What data does it extract?

Field	Meaning
datasetId, ref, url	Stable Kaggle identity, owner/slug, and canonical source URL
title, subtitle, description	Public catalog text; optional text may be null
ownerName, ownerRef, creatorName	Dataset owner and creator identity exposed by Kaggle
licenseName	Public license label, when supplied
totalBytes	Dataset size in bytes, when supplied
downloadCount, viewCount, voteCount	Public discovery and popularity signals
kernelCount, topicCount	Public usage/community signals
usabilityRating	Kaggle usability score, when supplied
currentVersionNumber, lastUpdated	Current version and source update timestamp
tags	Public Kaggle dataset tags
files	File names, byte sizes, and creation dates; no file content
versions	Public version number, date, status, and notes
matchedQueries	Search terms or explicit-URL route that found the dataset
scrapedAt	UTC timestamp when the record was normalized

Getting started

1. Open the Actor input page.
2. Keep the prefilled climate change query or enter your own search terms.
3. Optionally add known Kaggle dataset URLs, an owner, or a tag.
4. Choose the sort order and enrichment options.
5. Set maxItems to the number of records you need.
6. Start the Actor and open the Kaggle datasets dataset view.
7. Export JSON, CSV, Excel, XML, RSS, or connect the dataset API.

Input parameters

searchQueries

An array of keyword searches. The Actor processes each query and deduplicates records by Kaggle ref. Up to 20 queries are accepted.

datasetUrls

Known public dataset URLs or refs such as:

https://www.kaggle.com/datasets/uciml/iris
uciml/iris

Only kaggle.com/datasets/owner/slug URLs are accepted.

owner

An exact Kaggle owner username. It can be used alone for owner catalog discovery or combined with searches and explicit URLs. Explicit URLs owned by another account are filtered out.

tag

A public dataset tag name such as classification. It can be used alone or as an additional filter. Because the tag is verified against detailed records, keep includeDetails enabled when using it.

sortBy

Choose hottest, votes, updated, active, or published for discovered records.

Detail switches

• includeDetails fetches descriptions, full tags, and version metadata.
• includeFiles fetches the manifest but never downloads file content.
• includeVersions includes version history when details are enabled.

All three default to true for complete governance records.

Limits and retries

maxItems accepts 1–1,000 normalized records. maxRequestRetries accepts 0–5 and applies only to temporary network failures, HTTP 429, and 5xx responses.

Example: search for climate datasets

{
  "searchQueries": ["climate change"],
  "sortBy": "votes",
  "maxItems": 5
}

This is useful for a one-time source shortlist with licenses, file manifests, versions, and popularity signals.

Example: enrich a known dataset

{
  "datasetUrls": ["https://www.kaggle.com/datasets/uciml/iris"],
  "includeDetails": true,
  "includeFiles": true,
  "includeVersions": true,
  "maxItems": 1
}

The result includes Iris.csv, database.sqlite, the CC0 license label, current version, version history, and the canonical source URL as currently exposed by Kaggle.

Example output

{
  "type": "dataset",
  "datasetId": 19,
  "ref": "uciml/iris",
  "url": "https://www.kaggle.com/datasets/uciml/iris",
  "title": "Iris Species",
  "ownerRef": "uciml",
  "licenseName": "CC0: Public Domain",
  "totalBytes": 15347,
  "downloadCount": 931707,
  "voteCount": 4910,
  "usabilityRating": 0.7941176,
  "currentVersionNumber": 2,
  "tags": ["biology"],
  "files": [
    { "name": "Iris.csv", "totalBytes": 5107, "creationDate": "2019-09-19T03:05:19.853Z" },
    { "name": "database.sqlite", "totalBytes": 10240, "creationDate": "2019-09-19T03:05:19.901Z" }
  ],
  "versions": [
    { "versionNumber": 2, "creationDate": "2016-09-27T07:38:05.44Z", "status": "Ready", "notes": "Republishing files so they're formally in our system" }
  ],
  "matchedQueries": ["explicit URL"],
  "scrapedAt": "2026-09-10T06:10:00.000Z"
}

Counts and timestamps change as Kaggle updates its catalog.

How much does it cost to export Kaggle datasets?

The Actor uses pay-per-event pricing: a $0.005 start fee per run plus one tiered item charge for each dataset record saved. Files and version entries are included in the dataset item and have no separate event charge.

The BRONZE item rate is $0.0064. The six plan tiers are FREE $0.00736, BRONZE $0.0064, SILVER $0.004992, and GOLD/PLATINUM/DIAMOND $0.00384 per saved dataset. At BRONZE, the total is the start fee plus the item rate multiplied by saved datasets: 1, 5, 10, 25, and 100 results cost USD 0.0114, 0.037, 0.069, 0.165, and 0.645 respectively. Apify Console shows your applicable plan tier before a run. You are never charged an item event for a rejected, duplicate, filtered, or failed record.

Recurring catalog inventory workflow

1. Configure a query, owner, or tag and save it as an Apify Task.
2. Schedule the Task daily, weekly, or monthly.
3. Send the finished dataset to a webhook, Make, Zapier, Google Sheets, or your warehouse.
4. Compare records by ref, currentVersionNumber, lastUpdated, licenseName, or file names.
5. Review newly found or changed datasets in your governance process.

The Actor produces snapshots. Change detection, alerts, and historical retention are performed by your downstream workflow.

Export to a spreadsheet or pipeline

Use the dataset export menu for CSV or Excel. Nested files and versions are best preserved in JSON; spreadsheet tools may serialize these arrays into cells. For a warehouse, request dataset items through the API after the run succeeds and upsert on ref.

API usage with cURL

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~kaggle-dataset-catalog-export/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchQueries":["renewable energy"],"sortBy":"updated","maxItems":10}'

To wait for completion and return dataset items, use the run-sync-get-dataset-items endpoint where suitable for your timeout and item volume.

API usage with JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/kaggle-dataset-catalog-export').call({
  searchQueries: ['renewable energy'],
  sortBy: 'updated',
  maxItems: 10,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

API usage with Python

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/kaggle-dataset-catalog-export').call(run_input={
    'searchQueries': ['renewable energy'],
    'sortBy': 'updated',
    'maxItems': 10,
})
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)

Use through MCP

Add the Apify MCP server to Claude Code:

claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/kaggle-dataset-catalog-export"

Claude Desktop

Use this MCP server entry in Claude Desktop's configuration:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/kaggle-dataset-catalog-export"
    }
  }
}

Cursor

Add the same mcpServers.apify.url entry in Cursor's MCP settings.

VS Code

Add the same HTTP MCP server URL in your VS Code MCP configuration.

Example prompts:

• “Search Kaggle for five highly voted climate-change datasets and summarize their licenses and sizes.”
• “Export the metadata, version history, and file manifest for uciml/iris.”
• “Refresh a five-item classification dataset inventory ordered by update time.”

Reliability and failure behavior

The Actor uses anonymous public Kaggle JSON endpoints. It checks for JSON responses and recognizable shapes. Temporary transport failures, rate limits, and server errors receive bounded exponential retries. Stable client errors, malformed inputs, non-JSON challenge pages, and unexpected response shapes fail the run with a diagnostic log.

There is no automatic residential proxy or browser fallback, so a retry cannot silently create expensive transfer or browser charges.

Limits

• Public datasets only; private datasets and account-only metadata are unavailable.
• Dataset files are listed but not downloaded or parsed.
• Counts, usability ratings, tags, descriptions, and versions reflect Kaggle's current public response and may change.
• maxItems is a cap, not a promise that every query has that many matching datasets.
• A restrictive owner/tag combination can naturally return no records.
• Search relevance and sort order are controlled by Kaggle.
• Very large inventories make multiple detail and file requests and take longer.

Responsible use and legality

Use only public information you are authorized to collect. Review Kaggle's terms, dataset-specific licenses, and any restrictions stated by dataset owners. A catalog license label is metadata, not legal advice and not permission to reuse dataset contents. Do not use this Actor to bypass access controls or to process personal data unlawfully. Apply your organization's retention, attribution, security, and governance requirements.

Troubleshooting

Why did my run return no records?

Check spelling and remove overly restrictive owner or tag filters. Try the same query directly in Kaggle. A valid no-result search succeeds with an empty dataset.

Why did the run fail instead of returning an empty dataset?

The Actor distinguishes a real empty catalog result from an upstream error. Inspect logs for an HTTP status, content-type mismatch, timeout, or unexpected response shape. Retry later for a temporary Kaggle outage; do not raise retries indefinitely.

Why is a known URL filtered out?

The same owner and tag filters apply to explicit URLs. Remove the conflicting filter or provide a matching URL.

Why are files or versions empty?

Confirm includeFiles, includeDetails, and includeVersions are enabled. Kaggle may also omit metadata for some records.

FAQ

Does this download Kaggle datasets as CSV?

No. It exports the catalog metadata itself as JSON, CSV, Excel, and other Apify dataset formats. It lists dataset files but does not download their contents. This keeps recurring discovery runs small and avoids republishing source data.

Does it include Kaggle models?

No. This Actor intentionally covers datasets only. Models, competitions, notebooks, courses, and profiles are outside its contract.

Can I combine queries and URLs?

Yes. The Actor deduplicates by Kaggle owner/slug ref and records which queries matched. The final result count still respects maxItems.

Can I schedule it?

Yes. Save the input as an Apify Task, add a schedule, and connect the resulting dataset to your preferred storage or automation tool.

Is an API credential required?

No Kaggle credential is required for the supported public catalog metadata. Your Apify token is needed only when invoking the Actor through Apify's API or client.

Related Automation Lab Actors

• arXiv Paper Search & Export for recurring public research-paper metadata searches.
• Hugging Face Scraper when your workflow genuinely needs Hugging Face catalog records rather than Kaggle datasets.

Choose this Actor when the source of record is Kaggle and you need normalized, source-linked dataset catalog metadata rather than file content.