YouTube Data Scraper Pro

Collect structured public YouTube data from videos, Shorts, search results, channels, handles, playlists, and regional popular feeds. The Actor is HTTP-first, streams results to an Apify Dataset, applies strict global limits, and is designed for scheduled monitoring and automation.

Use one direct video URL or combine several source types in the same run. Every accepted row keeps its source context, uses a stable ID, and can flow into spreadsheets, databases, dashboards, webhooks, or scheduled monitoring without a custom scraper.

What it does

• Accepts one source or a mixed list of video, youtu.be, Shorts, search, channel, handle, /videos, /shorts, playlist, and live URLs.
• Runs native YouTube search filters for upload date, duration, feature, sort, country, and language.
• Enriches accepted videos with public metadata, optional caption/transcript data, derived engagement metrics, normalized contacts, and temporary streaming metadata.
• Deduplicates by stable video ID across every source and enforces maxItems exactly under concurrency.
• Supports incremental runs, safe field selection/mapping, JSON/CSV-friendly flat output, optional global sorting, adaptive retries, and isolated error storage.

What you get

Every primary Dataset row has a stable resultId, canonical URL, source attribution, video/channel metadata, normalized numbers and dates, complete or partial status, and a data-quality block. Channel and playlist rows can be enabled through resultTypes. The Key-value store contains RUN_SUMMARY, WORKLOAD_ESTIMATE, and optional incremental state.

Viral Score is a derived heuristic, not an official YouTube metric. Streaming URLs and caption-track URLs may expire. Only information already exposed publicly by YouTube is processed.

How to start

1. Enter a search keyword, YouTube URL, or channel handle.
2. Set Maximum number of results.
3. Press Start. Advanced filters are optional.

Minimal input:

{
  "keywords": ["open source robotics"],
  "maxItems": 20
}

Example output:

{
  "type": "video",
  "resultId": "youtube:video:dQw4w9WgXcQ",
  "title": "Example public video",
  "url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
  "views": 1250000,
  "publishDate": "2025-01-15T00:00:00.000Z",
  "channel": {
    "name": "Example Creator",
    "url": "https://www.youtube.com/@examplecreator"
  },
  "source": { "type": "direct_video" },
  "resultStatus": "complete"
}

Why this Actor?

• Direct single-video analysis is a first-class input.
• Search, channels, handles, channel Shorts, playlists, and direct URLs can share one run.
• Native search filters and consistent post-filters are kept separate and transparent.
• The accepted-result cap is exact under concurrency, and duplicates are not counted twice.
• Source attribution, monitoring state, Dataset views, and API-friendly fields are built in.
• Analytics are optional and labeled as derived values rather than official YouTube metrics.

Mixed-source input:

{
  "startUrls": [
    { "url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ" },
    { "url": "https://www.youtube.com/@GoogleDevelopers/shorts" },
    { "url": "https://www.youtube.com/playlist?list=PL590L5WQmH8fJ54F369BLDSqIwcs-TCfs" }
  ],
  "keywords": ["web performance"],
  "maxItems": 25,
  "distributionMode": "balanced"
}

Supported sources

Source	Examples
Video	youtube.com/watch?v=…, youtu.be/…, /embed/…, /live/…
Shorts	youtube.com/shorts/…, channel /shorts tab
Search	Keywords, searchQueries, and YouTube search-result URLs
Channel	/channel/…, /@handle, /@handle/videos, legacy /user/… and /c/…
Playlist	/playlist?list=… and watch URLs containing a playlist ID
Popular	Region-aware home/popular fallback, explicitly labeled popular_fallback when a stable Trending feed is unavailable

Unsupported hosts are rejected; this Actor is not a general-purpose URL crawler.

Common input examples

Single video:

{
  "startUrls": [{ "url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ" }],
  "maxItems": 1
}

Channel or handle:

{
  "youtubeHandles": ["@GoogleDevelopers"],
  "maxItems": 20,
  "resultTypes": ["video", "short", "channel"]
}

Channel Shorts:

{
  "startUrls": [{ "url": "https://www.youtube.com/@GoogleDevelopers/shorts" }],
  "shortsMode": "only",
  "maxItems": 20
}

Playlist:

{
  "startUrls": [
    { "url": "https://www.youtube.com/playlist?list=PL590L5WQmH8fJ54F369BLDSqIwcs-TCfs" }
  ],
  "maxItems": 20
}

Filtered search:

{
  "keywords": ["robotics"],
  "gl": "US",
  "hl": "en",
  "uploadDate": "month",
  "duration": "long",
  "sort": "view_count",
  "minViews": 10000,
  "maxItems": 25
}

Monitoring:

{
  "youtubeHandles": ["@GoogleDevelopers"],
  "maxItems": 20,
  "incrementalMode": true,
  "incrementalStateKey": "google-developers-weekly",
  "stopWhenKnownItemFound": true
}

Advanced output selection:

{
  "keywords": ["web performance"],
  "maxItems": 20,
  "outputFormat": "flat",
  "fields": ["resultId", "title", "url", "views", "channel.name"]
}

Important input behavior

• maxItems is a strict global count of accepted video and Short records. Filtered, failed, duplicate, channel, and playlist rows do not consume the cap.
• maxItemsPerQuery, maxVideosPerChannel, and maxVideosPerPlaylist are source caps; the global cap always wins.
• distributionMode: "balanced" rotates across sources. "sequential" completes sources in supplied order.
• maxConcurrency: "auto" backs off after rate limits and increases cautiously during stable batches. A number from 1 to 32 pins the ceiling.
• Native filters affect search discovery. Exact numeric/date/keyword filters are applied to normalized results from every source.
• outputSort: "none" streams with constant result memory. A sort mode buffers at most maxItems accepted rows.
• customMapFunction is intentionally disabled. Use fields or declarative outputMapping; arbitrary user code is never evaluated.

Output contract

Primary records use type: "video" | "short" and resultId: "youtube:video:<id>". Missing public metrics are null, not fabricated. When detail extraction fails but trustworthy discovery metadata exists, the Actor emits a labeled partial record instead of silently losing it.

Useful Dataset views:

• Overview for routine exports
• Videos for normalized video analytics
• Channels for public channel identity and performance fields
• Search results for query and position context
• Shorts for short-form research
• Compatibility for migrations
• Full / technical for source, caption, quality, and run metadata

The default Dataset can be exported as JSON, CSV, Excel, XML, or RSS through Apify storage APIs.

Search and post-processing filters

Native search filters: gl, hl, uploadDate, duration, features, and sort.

Cross-source filters: shortsMode, duration/view/like/subscriber ranges, exact publish-date bounds, required keywords, excluded keywords, and missingMetricBehavior.

searchQueries permits per-query overrides:

{
  "searchQueries": [
    {
      "query": "AI agents",
      "maxItems": 50,
      "gl": "US",
      "hl": "en",
      "uploadDate": "week",
      "sort": "view_count"
    },
    { "query": "n8n automation", "maxItems": 50 }
  ],
  "maxItems": 75
}

Captions, transcripts, and temporary formats

includeCaptions stores public caption-track metadata. transcriptMode can be none, text, or segments, with maxTranscriptCharacters enforcing a per-video safety cap. includeStreamingData stores public technical format metadata only; returned URLs are temporary and this Actor is not a media downloader.

Monitoring

Enable incrementalMode, choose a stable incrementalStateKey, and schedule the Actor. Stable IDs are checkpointed every 50 accepted results and at run completion. stopWhenKnownItemFound is useful for newest-first channel monitoring.

Typical flows:

• Apify Dataset → n8n → Google Sheets
• Scheduled Actor → new video → webhook → Slack
• Actor API → Make or Zapier → database/CRM

API examples

cURL / REST

curl -X POST "https://api.apify.com/v2/acts/YOUR_USERNAME~youtube-data-scraper-pro/runs?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"keywords":["robotics"],"maxItems":10}'

After the run finishes, retrieve its Dataset:

$curl "https://api.apify.com/v2/datasets/YOUR_DATASET_ID/items?clean=true&format=json&token=YOUR_APIFY_TOKEN"

JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });
const run = await client.actor('YOUR_USERNAME/youtube-data-scraper-pro').call({
  youtubeHandles: ['@GoogleDevelopers'],
  maxItems: 10,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems({ clean: true });
console.log(items);

Python

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("YOUR_USERNAME/youtube-data-scraper-pro").call(run_input={
    "startUrls": [{"url": "https://www.youtube.com/shorts/jNQXAC9IVRw"}],
    "maxItems": 1,
})
items = client.dataset(run["defaultDatasetId"]).list_items(clean=True).items
print(items)

Never commit a real Apify token or private browsing cookie.

Billing behavior

The intended PPE event is video-result. It fires only when one unique, accepted primary video/Short is successfully pushed. Duplicates, filtered candidates, failures, and supporting rows do not trigger that event. The implementation uses one dataset-plus-PPE path, so there is no second manual charge.

Pricing must not be published from a guess. See ./PRICING.md for the measured-cost gate and safety formula.

Reliability and privacy

• Retries are error-aware; rate limits reduce automatic concurrency.
• One failing source is isolated from other mixed inputs.
• Secrets are not logged, saved in Dataset rows, or included in summaries.
• Contact extraction is limited to addresses and links already visible in public text. No reveal endpoint, login bypass, CAPTCHA bypass, or guessing is used.
• No eval, dynamic function execution, browser extension, or arbitrary remote crawling is present.

Limitations

• Public YouTube data only; private, deleted, login-only, age-restricted, and region-blocked access is not promised.
• Public field availability varies, so some values are correctly returned as null.
• YouTube response changes can temporarily affect parsers.
• Caption and streaming URLs, when requested, can expire.
• Regional popular/trending behavior varies and fallback results are explicitly labeled.

This independent product is not affiliated with or endorsed by YouTube. Use public data in accordance with applicable laws and platform terms.

Troubleshooting and support

If a source fails, check its sanitized error Dataset and RUN_SUMMARY; other mixed sources continue where possible. When reporting a problem, provide the run ID, input type, affected public URL, expected result, actual result, and a short non-secret log excerpt. Never share tokens, passwords, cookies, or proxy credentials.

See ./FAQ.md for short answers about sources, filters, scheduling, API use, null fields, billing, and partial failures.

Local development

npm ci
npm run typecheck
npm run lint
npm test
npm run build
npm run validate:schemas

Live tests are opt-in:

$LIVE_TESTS=true npm run test:actor

See ./DEPLOYMENT.md, ./SECURITY.md, ./COMPATIBILITY.md, and ./CHANGELOG.md.