YouTube Scraper: search, channels, playlists, videos and comments

YouTube hands out 290 videos for a search term and then stops. Not 290 per page — 290 in total, however many pages you ask for. In the very same response YouTube says the term has 2 201 319 results. This Actor takes the same term, splits it into 64 separate searches that YouTube answers separately, merges them and returns 3581 unique videos — 12.3 times more.

Everything is one Actor and one table: search results, shorts, live streams, channels, playlists, full video details and comments. No API key, no Google Cloud project, no daily quota, no browser, no proxy.

The measurement behind the headline

The number is reproducible on your machine:

python3 tools/measure.py ceiling deep

Both runs use the Actor's own code, the same search term excavator repair and the same defaults. Only one switch differs.

	Plain search	Deep search
Unique videos	290	3581
Requests to YouTube	19	678
Slices of the query	1	64
Wall clock	seconds	587.8 s

Why it works: YouTube's own filters travel inside the request. Ask for "this month, over 20 minutes, sorted by view count" and YouTube runs a different query and answers with a different set of videos. Sixty-four such slices — four sort orders, four upload windows, four durations — overlap heavily, and the Actor removes duplicates by video ID, which is why 64 x 290 is nowhere near 3581. What remains after deduplication is still 12.3 times what one query gives you.

Deep search is off by default, because most people want the first hundred results and nothing more. Turn it on with deepSearch and raise maxResultsPerQuery to match.

What this Actor replaces

The strongest Actor in this niche, streamers/youtube-scraper, has 122351 users and 4.8 stars. It is good. It is also one of a family of 5 separate Actors — search, shorts, channels, comments and the downloader — each set up, scheduled and paid for on its own. This Actor covers four of those five jobs in one run, one input form and one dataset.

Side by side on what can be counted: we declare 59 output columns against their 34, and 32 input fields against their 29. The full line-by-line list, including the places where they are ahead of us, is in COMPARISON.md.

The niche itself is not empty, which is the point: 88 Actors with the token youtube in their name share 385079 users.

What you can ask for

Search terms. searchQueries takes one term per line. searchFor decides what comes back: videos, shorts, live streams, channels or playlists. Shorts are a separate listing on YouTube's side — asking for videos excludes them and asking for shorts excludes regular videos, so the choice is real rather than cosmetic.

Direct addresses. startUrls accepts a video URL, a channel URL, an @handle, a playlist URL, or the bare IDs behind any of them. A handle is resolved to a channel ID by YouTube itself. For a channel, channelTabs selects which of the 4 tabs to read: videos, shorts, live and playlists.

Filters YouTube applies itself. sortBy, uploadDate, videoDuration and features are handed to YouTube inside the request, so non-matching videos never reach your dataset and never reach your bill. There are 11 feature filters: HD, 4K, subtitles, Creative Commons, live, 360, VR180, 3D, HDR, location and purchased.

Filters we apply ourselves. titleContains, channelContains, minViews, maxViews, minDurationSeconds, maxDurationSeconds, publishedAfter and publishedBefore are things YouTube cannot express. They still run before a row is written, so they still save you money — and a row that is missing the field a filter needs is kept, never silently dropped. The count of such rows is in the kept_without_data field of the REPORT record.

Depth and detail. maxResultsPerQuery stops a term after a number of rows — 100 rows per search term by default. includeVideoDetails opens each video page for the exact view count, like count, comment count, full description, tags, category and a publication date accurate to the second. includeComments adds one row per comment with author, likes, replies and a direct link; 20 comments per video by default, ordered by commentsSortBy (top or newest) and capped by maxCommentsPerVideo. By default 4 detail requests run in parallel, which concurrency changes.

Locale. language (hl) and country (gl) change what YouTube ranks and how it words relative dates. They are not decoration: a search from the US and the same search from Germany return different videos.

Output shape. compactOutput cuts the table to 15 decision-making columns. dropEmptyFields removes null cells entirely. maxItems is a hard stop on delivered rows, comments included.

Monitoring. onlyNew remembers what earlier runs delivered in a named key-value store and writes only what is new or changed, with change_type and a changed_fields list naming the old and new value. emitUnchanged brings the rest back when you want a full snapshot. The fingerprint deliberately ignores the thumbnail URL, because YouTube signs it afresh on every request and a whole-row fingerprint would call every video "changed" every time.

Chaining. inputDatasetId and urlField read YouTube addresses out of another Actor's dataset, so a run of a different scraper can feed this one.

What comes back

One table, four kinds of row, told apart by the kind column: video, channel, playlist, comment. A column that does not apply to a row is empty rather than filled with a guess — shorts genuinely carry no duration and no channel name in YouTube's listings, and inventing them would be worse than leaving them blank.

Video rows carry the ID, canonical URL, title, description snippet, channel name, channel handle, channel ID and channel URL, duration as text and in seconds, view count as a number and as YouTube's own wording, the published text, an estimated publication date, thumbnail, badges, and the live and short flags. With includeVideoDetails on they also carry the exact view count, like count, comment count, full description, tags, category, the exact publication and upload timestamps, the family-safe flag, the ratings flag, the number of countries the video is available in, and a playability note when YouTube refuses to serve the video.

Channel rows carry the channel ID, name, handle, URL, subscriber count as a number and as text, video count, description, channel keywords, a verified flag, the country, the date the channel was created, the channel's lifetime view count and every external link the owner published - Discord, Instagram, a shop, a mailing list. Those links are read from the channel's own About page, which is where YouTube keeps them; they arrive with their YouTube redirect stripped, so they are the real addresses. Playlist rows carry the playlist ID, title, video count and the owning channel. Comment rows carry the comment ID, body, reply level, author name, author channel ID, verified and creator flags, avatar, like count, reply count, a hearted-by-creator flag and a link that opens that exact comment.

The last row of every run is a REPORT record: rows collected, rows pushed, videos detailed, comments fetched, duplicates skipped, which filter dropped how many, which rows were kept because a field was missing, how many requests went out, how many bytes came back, and — when a run ends empty — why.

Why an empty run is never a mystery

A green run with an empty dataset is the worst thing a scraper can do to you: you paid for the start and you have no idea what happened. This Actor names the cause. There are 15 named reasons, and 10 of them are decided from your input alone, before a single byte is downloaded:

no-target-given, all-urls-invalid, comments-without-videos, comments-with-a-zero-limit, duration-filter-on-shorts, date-filter-on-shorts-without-details, video-filter-on-a-channel-search, asked-for-zero-rows, upload-window-excludes-the-date-range and charge-limit-reached.

Each of those is a real trap found in YouTube's own data. Shorts carry no duration at all, so a minimum duration would drop every single short — the Actor says so instead of downloading a few megabytes first. Shorts carry no publication date either, so a date filter needs includeVideoDetails on. A channel has neither views nor duration, so a channel search with a view floor can only ever return nothing. An upload window of one week cannot contain a video published before that week began.

The remaining five are visible only after the crawl: filtered-out, nothing-changed, source-returned-nothing, all-targets-unreachable and source-asked-to-slow-down. nothing-changed is the legitimate one — a monitoring run with nothing new to report is working as asked, and you are not charged for rows you already have.

How we know YouTube answers Apify's servers

Before a line of this Actor was written, a probe Actor ran from Apify's own cloud and asked YouTube directly. Run OoqpPHoLljIZZnB0D put 13 targets to YouTube — search, channel browse, the comments endpoint, the player endpoint, URL resolution, a video page and a channel page. All 13 answered with real data, the video page alone weighing 1379740 bytes and the comments endpoint 410898. Run 2esigRv5zIYXrlJf2 then fired 18 different searches back to back from one Apify address: 18 answers, between 531546 and 983007 bytes each, not one rate-limit refusal.

This matters because a neighbouring source in the same portfolio (SofaScore) answers a laptop normally and refuses every Apify address, and an Actor built on it had to be thrown away. Checking the source from the cloud first is now a rule, not a courtesy.

Honest limitations

No subtitles and no transcripts. YouTube's timedtext endpoint answers with a success code and zero bytes unless a BotGuard token is present, measured both from Apify and from a laptop. That is not a block on us; it does not work for anyone without a browser. streamers/youtube-scraper offers subtitles and AI transcription as paid add-ons. If you need transcripts, use a transcript Actor.

No video downloads. Downloading requires the signature-cipher work that a browser does. Their family has a downloader; we do not.

No AI summaries or AI descriptions. Those are paid add-ons on their side and would be a different product on ours.

Sorting is a hint, not an order. sortBy changes which videos YouTube returns — measured: a view-count sort shares only 1 of 20 results with the relevance sort — but the rows come back in YouTube's ranking order, not in strict chronological or view order. We use it as a slicing dimension, which is what deep search needs, and we do not promise a sorted list.

Dates in listings are estimates. A search result says "4 years ago" and nothing more precise. The Actor converts that into published_at_estimated and keeps it clearly separate from published_at, which is exact and arrives only with includeVideoDetails.

No replies to comments. Top-level comments are collected; the reply threads under them are not.

No private or members-only content. Anything that needs a signed-in account is out of reach by design. This Actor collects public creator and business information only.

FAQ

Do I need a YouTube Data API key? No. Nothing here touches the Data API, so its daily unit quota does not apply and there is nothing to register, bill or rotate.

Does deep search cost more? Yes, in requests and in time: 678 requests and 587.8 seconds against 19 requests for the plain search in the measurement above. It is off by default for that reason. Turn it on when reach matters more than speed.

Why did I get fewer rows than maxResultsPerQuery? Either YouTube ran out of continuation tokens for that slice — the ceiling this Actor exists to work around — or your own filters removed rows after collection. The REPORT record tells you which, filter by filter.

Can I monitor a channel for new uploads? Yes. Put the channel URL in startUrls, turn on onlyNew, and schedule the run. The first run delivers everything; later runs deliver only what is new or changed, and say nothing-changed when there is nothing.

Are comments charged as rows? Yes, a comment is a row like any other, which is why maxCommentsPerVideo exists and why maxItems counts comments too.

What happens if YouTube blocks the run? The Actor slows down, retries, and if nothing comes back at all it ends with source-asked-to-slow-down rather than a silent empty dataset. Whatever was collected before that point is delivered first.

Can I feed it URLs from another Actor? Yes — set inputDatasetId to that Actor's dataset and urlField to the column holding the address.