# Changelog of YouTube Scraper $2/1K — Videos, Channels, Comments, Transcripts (`blackfalcondata/youtube-data-scraper`) Actor

- **URL**: https://apify.com/blackfalcondata/youtube-data-scraper/changelog.md
- **Full Actor documentation**: https://apify.com/blackfalcondata/youtube-data-scraper.md

## Changelog

All notable, user-facing capabilities of the YouTube Data Scraper. This is a
feature log rather than a strict per-release history; it summarizes what the
actor can do.

### \[0.1.3] — self-audit of the audit fixes

- **A run that hits its charge limit no longer forgets the records it never
  delivered.** When the run's maximum charge is reached the platform writes only
  part of the batch and reports success; those unwritten records were counted as
  delivered and recorded as seen, so a later run would skip them. Only what the
  platform accepted now counts.

- **A record chosen by incremental mode arrives complete.** The reordering that
  picks the new items ran after the detail and translation fetches, so a newly
  chosen record could be delivered - and charged - without the description,
  category or translation it was supposed to carry, while three enriched records
  were discarded.

- **A channel that could not be read is no longer delivered as a record.** A
  blocked or invalid response produced a row with no channel id, no title and no
  fields, charged like any other result. It now counts as a failed target.

- **You are told when a channel has no Shorts or Live list** and the run used the
  channel's main video list instead.

- **`compact` works.** It was overridden by the detail level's own default, so
  ticking it changed nothing and the run still paid for a detail fetch per video.

- **Bare playlist IDs are accepted**, as the input has always said. A pasted
  `PL...` id used to be skipped and the run finished with zero rows.

- **The same video returned by two searches takes one slot** of the result limit
  instead of two.

- **With RAG output, an item with no text to embed is no longer recorded as
  delivered** - it produced no document, so it stayed suppressed for ever.

- **Expired notices are counted correctly** when only part of a run's results
  could be saved, and notifications, the run summary and the connected-app export
  now all describe what actually reached the dataset.

- **The result limit now really bounds the detail fetching.** The cap on how
  many videos get enriched was computed, logged, and then not applied - the run
  still fetched a watch page for every video it had discovered, which under
  incremental mode is far more than the limit. Same for localized titles.

- **A video that both search and trending found is no longer re-delivered every
  run.** The two copies collapse into one record, but only that record's
  identity was remembered, so the other one counted as new for ever.

- **`maxShorts: 0` no longer expires Shorts that are still there.** Removing
  every Short left the run claiming full coverage of a channel it had filtered.

- **Two spellings of the same link take one slot.** `watch?v=A` and `youtu.be/A`
  each consumed a slot of the result limit before the duplicate was removed, so
  a third, different target could be skipped.

- **A playlist is no longer paginated past the per-target limit** - with a
  per-target limit of 1 the run walked hundreds of entries to keep one.

- **Turning on RAG output, replies, a different transcript format, or another
  channel tab now uses its own incremental state**, instead of inheriting a
  bucket that reports everything as unchanged. Runs already using the defaults
  keep the state they have - nothing is re-delivered.

- **A refused transcript is no longer recorded as delivered** when RAG output is
  on.

- **The pricing table said a result is charged once per source item, never per
  chunk.** The FAQ said the opposite, and the FAQ was right: every record
  written to the dataset is charged, and with a chunk size each chunk is a
  record. The table now says so.

- **An incremental feed no longer starves behind its own result limit.** With
  `maxResults` set, the limit was applied to the raw discovery order before the
  change detector ran — so a run could keep N items you had already received and
  slice off the one new item just past the cut, delivering nothing, run after
  run. Items the run has not seen before (or that changed) now sort ahead of the
  rest, so the limit keeps the ones you are actually waiting for, and their
  comments and transcripts are the ones fetched.

- **A long transcript is charged once, not once per dataset write.** With RAG
  chunking on, a transcript whose chunks spanned two writes was charged the
  transcript event twice, against a contract that promises one charge per source
  item.

- **A record the dataset refuses is now always identifiable**, including RAG
  documents (whose fields live under `metadata`) and transcripts trimmed by
  `transcriptFields` — previously those could be recorded as delivered and never
  re-sent. A trimmed transcript now always keeps its `videoId` and `language`,
  which a consumer needs to join it to anything anyway.

- **One upload is one record, whichever surface found it.** A video reached by
  both a search and Trending was delivered — and charged — twice, because the two
  surfaces label it differently internally.

- **`maxShorts` no longer produces false "removed" notices.** Like the other
  filters, it drops present videos, so a Short displaced by another Short could
  be reported as removed and billed as a notice.

- **`country` reaches a pasted channel URL too.** The search path carried the
  locale; the explicit channel path did not.

- **A run that reaches its time limit delivers what it collected.** Every video
  in a batch starts before the deadline and an in-flight fetch cannot be
  interrupted, so one batch could outlive the whole budget and the run was killed
  with its results still in memory. Each video's detail, comment and transcript
  fetch is now bounded by the time actually left. Measured on the sibling actor:
  the same job went from delivering nothing to delivering 1,400 records.

- **The run time limit is honoured again.** The graceful stop read one
  environment variable while the platform supplies another, and its safety margin
  was smaller than the wall-clock guard's — so it never ran, and the guard ended
  the process with results still buffered.

- **Aliases now win over the canonical field.** The platform writes every
  declared default into the input, so `limit`, `lang`, `geo` and `proxy` were
  dead whenever their canonical field had a default — `limit: 1` could still
  collect and charge up to fifty records. An alias is only ever typed on purpose,
  so it is the explicit signal; a value that cannot be used for the field (a typo
  in a number) is ignored rather than allowed to shadow yours.

- **A trending-only run no longer pays for a search it throws away.** With only
  Trending enabled, each query still ran a full search whose results nothing
  consumed — and those discarded sources ate the discovery budget, which could
  leave Trending itself with nothing to collect.

- **`maxResultsPerQuery` counts the channel record.** The cap is documented as
  videos *and channels* per target, but the channel record was emitted without
  consuming a slot, so a cap of 5 returned one channel plus five videos.

- Run counts, run metadata and notifications now describe what was actually
  delivered. After de-duplication was added they still counted the duplicates
  that were no longer pushed.

### \[0.1.2] — audit fixes

- **A small `maxResults` no longer fetches sub-resources for hundreds of videos.**
  In incremental mode discovery widens to 500 sources so the state can see the
  whole universe; every one of those was then used as a comment/transcript
  driver, so `maxResults: 1` with comments on could fetch and bill comments for
  up to 500 videos. The video budget now bounds what is fetched, not just what is
  emitted.
- **Records that never reached the dataset are no longer remembered as
  delivered.** If a batch failed every retry, those records were recorded as seen
  and suppressed on every later run — permanently lost to the customer.
- **A sub-resource-only run where every fetch came back unreadable now fails**,
  instead of reporting "no results" and suggesting your URLs or filters were
  wrong.
- The run time limit is now honoured during the comment and transcript phases,
  not only before they start.
- **Detail fetching stops at your result limit.** Enrichment ran over everything
  discovered, and incremental mode widens discovery to 500 sources — so a run
  with `maxResults: 3` fetched 200 watch pages to deliver three records, paid for
  in run time and proxy traffic. It now enriches only what can be delivered.
- **Removed `includeDetails`**, which was job-board template copy that no code
  path read — setting it changed nothing. Detail depth is `detailLevel`.
- **"0 = unlimited" now names its real ceilings**: a search contributes at most
  200 videos per query, a playlist 100, a channel its latest page of uploads, and
  an unlimited comment fetch stops after 200 pages (~4,000 comments).
- **Blank `transcriptLanguage` is documented as English**, which is what the code
  requests — it never used the video's own default track.
- **`country` and `hl` now actually reach YouTube.** Search, channel, playlist
  and comment requests were built without locale, so the context fell back to
  US/English no matter what you set — the inputs only ever changed which
  incremental bucket your run wrote to. Comments deliberately stay on English:
  `hl` localizes the relative timestamp the ISO `publishedAt` is derived from,
  and a localized one nulls that field on every record.
- **A filtered run no longer marks still-present items as removed.** With a date
  window, a keyword filter or a custom filter active, what remains is a view of
  the universe rather than the universe — so a video that simply fell outside the
  window was classified EXPIRED and deactivated, and with `emitExpired` on, a
  false notice was emitted and charged. Filtered runs now report incomplete
  coverage, which suppresses the EXPIRED scan.
- **An item reached by two of your targets is delivered once, not twice.** The
  same video pasted as `watch?v=` and as `youtu.be/`, two searches whose results
  overlap, or a channel that also shows up in a search all produced duplicate
  records — each of them charged. Measured on a live run before the fix: one
  video, two URL forms, two billed rows.
- Billing copy corrected: a transcript is charged the result event plus the
  transcript event, RAG chunking costs one result event per chunk, and EXPIRED
  notices are charged like any other record.

### \[0.1.1] — comment transport fixes

- **`commentsSort: new` now actually sorts.** YouTube ships the comment order as
  a second continuation token, and the transport always walked the default one,
  so a "newest" request returned top comments.
- **No more duplicate comments on short videos.** The next-page token was found
  by scanning the payload for the first continuation token, which on a last page
  is the sort menu's — so the walk re-fetched the same page and appended the same
  comments again. Videos whose comments fit one page were affected.
- **The pinned comment is no longer delivered twice.** YouTube serves it at the
  top of the first page and again at its own place in the feed, and the second
  copy came back without the pinned flag — one duplicate record per affected
  video, with contradictory `isPinned` values.
- Input aliases no longer warn about a "conflict" when an alias carries the same
  value as its canonical field.

### [0.1.0]

#### Modes

- **Search** — find videos by keyword, with each query run as its own task.
- **Channel** — collect channel profiles (subscriber / video / view counts,
  country, join date, links, banner) from channel URLs, @handles, or channel
  search results.
- **Video** — rich per-video metadata and statistics.
- **Comments** — fetch comments with author flags, like counts, and optional
  nested replies.
- **Transcripts** — download full captions in multiple formats.
- **Trending** — surface the most-viewed recent videos for a topic.
- Flexible targeting: paste video / channel / playlist URLs, bare video IDs, or
  bare @handles; or run keyword searches. Typed URL lists (`videoUrls`,
  `channelUrls`, `playlistUrls`) merge with `startUrls`.
- Friendly input aliases so common alternate field names are accepted
  (e.g. `query` / `q` / `keywords`, `urls`, `limit` / `maxItems`,
  `lang`, `gl` / `region`, `proxy`).

#### Filters

- Search filters: sort by relevance / date / view count / rating; upload-date
  window; video duration; result type (videos / channels / playlists / any).
- Feature filters: HD, 4K, HDR, 360°, subtitles/CC, Creative Commons, live.
- Channel options: choose the Videos / Shorts / Live tab and the sort order;
  optionally fetch the About panel for join date, country, total views, links.
- Shorts handling: include, exclude, cap, or return only Shorts.
- Post-fetch filters across all modes: include/exclude keywords, published-date
  range (`fromDate` / `toDate`, absolute or relative), max age in days, and a
  custom filter expression.

#### Transcripts

- Output as timed segments (JSON), plain text, SRT, WebVTT, or raw XML.
- Choose a transcript language, prefer auto-generated tracks, and optionally
  translate the transcript into another language.
- Emit only a chosen subset of transcript fields.

#### Output enrichment

- Engagement metrics: like-to-view and comment-to-view ratios, and a combined
  engagement rate.
- Extracted hashtags, description URLs, and social-profile links.
- Parsed chapters, advertised caption languages, and high-resolution thumbnails.
- Channel-level derived metrics: average views per video and estimated uploads
  per month.
- Description rendering in plain text, HTML, Markdown, or all formats.
- Optional translated titles and descriptions via YouTube's own localization.
- Canonical, embed, and short URLs on every video.
- Per-field metadata surfaced in the Apify console, plus tabular dataset views
  for Videos, Channels, Comments, Transcripts, and Playlist Videos.

#### Volume & control

- Global result cap, per-query/per-target cap, adjustable concurrency, and an
  optional soft run-time limit that saves partial results gracefully.
- Detail levels from Basic (listing fields only) to Complete (all fields).
- Locale controls for country and interface language.
- Optional run metadata (`runId`, `actorId`, `startedAt`) on every item, and an
  option to strip empty fields for compact output.

#### Incremental updates

- Track seen items across runs and emit only new and changed items on repeat
  runs, keyed by a stable state key.

#### Notifications

- Send a run summary to Telegram, Discord, Slack, WhatsApp, or a custom webhook,
  with a configurable item limit and an option to notify only on new/updated
  items.

#### Robustness

- Invalid locale codes and out-of-range numeric inputs are normalized to safe
  defaults instead of failing the run.
- Optional proxy routing (enabled by default).
- Results are written to the dataset in bounded batches as the run progresses,
  so a timeout or interruption keeps the data already collected and memory
  stays bounded on large runs.
- Saving each batch is retried automatically on a transient storage hiccup; a
  batch that still fails is skipped with a warning rather than aborting the run.

[0.1.0]: #010