🎬 YouTube Shorts Scraper — Apify Actor 🔎

🎯 Purpose: A lightweight Apify Actor that scrapes YouTube Shorts metadata from one or more channel URLs. Designed to run in Apify using the Python SDK, it focuses on ⚙️ reliability and 🧭 proxy support (Apify proxy or custom proxy URLs).

---

✨ Key Features

✅ Extracts Shorts videoIds from channel pages (tries multiple channel endpoints). 📄 Fetches per-short details — title, description, duration, thumbnails, view counts, tags, captions URL, publish date, and channel info. 🌐 Built-in support for Apify Proxy (RESIDENTIAL/DATACENTER groups) via Actor.create_proxy_configuration(). 🔐 Accepts custom proxy strings (e.g., http://user:pass@host:port or host:port) and normalizes the scheme. 🔁 Retries with exponential backoff, configurable ⏱️ timeouts, and concurrency. 📤 Incremental Actor.push_data() for streaming results to the Apify dataset. 💾 Writes a local JSON file shorts_filled_{channel_or_id}.json with final results.

---

⚙️ Input (Actor Input JSON)

Provide actor input using the Apify run calls 🧩

🏷️ Field	Type	Required	💡 Default / Notes
startUrls 🌐	array<{url: string}> or string	✅	One or more channel URLs. Example: [{"url":"https://www.youtube.com/PenMovies/shorts"}] or ["https://www.youtube.com/PenMovies"]
maxshorts 🔢	integer	✅	Limit of Shorts per channel to collect IDs for further detail fetching.
proxyConfiguration 🧠	object	✅	Standard Apify proxy editor object. Example: { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] } or use a custom proxy (see below).
concurrentWorkers ⚡	integer	✅	Overrides default concurrency (CONCURRENT_WORKERS env var).

---

🌍 Custom Proxy (in proxyConfiguration)

• 🧩 If you provide a custom proxy string, use the customProxy / proxyUrl / proxy key in proxyConfiguration. The actor automatically adds http:// if you only provide host:port.
• 🚀 If useApifyProxy is set to true, the actor will use Apify’s proxy configuration via proxy_conf.new_url().

---

Would you like me to also add emojis in the Output, Examples, and How it works sections (if you have them)? I can style the whole README consistently like a polished Apify Store project.

Input Example :

{
  "startUrls": [
    { "url": "https://www.youtube.com/@TseriesKannadaMusic/shorts" }
  ],

  "maxshorts": 30,

  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"],
    
  }
}

---

Output Example :

[
  {
    "title": "🎬 Romeo S3 – July 18 → Prime Video + Pen Movies.",
    "type": "shorts",
    "id": "BJsbRhaXBPg",
    "video_url": "https://www.youtube.com/watch?v=BJsbRhaXBPg",
    "thumbnailUrl": "https://i.ytimg.com/vi/BJsbRhaXBPg/maxres2.jpg?sqp=-oaymwEoCIAKENAF8quKqQMcGADwAQH4AbYIgAKAD4oCDAgAEAEYZSBKKD8wDw==&rs=AOn4CLBl_qKaB5Hbo7MWFFCG6SCt7KM1-Q",
    "viewCount": "35.1k",
    "viewCountRaw": 35069,
    "uploadDate": "2025-07-17T06:20:47-07:00",
    "duration": 15,
    "liveBroadcastDetails": {
      "live_status": "not_live",
      "is_live": false
    },
    "processingDetails": {
      "status": "processed",
      "playable_in_embed": true
    },
    "description": "Romeo S3 – Streaming July 18\nOn Amazon Prime Video & Pen Movies YouTube!\nEnjoy Super Hit Movies Collection :- \nAcharya - https://youtu.be/MZGKv-UqD4c\nJaya Janaki Nayaka KHOONKHAR - https://youtu.be/1lFk1LSsI_c \nSHER KA SHIKAAR - https://youtu.be/FE435iFT6jw \nTholi Prema -  https://youtu.be/J4nBcaYMuI4\nEntertainment- https://youtu.be/LXXkiUKDK4w            \n\n\nFor your favorite Bollywood Movies, Subscribe now: \nhttp://bit.ly/2VXistY\n\nSUBSCRIBE for the best Bollywood clips, movies and scenes all in your PEN Multiplex channel \nhttp://bit.ly/2QxELp3\n\nFor your favorite Gujarati Movies, Subscribe now: http://bit.ly/2VZm9Q6\n\nFor Action-Packed Movies, Click on the below link now...\nPen Action: http://bit.ly/2XaSS6t\n\n\nEnjoy & stay connected with us! \nLike us on Facebook: https://www.facebook.com/PenMovies\nFollow us on Twitter: https://twitter.com/penmovies\nFollow us on Instagram: https://www.instagram.com/penmovies 👉 Log On To Our Official Website : https://penstudios.in/",
    "channelName": "Pen Movies",
    "channelUrl": "https://www.youtube.com/channel/UC3ar28GS6o1p0m_wabfk2zw",
    "channelId": "UC3ar28GS6o1p0m_wabfk2zw",
    "inputChannelUrl": "https://www.youtube.com/PenMovies/shorts",
    "hashtags": [],
    "fromYTUrl": "https://www.youtube.com/watch?v=BJsbRhaXBPg",
    "tags": [],
    "durationAlt": "15s",
    "transcriptSample": []
  },
  {
    "title": "No popcorn needed. Just WiFi. 🎬 Romeo S3 – July 18 → Prime Video + Pen Movies.",
    "type": "shorts",
    "id": "s9Pr4rydv_8",
    "video_url": "https://www.youtube.com/watch?v=s9Pr4rydv_8",
    "thumbnailUrl": "https://i.ytimg.com/vi/s9Pr4rydv_8/maxres2.jpg?sqp=-oaymwEoCIAKENAF8quKqQMcGADwAQH4AbYIgAKAD4oCDAgAEAEYciBWKEAwDw==&rs=AOn4CLCj8NKhwTuqOhxvoXMYqlEcYAlLag",
    "viewCount": "16.6k",
    "viewCountRaw": 16641,
    "uploadDate": "2025-07-13T07:30:39-07:00",
    "duration": 20,
    "liveBroadcastDetails": {
      "live_status": "not_live",
      "is_live": false
    },
    "processingDetails": {
      "status": "processed",
      "playable_in_embed": true
    },
    "description": "\"No popcorn needed. Just WiFi.\n 🎬 Romeo S3 – July 18 → Prime Video + Pen Movies.\"\nEnjoy Super Hit Movies Collection :- \nAcharya - https://youtu.be/MZGKv-UqD4c\nJaya Janaki Nayaka KHOONKHAR - https://youtu.be/1lFk1LSsI_c \nSHER KA SHIKAAR - https://youtu.be/FE435iFT6jw \nTholi Prema -  https://youtu.be/J4nBcaYMuI4\nEntertainment- https://youtu.be/LXXkiUKDK4w            \n\n\nFor your favorite Bollywood Movies, Subscribe now: \nhttp://bit.ly/2VXistY\n\nSUBSCRIBE for the best Bollywood clips, movies and scenes all in your PEN Multiplex channel \nhttp://bit.ly/2QxELp3\n\nFor your favorite Gujarati Movies, Subscribe now: http://bit.ly/2VZm9Q6\n\nFor Action-Packed Movies, Click on the below link now...\nPen Action: http://bit.ly/2XaSS6t\n\n\nEnjoy & stay connected with us! \nLike us on Facebook: https://www.facebook.com/PenMovies\nFollow us on Twitter: https://twitter.com/penmovies\nFollow us on Instagram: https://www.instagram.com/penmovies 👉 Log On To Our Official Website : https://penstudios.in/",
    "channelName": "Pen Movies",
    "channelUrl": "https://www.youtube.com/channel/UC3ar28GS6o1p0m_wabfk2zw",
    "channelId": "UC3ar28GS6o1p0m_wabfk2zw",
    "inputChannelUrl": "https://www.youtube.com/PenMovies/shorts",
    "hashtags": [],
    "fromYTUrl": "https://www.youtube.com/watch?v=s9Pr4rydv_8",
    "tags": [],
    "durationAlt": "20s",
    "transcriptSample": []
  },
  {
    "title": "Why \"Retro\" Matters? | Suriya | Pooja Hegde | Karthik Subbaraj | Pen Studios | 1st May 25",
    "type": "shorts",
    "id": "wR4C5mCLxSM",
    "video_url": "https://www.youtube.com/watch?v=wR4C5mCLxSM",
    "thumbnailUrl": "https://i.ytimg.com/vi/wR4C5mCLxSM/maxres2.jpg?sqp=-oaymwEoCIAKENAF8quKqQMcGADwAQH4AbYIgAKAD4oCDAgAEAEYZSBVKEwwDw==&rs=AOn4CLDWPuQjNpaeeQk04mKXxSNtGgxrtQ",
    "viewCount": "19.2k",
    "viewCountRaw": 19178,
    "uploadDate": "2025-04-30T05:08:39-07:00",
    "duration": 51,
    "liveBroadcastDetails": {
      "live_status": "not_live",
      "is_live": false
    },
    "processingDetails": {
      "status": "processed",
      "playable_in_embed": true
    },
    "description": "With Suriya, it’s always magic — whether on the big screen or on satellite. The audience just can’t get enough,” says Dr. Jayantilal Gada.\nBook your ticket here - https://linktr.ee/RetroTheMovie\n\nEnjoy Super Hit Movies Collection :- \nAcharya - https://youtu.be/MZGKv-UqD4c\nJaya Janaki Nayaka KHOONKHAR - https://youtu.be/1lFk1LSsI_c \nSHER KA SHIKAAR - https://youtu.be/FE435iFT6jw \nTholi Prema -  https://youtu.be/J4nBcaYMuI4\nEntertainment- https://youtu.be/LXXkiUKDK4w            \n\n\nFor your favorite Bollywood Movies, Subscribe now: \nhttp://bit.ly/2VXistY\n\nSUBSCRIBE for the best Bollywood clips, movies and scenes all in your PEN Multiplex channel \nhttp://bit.ly/2QxELp3\n\nFor your favorite Gujarati Movies, Subscribe now: http://bit.ly/2VZm9Q6\n\nFor Action-Packed Movies, Click on the below link now...\nPen Action: http://bit.ly/2XaSS6t\n\n\nEnjoy & stay connected with us! \nLike us on Facebook: https://www.facebook.com/PenMovies\nFollow us on Twitter: https://twitter.com/penmovies\nFollow us on Instagram: https://www.instagram.com/penmovies 👉 Log On To Our Official Website : https://penstudios.in/",
    "channelName": "Pen Movies",
    "channelUrl": "https://www.youtube.com/channel/UC3ar28GS6o1p0m_wabfk2zw",
    "channelId": "UC3ar28GS6o1p0m_wabfk2zw",
    "inputChannelUrl": "https://www.youtube.com/PenMovies/shorts",
    "hashtags": [],
    "fromYTUrl": "https://www.youtube.com/watch?v=wR4C5mCLxSM",
    "tags": [
      "hindi songs 2025",
      "hindi songs new",
      "bollywood songs 2025",
      "bollywood movies 2025",
      "tseries",
      "tseries songs",
      "retro trailer hindi",
      "retro hindi trailer",
      "retro suriya",
      "suriya retro trailer hindi",
      "retro suriya movie",
      "suriya 44",
      "retro suriya 44",
      "retro trailer suriya",
      "retro movie trailer hindi",
      "suriya retro movie",
      "suriya 44 movie",
      "retro suriya 44 trailer",
      "suriya movie trailer",
      "retro movie trailer suriya",
      "retro movie pooja hegde",
      "suriya and pooja hegde movie",
      "suriya 44 trailer",
      "retro"
    ],
    "durationAlt": "51s",
    "transcriptSample": []
  },
  {
    "title": "The Story Behind Movie \"Retro\" | Karthik Subbaraj | Suriya | Pooja Hegde | Pen Studios | 1st May 25",
    "type": "shorts",
    "id": "psKewJxFXgc",
    "video_url": "https://www.youtube.com/watch?v=psKewJxFXgc",
    "thumbnailUrl": "https://i.ytimg.com/vi/psKewJxFXgc/maxres2.jpg?sqp=-oaymwEoCIAKENAF8quKqQMcGADwAQH4AbYIgAKAD4oCDAgAEAEYZSBdKEcwDw==&rs=AOn4CLA3JXOHia2mN6b427Bev_jyG0AaFg",
    "viewCount": "27.2k",
    "viewCountRaw": 27210,
    "uploadDate": "2025-04-30T05:08:39-07:00",
    "duration": 35,
    "liveBroadcastDetails": {
      "live_status": "not_live",
      "is_live": false
    },
    "processingDetails": {
      "status": "processed",
      "playable_in_embed": true
    },
    "description": "Watch Karthik Subbaraj talk about what makes this film truly special.\nBook your ticket here - https://linktr.ee/RetroTheMovie\n\nEnjoy Super Hit Movies Collection :- \nAcharya - https://youtu.be/MZGKv-UqD4c\nJaya Janaki Nayaka KHOONKHAR - https://youtu.be/1lFk1LSsI_c \nSHER KA SHIKAAR - https://youtu.be/FE435iFT6jw \nTholi Prema -  https://youtu.be/J4nBcaYMuI4\nEntertainment- https://youtu.be/LXXkiUKDK4w            \n\n\nFor your favorite Bollywood Movies, Subscribe now: \nhttp://bit.ly/2VXistY\n\nSUBSCRIBE for the best Bollywood clips, movies and scenes all in your PEN Multiplex channel \nhttp://bit.ly/2QxELp3\n\nFor your favorite Gujarati Movies, Subscribe now: http://bit.ly/2VZm9Q6\n\nFor Action-Packed Movies, Click on the below link now...\nPen Action: http://bit.ly/2XaSS6t\n\n\nEnjoy & stay connected with us! \nLike us on Facebook: https://www.facebook.com/PenMovies\nFollow us on Twitter: https://twitter.com/penmovies\nFollow us on Instagram: https://www.instagram.com/penmovies 👉 Log On To Our Official Website : https://penstudios.in/",
    "channelName": "Pen Movies",
    "channelUrl": "https://www.youtube.com/channel/UC3ar28GS6o1p0m_wabfk2zw",
    "channelId": "UC3ar28GS6o1p0m_wabfk2zw",
    "inputChannelUrl": "https://www.youtube.com/PenMovies/shorts",
    "hashtags": [],
    "fromYTUrl": "https://www.youtube.com/watch?v=psKewJxFXgc",
    "tags": [],
    "durationAlt": "35s",
    "transcriptSample": []
  },
  {
    "title": "The Magic of \"Retro\" | Suriya | Pooja Hegde | Karthik Subbaraj | Pen Studios | 1st May 25",
    "type": "shorts",
    "id": "1cNBR6iW1-A",
    "video_url": "https://www.youtube.com/watch?v=1cNBR6iW1-A",
    "thumbnailUrl": "https://i.ytimg.com/vi/1cNBR6iW1-A/maxresdefault.jpg?sqp=-oaymwEmCIAKENAF8quKqQMa8AEB-AG2CIACgA-KAgwIABABGHIgSCgzMA8=&rs=AOn4CLBJ76T8vZ2He3QBvKkhFPTA6a8mvg",
    "viewCount": "30.8k",
    "viewCountRaw": 30839,
    "uploadDate": "2025-04-30T05:08:39-07:00",
    "duration": 107,
    "liveBroadcastDetails": {
      "live_status": "not_live",
      "is_live": false
    },
    "processingDetails": {
      "status": "processed",
      "playable_in_embed": true
    },
    "description": "🎬 Suriya spills the beans on what it was like shooting for this time-travelling ride!\nBook your ticket here - https://linktr.ee/RetroTheMovie\n\nEnjoy Super Hit Movies Collection :- \nAcharya - https://youtu.be/MZGKv-UqD4c\nJaya Janaki Nayaka KHOONKHAR - https://youtu.be/1lFk1LSsI_c \nSHER KA SHIKAAR - https://youtu.be/FE435iFT6jw \nTholi Prema -  https://youtu.be/J4nBcaYMuI4\nEntertainment- https://youtu.be/LXXkiUKDK4w            \n\n\nFor your favorite Bollywood Movies, Subscribe now: \nhttp://bit.ly/2VXistY\n\nSUBSCRIBE for the best Bollywood clips, movies and scenes all in your PEN Multiplex channel \nhttp://bit.ly/2QxELp3\n\nFor your favorite Gujarati Movies, Subscribe now: http://bit.ly/2VZm9Q6\n\nFor Action-Packed Movies, Click on the below link now...\nPen Action: http://bit.ly/2XaSS6t\n\n\nEnjoy & stay connected with us! \nLike us on Facebook: https://www.facebook.com/PenMovies\nFollow us on Twitter: https://twitter.com/penmovies\nFollow us on Instagram: https://www.instagram.com/penmovies 👉 Log On To Our Official Website : https://penstudios.in/",
    "channelName": "Pen Movies",
    "channelUrl": "https://www.youtube.com/channel/UC3ar28GS6o1p0m_wabfk2zw",
    "channelId": "UC3ar28GS6o1p0m_wabfk2zw",
    "inputChannelUrl": "https://www.youtube.com/PenMovies/shorts",
    "hashtags": [],
    "fromYTUrl": "https://www.youtube.com/watch?v=1cNBR6iW1-A",
    "tags": [
      "hindi songs 2025",
      "hindi songs new",
      "bollywood songs 2025",
      "bollywood movies 2025",
      "tseries",
      "tseries songs",
      "retro trailer hindi",
      "retro hindi trailer",
      "retro suriya",
      "suriya retro trailer hindi",
      "retro suriya movie",
      "suriya 44",
      "retro suriya 44",
      "retro trailer suriya",
      "retro movie trailer hindi",
      "suriya retro movie",
      "suriya 44 movie",
      "retro suriya 44 trailer",
      "suriya movie trailer",
      "retro movie trailer suriya",
      "retro movie pooja hegde",
      "suriya and pooja hegde movie",
      "suriya 44 trailer",
      "retro"
    ],
    "durationAlt": "107s",
    "transcriptSample": []
  },
]

Field reference

Video metadata

Field	Type	Description
title	string	Video title
video_id	string	YouTube video ID
uploader	string	null
upload_date	string	null
publish_date	string	null
duration	integer	null
duration_str	string	null
description	string	null
categories	array	null
tags	array	null
thumbnail	string	null
url	string	Video URL
subtitles	object	null

Engagement (raw + humanized)

Field	Type	Description
view_count	integer	null
view_count_str	string	null
like_count	integer	null
like_count_str	string	null
comment_count	integer	null
comment_count_str	string	null
dislike_count	integer	null

Channel enrichment

Field	Type	Description
channel_name	string	null
channel_url	string	null
channel_id	string	null
channel_subscriber_count	integer	null
channel_subscriber_count_str	string	null
about	string	null
tiktok	string	null
instagram	string	null
x	string	null

---

Troubleshooting checklist (fast)

• ✅ Did you pick the right proxy group? (RESIDENTIAL often works best for YouTube scraping.)
• ✅ Check your Apify proxy credits / permissions.
• ✅ If using custom proxy, ensure it accepts HTTPS and supports SNI.
• ✅ Check actor logs for failed to fetch entries — these contain the exception message showing whether it was timeout, TLS, or HTTP error.
• ✅ If you get few or zero videoIds, try enabling Playwright-based scraping (outside scope of this actor) or increase maxshorts and retries.

---