Youtube Search Results Scraper

Overview

Unlock the power of YouTube's vast content library with our Search Results Scraper! Extract valuable data from search results, including video details, channel information, and more. Perfect for content creators, marketers, and researchers looking to analyze trends, track competitors, or discover new content opportunities.

How it works

This actor allows you to scrape search results from YouTube, extracting important details such as video titles, thumbnails, channel information, and engagement metrics. It provides valuable insights into content trends and popularity across various topics on the platform.

Features

• Flexible Input: Supports both YouTube search URLs and keywords/search terms.
• Detailed Video Information: Extracts comprehensive data about each video in the search results.
• Channel Details: Retrieves information about the channels associated with the videos.
• Engagement Metrics: Captures view counts, video durations, and other relevant statistics.
• Customizable: Flexible configuration options to suit various scraping needs.
• Proxy Support: Built-in proxy configuration to enhance scraping reliability and avoid blocks.

How to Use

1. Set Up: Ensure you have an Apify account and access to the Apify platform.
2. Configure Input: Set the YouTube search URL(s) or keyword(s) you want to scrape results for (see Input Data section).
3. (Optional) Adjust additional parameters like maxItems, concurrency, and proxy settings.
4. Run the Scraper: Execute the scraper on the Apify platform.
5. Data Collection: The scraper will output data in your chosen format, which can include JSON, CSV, or others as supported by Apify.

Input Configuration

Here's an example of how to set up the input for the YouTube Search Results Scraper:

1{
2    "startUrls": [
3        "music",
4        "https://www.youtube.com/results?search_query=medical"
5    ],
6    "maxItems": 100,
7    "maxConcurrency": 10,
8    "minConcurrency": 1,
9    "maxRequestRetries": 3,
10    "proxy": {
11        "useApifyProxy": true,
12        "apifyProxyGroups": [
13            "RESIDENTIAL"
14        ],
15        "apifyProxyCountry": "US"
16    }
17}

Input Fields Explanation

• startUrls: Array of YouTube search URLs or keywords/search terms.
• maxItems: Maximum number of search result items to scrape (default: 100).
• maxConcurrency: Maximum number of pages processed simultaneously (default: 10).
• minConcurrency: Minimum number of pages processed simultaneously (default: 1).
• maxRequestRetries: Number of retries for failed requests (default: 3).
• proxy: Proxy configuration for enhanced scraping reliability.

Output Structure

The output data includes detailed information about each search result item. The structure can vary depending on the type of result (video, playlist, channel, etc.). Here's a more comprehensive breakdown of the output structure, focusing on a playlist item:

1{
2    "radioRenderer": {
3        "playlistId": "RDQMsHhx03c4Dwk",
4        "title": {
5            "simpleText": "Mix - Music"
6        },
7        "thumbnail": {
8            "thumbnails": [
9                {
10                    "url": "https://i.ytimg.com/vi/859u6uiZdds/hqdefault.jpg?sqp=-oaymwEWCKgBEF5IWvKriqkDCQgBFQAAiEIYAQ==&rs=AOn4CLAEZ8uHdyBRjh16O5H9eUfB5hTotQ",
11                    "width": 168,
12                    "height": 94
13                },
14                // ... more thumbnail sizes ...
15            ],
16            "sampledThumbnailColor": {
17                "red": 89,
18                "green": 66,
19                "blue": 53
20            }
21        },
22        "videoCountText": {
23            "runs": [
24                {
25                    "text": "50+ videos"
26                }
27            ]
28        },
29        "navigationEndpoint": {
30            "clickTrackingParams": "CLcDELs3GAAiEwj558GAhd6IAxVIcA8CHV5aOmUyBnNlYXJjaA==",
31            "commandMetadata": {
32                "webCommandMetadata": {
33                    "url": "/watch?v=859u6uiZdds&list=RDQMsHhx03c4Dwk&start_radio=1",
34                    "webPageType": "WEB_PAGE_TYPE_WATCH",
35                    "rootVe": 3832
36                }
37            },
38            "watchEndpoint": {
39                "videoId": "859u6uiZdds",
40                "playlistId": "RDQMsHhx03c4Dwk",
41                "params": "OALAAQE%3D",
42                "continuePlayback": true,
43                "loggingContext": {
44                    "vssLoggingContext": {
45                        "serializedContextData": "Gg9SRFFNc0hoeDAzYzREd2s%3D"
46                    }
47                }
48            }
49        },
50        "videos": [
51            {
52                "childVideoRenderer": {
53                    "title": {
54                        "simpleText": "Tujamo - Down (Official Music Video)"
55                    },
56                    "lengthText": {
57                        "accessibility": {
58                            "accessibilityData": {
59                                "label": "3 minutes, 15 seconds"
60                            }
61                        },
62                        "simpleText": "3:15"
63                    },
64                    "videoId": "859u6uiZdds"
65                }
66            },
67            // ... more videos ...
68        ],
69        "longBylineText": {
70            "simpleText": "C. Tangana, Öwnboss, ELYAZ, and more"
71        },
72        "thumbnailOverlays": [
73            {
74                "thumbnailOverlayBottomPanelRenderer": {
75                    "text": {
76                        "simpleText": "Mix"
77                    },
78                    "icon": {
79                        "iconType": "MIX"
80                    }
81                }
82            },
83            // ... more overlays ...
84        ]
85    }
86}

Field Explanations

• radioRenderer: Contains information about a YouTube Mix playlist.
  • playlistId: Unique identifier for the playlist.
  • title: Title of the playlist.
    • simpleText: Plain text of the title.
  • thumbnail: Information about the playlist's thumbnail image.
    • thumbnails: Array of thumbnail objects with different sizes.
      • url: URL of the thumbnail image.
      • width: Width of the thumbnail in pixels.
      • height: Height of the thumbnail in pixels.
    • sampledThumbnailColor: Average color of the thumbnail, useful for UI design.
  • videoCountText: Indicates the number of videos in the playlist.
  • navigationEndpoint: Contains data for navigating to the playlist.
    • clickTrackingParams: Used by YouTube for analytics.
    • commandMetadata: Metadata for the navigation action.
    • watchEndpoint: Specific data for starting playback.
      • videoId: ID of the first video in the playlist.
      • playlistId: ID of the playlist to be played.
      • params: Additional parameters for playback.
  • videos: Array of video objects in the playlist.
    • childVideoRenderer: Contains information about each video.
      • title: Title of the video.
      • lengthText: Duration of the video.
        • accessibility: Accessible version of the duration.
        • simpleText: Formatted duration string.
      • videoId: Unique identifier for the video.
  • longBylineText: Description of the playlist content, often listing featured artists.
  • thumbnailOverlays: Array of overlay information displayed on the thumbnail.
    • thumbnailOverlayBottomPanelRenderer: Information for the bottom panel overlay.
      • text: Text displayed in the overlay.
      • icon: Icon type used in the overlay.

This structure provides a comprehensive view of a playlist item in YouTube search results. It includes details about the playlist itself, its thumbnail, navigation data, and information about the videos contained within the playlist. The actual output may include additional fields or vary slightly depending on the specific type of search result (e.g., individual videos, channels, or other content types).

Support

• For issues or feature requests, please use the Issues section of this actor.
• If you need customization or have questions, feel free to contact the author:
  • Author's website: https://muhamed-didovic.github.io/
  • Email: muhamed.didovic@gmail.com

Additional Services

• Request customization or whole dataset: muhamed.didovic@gmail.com
• If you need anything else scraped, or this actor customized, email: muhamed.didovic@gmail.com
• For API services of this scraper (no Apify fee, just usage fee for the API), contact: muhamed.didovic@gmail.com