# Bilibili Video Transcript & Metadata Extractor (`scrapers-hub/bilibili-video-transcript-api`) Actor Bilibili Video Transcript API extracts transcripts and subtitles from public Bilibili videos, making video content searchable and easy to analyze ๐Ÿ“๐ŸŽฅ Perfect for content analysis, research, accessibility, AI workflows, and multilingual processing. - **URL**: https://apify.com/scrapers-hub/bilibili-video-transcript-api.md - **Developed by:** [Scrapers Hub](https://apify.com/scrapers-hub) (community) - **Categories:** AI, Videos, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $1.99 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ### ๐Ÿ“น Bilibili Video Transcript & Metadata Extractor --- #### ๐Ÿ’ก Common Use Cases & Applications This actor is designed to enable various data workflows. Some of the primary use cases include: 1. **LLM Summarization & QA Platforms**: Extract transcripts from Bilibili educational videos, tutorials, or lectures and feed them into LLMs (like GPT, Claude, or local LLMs) for bullet-point summaries, key takeaway extraction, or building Q&A systems. 2. **Content Archiving & SEO Indexing**: Save video scripts, uploader details, metadata statistics, and direct CDN URLs to local databases for media archiving, search engine indexing, or historical tracking of engagement metrics. 3. **Language Learning & Translation**: Leverage dual-track subtitles and speech-to-text outputs to build bilingual corpora, generate transcripts for regional accents, or validate machine translation models against spoken audio. 4. **Competitor & Content Trend Analytics**: Scrape statistics such as views, likes, danmaku counts, coins, and replies over time to track audience engagement, analyze creators' content strategies, and discover trending topics. --- ### 1. Introduction and Overview ๐ŸŒŸ Welcome to the official developer documentation for the **Bilibili Video Transcript & Metadata Extractor**! ๐Ÿ“น Bilibili is one of the most prominent video-sharing platforms in China, famous for its interactive video playback experience, bullet-chat (Danmaku) culture, and an extensive repository of user-generated content ranging from tech tutorials, anime, gaming guides, to lifestyle vlogs. As the platform grows, the demand for extracting textual content from Bilibili videos has increased exponentially. Whether you are building an LLM-based summarization application, training translation models, archiving educational video content, or performing deep visual-textual analytics, accessing high-quality subtitles and metadata is a crucial prerequisite. The **Bilibili Video Transcript & Metadata Extractor** is a high-performance, containerized scraper built as an Apify Actor. It provides a robust, dual-strategy extraction pipeline designed to work under production loads: - **Primary Strategy (API Scraping):** It queries Bilibili's internal endpoints to retrieve official developer-uploaded or auto-generated subtitle tracks (e.g., in Simplified Chinese `zh-Hans`, Traditional Chinese `zh-Hant`, etc.). This process is incredibly fast, returning results in milliseconds. - **Secondary Strategy (Whisper ASR Fallback):** If no subtitles are available on the video, the actor triggers a local speech-to-text pipeline. It downloads the highest-quality audio track using `yt-dlp`, dynamically locates the `ffmpeg` binary via `imageio-ffmpeg` to process the audio stream, loads OpenAIโ€™s Whisper `tiny` neural network model, and transcribes the audio locally within the container. In addition to transcripts, the actor gathers comprehensive metadata about the video, including view count, likes, shares, danmaku count, coins, replies, favorites, publish times, high-resolution thumbnail URLs, video author (uploader) profile information (avatars and names), and direct CDN URLs for streaming audio and video. By combining direct HTTP requests, proxy rotation (via Apify Proxy), and on-device Whisper machine learning, this actor delivers a highly resilient scraping solution that handles rate-limiting and missing data natively. --- ### 2. Key Features and Workflows ๐Ÿ› ๏ธ The **Bilibili Video Transcript & Metadata Extractor** is designed to handle complex edge cases and provide a smooth, reliable extraction pipeline. Below is a detailed breakdown of its core features and internal workflows. ```mermaid graph TD A[Start Actor] --> B[Parse Input URL & proxyConfiguration] B --> C[Extract BVID from URL] C --> D[Fetch Video Metadata from Bilibili API] D --> E{Metadata Fetch Success?} E -- No --> F[Log Error & Terminate] E -- Yes --> G[Extract Stream URLs using yt-dlp] G --> H{Are Official Subtitles Available?} H -- Yes --> I[Download Subtitle JSON & Parse Segments] H -- No --> J[Download Best Audio Stream with yt-dlp] J --> K[Locate & Configure ffmpeg PATH] K --> L[Load Whisper tiny Model] L --> M[Transcribe Audio locally & generate Segments] I --> N[Combine Metadata, Stream URLs, and Subtitles/Segments] M --> N N --> O[Push Result JSON to Apify Dataset] O --> P[End Actor] ```` #### 2.1 Comprehensive Metadata Extraction ๐Ÿ“Š The actor scrapes a wide array of metadata fields directly from the Bilibili platform. Unlike simple scrapers that only extract the title, our tool accesses deep statistic arrays: - **BVID, AID, CID Resolution:** It resolves the modern alphanumeric Bilibili Video ID (`BVID`), the legacy integer Archive ID (`AID`), and the specific Part/Page ID (`CID`). This makes the data fully compatible with legacy APIs and third-party tools. - **Interaction Statistics:** It captures the complete popularity metricsโ€”views, danmaku (bullet chat count), comments (replies), favorites, virtual coins (inserted by fans), shares, and likes. These metrics are critical for analytics and virality modeling. - **Uploader (Owner) Profiles:** Retrieves the uploader's user name (`nickname`) and their avatar URI (`avatarUri`), allowing you to map content back to specific creators. - **Video Details:** Extracts the high-res cover image (`img`), publication/creation timestamp (`createTime`), raw description text (`desc`), title, and duration in seconds. #### 2.2 Official Subtitle Fetching API ๐Ÿ“ Bilibili supports bilingual and creator-provided subtitles. 1. The actor inspects the `subtitle` array returned by the Bilibili view API. 2. It filters for preferred languages, prioritizing Chinese Simplified (`zh-Hans` / `zh-CN`) or falling back to the first available language track. 3. Once the subtitle URL is retrieved, the actor issues an authenticated GET request to download Bilibili's native JSON subtitle format. 4. It loops through the subtitle timeline body, converting the time formats into seconds and building a clean array of objects containing `start`, `end`, and `text` segments. #### 2.3 Local Whisper Automatic Speech Recognition (ASR) Fallback ๐Ÿค– If the uploader did not submit subtitles and Bilibili's automatic translation isn't available, the actor falls back to local machine learning. 1. **yt-dlp Extraction:** The actor isolates the audio track from the Bilibili page and downloads it directly to a temporary directory. 2. **FFmpeg Pipeline:** Whisper requires `ffmpeg` to load and decode audio files. The actor uses `imageio-ffmpeg` to locate the platform-specific `ffmpeg` executable. If the host machine doesn't have `ffmpeg` installed globally, it copies the executable to a temporary binary folder and dynamically inserts it into the system's `PATH` environment variable. 3. **Whisper Model Loader:** The actor loads OpenAIโ€™s `whisper` model. By default, the `tiny` model is selected. It provides a fantastic balance of speed, low memory footprint (perfect for serverless containers with 1-2GB RAM), and acceptable transcription quality. 4. **Local Inference:** The audio file is transcribed. Whisper automatically detects the spoken language, segments the audio based on silences, and translates/transcribes the voice into structured segments containing timestamps. #### 2.4 Streaming URL Extraction ๐ŸŽฌ For offline archiving, media player integrations, or secondary audio analyses, the actor uses a background `yt-dlp` context to fetch the direct streaming URLs of the raw video and audio files from Bilibiliโ€™s content delivery network (CDN). Because these links are direct CDN URLs, they are highly time-sensitive and are meant to be consumed immediately upon execution. #### 2.5 Proxy Support and Anti-Blocking Measures ๐Ÿ” Bilibili employs strict rate-limiting on its public APIs, returning status code `412` (Precondition Failed) or IP bans if multiple requests originate from the same IP address. The actor is fully integrated with Apify Proxy, allowing you to pass residential proxy parameters. This rotates the request IP for every execution, ensuring high reliability under heavy load. *** ### 3. Input and Output Specification ๐Ÿ“ฅ๐Ÿ“ค > \[!IMPORTANT] > This section outlines the structural formats of inputs accepted by the actor and the JSON schema returned upon successful execution. #### 3.1 Input Parameters Configuration The actor accepts JSON inputs containing the Bilibili URL to process and proxy configuration. | Field Name | Type | Required | Default Value | Description | | :--- | :--- | :--- | :--- | :--- | | `videoUrl` | `String` | **Yes** | `https://www.bilibili.com/video/BV1ojn8z1EUq` | The full HTTP URL of the Bilibili video to scrape. Supports standard formats (e.g. `https://www.bilibili.com/video/BV...`). | | `proxyConfiguration` | `Object` | No | `{"useApifyProxy": true}` | Proxy configuration object. Residential proxies are highly recommended to prevent rate limits and Bilibili `412` errors. | ##### Input JSON Example ```json { "videoUrl": "https://www.bilibili.com/video/BV1ojn8z1EUq", "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } } ``` *** #### 3.2 Output Data Schema The **Bilibili Video Transcript & Metadata Extractor** outputs results to the Apify dataset as a JSON array. Below is the list of fields returned in the output object: | Field Name | Type | Description | | :--- | :--- | :--- | | `url` | `String` | The original target URL processed by the scraper. | | `bvid` | `String` | The unique Bilibili Video ID alphanumeric identifier (starts with `BV`). | | `aid` | `Integer` | The legacy Archive ID (AV number numerical representation). | | `cid` | `Integer` | The internal Part/Page ID of the specific video file. | | `title` | `String` | The official title of the video. | | `desc` | `String` | The uploader's video description text. | | `createTime` | `Integer` | Unix epoch timestamp indicating when the video was published. | | `img` | `String` | URL pointing to the video's cover image / thumbnail. | | `view` | `Integer` | Total view count at the time of scraping. | | `danmaku` | `Integer` | Total number of bullet comments (danmaku) sent by viewers. | | `reply` | `Integer` | The number of root-level and nested replies in the comment section. | | `favorite` | `Integer` | The count of users who saved the video to their personal favorites folders. | | `coin` | `Integer` | The total number of virtual coins donated to the video creator. | | `share` | `Integer` | The number of times the video has been shared across social media. | | `like` | `Integer` | Total count of positive feedback (likes) received. | | `duration` | `Integer` | The total duration of the video in seconds. | | `nickname` | `String` | The display name of the video creator / uploader. | | `avatarUri` | `String` | The URL profile image of the video creator / uploader. | | `videoUrl` | `String` | A temporary direct CDN video stream URL (no audio). | | `audioUrl` | `String` | A temporary direct CDN audio stream URL (no video). | | `text` | `String` | The complete, flattened transcription or subtitle text. | | `segments` | `Array` | An array of time-aligned transcript segments containing timestamps. | Each element inside the `segments` array has the following structure: - `start` (`Float`): The starting timestamp in seconds. - `end` (`Float`): The ending timestamp in seconds. - `text` (`String`): The text spoken or displayed during this time frame. ##### Output JSON Example Below is an example of the structured JSON data returned by the scraper. ```json [ { "url": "https://www.bilibili.com/video/BV1ojn8z1EUq", "bvid": "BV1ojn8z1EUq", "aid": 1130541764, "cid": 1658421045, "title": "Bilibili Web API Development Tutorial and Best Practices", "desc": "In this guide, we dive deep into scraping, processing, and analyzing video feeds from modern platform systems. Source code and guidelines are shared.", "createTime": 1716124800, "img": "https://i0.hdslb.com/bfs/archive/a1b2c3d4e5f6g7h8i9j0.jpg", "view": 128400, "danmaku": 1540, "reply": 842, "favorite": 5301, "coin": 2180, "share": 750, "like": 10240, "duration": 345, "nickname": "TechDeveloperCN", "avatarUri": "https://i0.hdslb.com/bfs/face/member_avatar.jpg", "videoUrl": "https://upos-sz-mirrorcos.bilivideo.com/upgcxcode/45/10/1658421045/1658421045-1-30080.m4s?e=ig8epXgD61F6hwdhiSgVhwd17WdHhwdV7W...", "audioUrl": "https://upos-sz-mirrorcos.bilivideo.com/upgcxcode/45/10/1658421045/1658421045-1-30280.m4s?e=ig8epXgD61F6hwdhiSgVhwd17WdHhwdV7W...", "text": "Hello everyone. Today we are going to talk about Web API structures. We will show you how to write robust client applications.", "segments": [ { "start": 0.0, "end": 3.25, "text": "Hello everyone." }, { "start": 3.25, "end": 6.8, "text": "Today we are going to talk about Web API structures." }, { "start": 6.8, "end": 10.5, "text": "We will show you how to write robust client applications." } ] } ] ``` *** ### 4. Deep-Dive Technical Architecture ๐Ÿ—๏ธ The codebase is built cleanly with modular Python scripts. The logic is separated into a local execution script (`app.py`) for off-platform developers and a production entry point (`src/main.py`) optimized for the Apify platform runtime. #### 4.1 Bilibili HTTP Metadata Resolution The scraper makes an HTTP GET request to: `https://api.bilibili.com/x/web-interface/view?bvid={BVID}` To avoid headers detection, the actor sets: - `User-Agent`: A modern browser user agent string (Chrome/Windows). - `Referer`: `https://www.bilibili.com/` (Crucial for Bilibili, as they block requests lacking referrer headers). The returned JSON payload is parsed. If the Bilibili server returns a response code other than `0` (e.g. `-400` for bad parameters, `-403` for geoblocks, `-412` for rate limiting), the script exits with descriptive logs. #### 4.2 Official Subtitle Download Pipeline When subtitle tracks are defined, Bilibili embeds their config in the `subtitle` array inside the `data` node of the JSON response: ```json "subtitle": { "allow_submit": true, "list": [ { "id": 123456789, "lan": "zh-Hans", "lan_doc": "ไธญๆ–‡๏ผˆ็ฎ€ไฝ“๏ผ‰", "is_lock": false, "subtitle_url": "//subtitle.hdslb.com/bfs/subtitle/a1b2c3d4.json", "type": 1, "ai_type": 0 } ] } ``` The scraper iterates through this list, matching the `lan` tag. It formats the URL (appending `https:` if it begins with double slashes) and queries the JSON. Inside Bilibiliโ€™s subtitle files, content segments are modeled with `from` and `to` timestamps representing seconds. The scraper extracts these, rounds them to 2 decimal places, and maps them to a normalized output format. #### 4.3 Local Whisper ASR Offline Transcriber When official transcripts are not present, the machine learning workflow is initialized. ##### Audio Downloader (yt-dlp) The script configures `yt-dlp` in python code using direct options: - `format`: `bestaudio/best` ensures we extract the lightweight audio stream rather than downloading the entire heavy video file. This conserves bandwidth, increases speed, and keeps the container disk space usage low. - `outtmpl`: Audio is written to a secure temporary folder resolved using python's `tempfile` module. ##### FFmpeg Path Resolver Whisper requires `ffmpeg` to process audio files. Since local environments and Docker containers differ, we employ a dynamic search mechanism. The actor imports `imageio_ffmpeg` and calls `imageio_ffmpeg.get_ffmpeg_exe()`. If the executable isn't globally available, it is copied to a temp directory and appended to `os.environ["PATH"]`. ```python def get_ffmpeg_path(temp_dir): try: import imageio_ffmpeg imageio_ffmpeg_exe = imageio_ffmpeg.get_ffmpeg_exe() exe_name = "ffmpeg.exe" if os.name == "nt" else "ffmpeg" ffmpeg_exe_path = os.path.join(temp_dir, exe_name) if not os.path.exists(ffmpeg_exe_path): shutil.copy(imageio_ffmpeg_exe, ffmpeg_exe_path) if os.name != "nt": os.chmod(ffmpeg_exe_path, 0o755) return temp_dir except Exception as e: # Falls back to system ffmpeg if copy fails return None ``` ##### Whisper Model Execution We load the `tiny` model: ```python import whisper model = whisper.load_model("tiny") result = model.transcribe(audio_output_path) ``` The transcription result returns a list of dictionaries representing segments. These contain start, end, and text fields. Using `tiny` keeps memory footprint low (approx 150MB-250MB VRAM/RAM) while still giving high accuracy on Chinese and English speeches. *** ### 5. Local Development and Setup ๐Ÿ’ป If you want to run the Bilibili Video Comments Extractor on your local machine for debugging, development, or testing, follow this step-by-step guide. #### 5.1 Prerequisites - **Python 3.11** or higher. - **PIP** (Python Package Installer). - **FFmpeg**: Although `imageio-ffmpeg` is configured, having `ffmpeg` installed globally on your operating system is highly recommended. - *Windows*: Install via Chocolatey (`choco install ffmpeg`) or download the binaries from the official site and add them to your system PATH. - *macOS*: Install via Homebrew (`brew install ffmpeg`). - *Linux*: Install via apt (`sudo apt-get update && sudo apt-get install ffmpeg -y`). #### 5.2 Step-by-Step Installation 1. **Clone or Copy the Files:** Extract all the repository files into your local directory. 2. **Create a Virtual Environment:** Navigate to the project root and create a virtual environment to avoid dependency conflicts: ```bash python -m venv venv ``` 3. **Activate the Virtual Environment:** - *Windows (CMD/PowerShell)*: ```powershell .\venv\Scripts\activate ``` - *macOS/Linux*: ```bash source venv/bin/activate ``` 4. **Install Dependencies:** Run pip with the requirements file: ```bash pip install -r requirements.txt ``` This command downloads: - `torch` (CPU version specified via the extra index link to keep the image lightweight). - `openai-whisper` for machine learning transcription. - `apify` for platform integration. - `requests` for API scraping. - `yt-dlp` for media stream downloading. - `imageio-ffmpeg` for path configurations. #### 5.3 Running the Local Script The **Bilibili Video Transcript & Metadata Extractor** project includes a standalone file `app.py` built for local testing. It does not require Apify SDK environments or credentials. 1. Open `app.py` in your text editor. 2. Look at the top of the file where the hardcoded input is defined: ```python # 1. Direct hardcoded input at the top input_data = { "videoUrl": "https://www.bilibili.com/video/BV1ojn8z1EUq" } ``` 3. Replace the URL with the Bilibili video link you wish to test. 4. Execute the script: ```bash python app.py ``` 5. **Output Review:** The script prints the JSON result directly to your terminal. Additionally, it writes the result to `output.json` in the current working directory. You can inspect `output.json` to verify the structure, transcription text, and segment timelines. *** ### 6. Deployment and Apify Hosting ๐Ÿš€ > \[!TIP] > Deploying to the Apify platform allows you to scale this crawler, run it periodically via schedulers, hook it to webhooks, or expose it as a custom API. #### 6.1 Project Configuration Files Three files configure the environment for the **Bilibili Video Transcript & Metadata Extractor** on the Apify platform: 1. **`.actor/actor.json`**: This defines the metadata of the actor, the input schemas, user options, and proxy editors. 2. **`Dockerfile`**: Defines the virtual environment container settings. Let's look at how the Dockerfile is structured: ```dockerfile FROM apify/actor-python:3.11 # Install system dependencies (like ffmpeg for Whisper audio processing) USER root RUN apt-get update && apt-get install -y ffmpeg && rm -rf /var/lib/apt/lists/* USER myuser COPY requirements.txt ./ RUN pip install --no-cache-dir -r requirements.txt COPY . ./ CMD ["python", "-m", "src.main"] ``` *Why do we run `apt-get install -y ffmpeg`?* While the fallback helper can run binary copies, installing system-level ffmpeg in the Linux container guarantees smooth Whisper decoding without permission or path failures. 3. **`requirements.txt`**: Specifies PyTorch CPU libraries: ```txt --extra-index-url https://download.pytorch.org/whl/cpu torch openai-whisper apify>=1.6.0 requests yt-dlp imageio-ffmpeg ``` By explicitly requesting PyTorch CPU builds (`--extra-index-url https://download.pytorch.org/whl/cpu`), we prevent Docker from downloading the default CUDA GPU builds, reducing the final Docker image size from ~5GB down to ~1.2GB. #### 6.2 Deploying via CLI To deploy this actor, use the official Apify CLI. 1. Install Apify CLI globally: ```bash npm install -g apify-cli ``` 2. Log in to your Apify Account: ```bash apify login ``` (Enter your API token when prompted). 3. Run the actor locally in an Apify simulated environment to verify integration: ```bash apify run ``` 4. Deploy to the Apify cloud platform: ```bash apify push ``` This command packages your directory, uploads it to Apify, builds the Docker image on their servers, and deploys it under your account. Once built, you can trigger the actor via API calls, scheduler templates, or the online Console. *** ### 7. Troubleshooting, Rate Limits, and Best Practices โš ๏ธ When using the **Bilibili Video Transcript & Metadata Extractor** to scrape data at scale, you might encounter issues. Here are common problems and recommended solutions. #### 7.1 Bilibili Code 412 Errors and Rate Limits - **Symptom:** The API request returns `"code": -412` or an HTTP status code `412` with a message like `"่ฏทๆฑ‚่ขซๆ‹ฆๆˆช"` (Request intercepted) or `"่ฎฟ้—ฎ่ขซๆ‹’็ป"` (Access denied). - **Cause:** Bilibili has detected that your IP address is sending too many requests in a short period. - **Solutions:** - **Residential Proxies:** Enable Apify Proxy in your input settings and specify residential IP ranges. Bilibili rarely blocks residential IPs. - **Request Spacing:** If you are running multiple actors or iterating over many links, add random delays (e.g., `time.sleep(random.uniform(2, 5))`) between requests. - **Referer Headers:** Ensure your request headers include `"Referer": "https://www.bilibili.com/"`. Bilibili's CDN and API servers immediately drop requests that do not specify a valid referer. #### 7.2 Whisper Out-Of-Memory (OOM) Errors - **Symptom:** The actor crashes during the transcription phase, showing logs such as `Killed` or `Memory Limit Exceeded`. - **Cause:** Whisper models require significant memory. Even the `tiny` model can consume up to 1GB of RAM during audio decoding and inference. - **Solutions:** - **Increase Memory Allocation:** Go to your actor settings in the Apify Console and set the memory allocation to **at least 2048 MB (2 GB)**. - **CPU Limitation:** Running on CPU requires memory overhead. Do not run the actor with 512MB RAM if you expect it to transcribe videos using the Whisper fallback strategy. #### 7.3 Long Execution Timeout Issues - **Symptom:** The actor takes a very long time to complete and is terminated by Apify's default timeout limits. - **Cause:** Transcription of long videos (e.g., videos over 1 hour) on CPU via Whisper is slow. - **Solutions:** - Check if the video has official subtitles. If it does, make sure the API requests are succeeding so the script doesn't fall back to Whisper. - Adjust the Apify Timeout limit on the run configuration page to allow up to 10-15 minutes of execution for long-form content. *** ### 8. FAQ & Reference Guide ๐Ÿ’ก Here is a collection of frequently asked questions to help developers implement the scraper effectively. #### Q1: Can this scraper extract subtitles in languages other than Chinese? Yes. Bilibili creators can upload multiple subtitles. While the script defaults to searching for Simplified Chinese (`zh-Hans` / `zh-CN`), it will fall back to the first available language in the subtitle list if Chinese is missing. If you want to force English or Japanese tracks, you can modify the search sequence in `src/main.py` where it inspects `subtitle_list`. #### Q2: What happens if a Bilibili video is restricted or geoblocked? Bilibili geoblocks certain content (e.g., anime series, regional shows) to users inside mainland China. If the target video is geoblocked, requests made from US or European IP addresses will fail. To bypass this, configure your Apify Proxy settings to route requests through Chinese proxy servers. #### Q3: Why does yt-dlp fail to extract video streams occasionally? Bilibili frequently updates its video signature algorithms and video player code. When this happens, old versions of `yt-dlp` might fail to resolve the formats. The solution is to ensure your `requirements.txt` pulls the latest version of `yt-dlp`. You can also force a build update on your Apify console to rebuild the Docker cache with updated libraries. #### Q4: Is there a way to use a larger Whisper model (e.g., base, small, medium)? Yes, but you must be aware of the hardware constraints. You can modify the load parameter in `src/main.py`: `model = whisper.load_model("base")` Note that larger models take longer to download on launch, require significantly more RAM (e.g., 4GB+ for `small` / `medium`), and run much slower on CPUs. For serverless deployments, `tiny` is the most cost-effective and practical choice. #### Q5: How accurate are the timestamps in Whisper output? Whisper segment timestamps are highly accurate (usually within 50-100 milliseconds). However, Whisper segments audio based on natural speaking pauses. Sometimes, sentences might be split differently compared to manual subtitles, but the overall alignment remains fully synchronized. #### Q6: Can I parse list formats like Bilibili Playlists (Bw/ep)? Currently, this actor is designed to process individual video URLs. To parse lists, write a wrapper script that uses the Bilibili playlist APIs to retrieve all individual video URLs, then dispatch them to this actor in parallel or sequentially. #### Q7: Are the direct CDN URLs (videoUrl and audioUrl) permanent? No. Bilibili stream URLs returned in the metadata contain verification tokens in the query string (`?e=ig8ep...`). These signatures expire after a few hours (typically 2-6 hours). These URLs are designed for immediate consumption, processing, or downloading. *** ### 9. Conclusion & License ๐Ÿ† The **Bilibili Video Transcript & Metadata Extractor** represents a robust developer tool for parsing modern web media content. By packaging this tool into an Apify Actor, you gain immediate access to automated scalability, scheduling, and API distribution. This project is open-source and free for developers. Please ensure you respect Bilibili's Terms of Service and content copyright guidelines when using extracted media files. *Happy Coding!* ๐Ÿš€๐Ÿ“น๐Ÿค– # Actor input Schema ## `videoUrl` (type: `string`): Enter the Bilibili video URL (e.g. https://www.bilibili.com/video/BV1ojn8z1EUq) ## Actor input object example ```json { "videoUrl": "https://www.bilibili.com/video/BV1ojn8z1EUq" } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "videoUrl": "https://www.bilibili.com/video/BV1ojn8z1EUq" }; // Run the Actor and wait for it to finish const run = await client.actor("scrapers-hub/bilibili-video-transcript-api").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "videoUrl": "https://www.bilibili.com/video/BV1ojn8z1EUq" } # Run the Actor and wait for it to finish run = client.actor("scrapers-hub/bilibili-video-transcript-api").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "videoUrl": "https://www.bilibili.com/video/BV1ojn8z1EUq" }' | apify call scrapers-hub/bilibili-video-transcript-api --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=scrapers-hub/bilibili-video-transcript-api", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Bilibili Video Transcript & Metadata Extractor", "description": "Bilibili Video Transcript API extracts transcripts and subtitles from public Bilibili videos, making video content searchable and easy to analyze ๐Ÿ“๐ŸŽฅ Perfect for content analysis, research, accessibility, AI workflows, and multilingual processing.", "version": "0.1", "x-build-id": "RKlnhFMIW0TBbAxSe" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/scrapers-hub~bilibili-video-transcript-api/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-scrapers-hub-bilibili-video-transcript-api", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/scrapers-hub~bilibili-video-transcript-api/runs": { "post": { "operationId": "runs-sync-scrapers-hub-bilibili-video-transcript-api", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/scrapers-hub~bilibili-video-transcript-api/run-sync": { "post": { "operationId": "run-sync-scrapers-hub-bilibili-video-transcript-api", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "videoUrl" ], "properties": { "videoUrl": { "title": "Bilibili Video URL", "type": "string", "description": "Enter the Bilibili video URL (e.g. https://www.bilibili.com/video/BV1ojn8z1EUq)" } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```