Twitter video downloader

Download multiple videos from Twitter / X.com and save them to Apify key value store

Supported URLs

• https://twitter.com/username/status/...
• https://x.com/username/status/...
• https://twitter.com/i/status/...
• https://x.com/i/status/...

Custom data

If you need to pass custom data to the output, set the userData object of the request in the startUrls array and it will be appended to the output

1{
2    "startUrls": [{
3        "url": "https://www.twitter.com/.../status/...",
4        "userData": {
5            "userid": "1454784661528485889"
6        }
7    }]
8}

Output

The generated output is the following:

1{
2	"url": "https://x.com/apify/status/1454784661528485889", // original URL requested
3	"hash": "83143134", // the internal hash of the video, never changes between requests
4	"partsUrl": "https://api.apify.com/v2/key-value-stores/SOME_ID/records/83143134", // the URL to the Key Value store where the video parts are stored
5    "userData": {
6        "userid": "1454784661528485889"
7    },
8	"#error": false // hidden field if there was an error
9}

The video is chunked in parts of ~2MB each and saved to the Key Value store. To be able to download it, you'll need to download all the parts and concatenate them together. Here's a small example that can be used in the browser:

1const { appendFileSync } = require('node:fs');
2
3async function main(datasetId) {
4    const items = await fetch(`https://api.apify.com/v2/datasets/${datasetId}/items?clean=true&format=json`).then((response) => response.json());
5
6    const firstVideo = items[0].partsUrl; // partsUrl contains the location to the Key Value store
7
8    // get the parts from the Key value store
9    const { parts, length, contentType } = await fetch(firstVideo).then((response) => response.json());
10
11    console.log({ parts, length, contentType });
12
13    // wait for all parts to be downloaded
14    for (const url of parts) {
15        // download part using fetch
16        const downloaded = await fetch(url);
17        // get an arrayBuffer from the chunk
18        const arrayBuffer = new Uint8Array(await downloaded.arrayBuffer());
19
20        console.log(`Downloaded ${arrayBuffer.byteLength} bytes from ${url}`);
21
22        // write downloaded chunk to file
23        appendFileSync('video.mp4', arrayBuffer);
24    }
25}
26
27main('YOUR_DATASET_ID');