Twitter/X Image & Video Scraper Actor

Overview

The Twitter/X Image & Video Scraper is a powerful tool designed to extract images and videos from Twitter/X based on search queries quickly and efficiently. This actor simplifies retrieving media content from Twitter/X, making it ideal for researchers, content creators, marketers, and developers who need social media content for analysis, curating collections, or research purposes.

This actor supports authenticated search queries, ensuring reliable access even with X.com's authentication requirements.

Features

• Fast and Efficient: Retrieves Twitter/X media (images and videos) in seconds using optimized Selenium automation
• Accurate Extraction: Captures media items with full metadata including usernames, descriptions, and source URLs
• Intelligent Cookie Management: Automatically validates authentication cookies and warns when they expire
• Flexible Media Filtering: Search for images, videos, or both with separate URL filters
• Metadata Preservation: Stores username, tweet URL, and description for each downloaded item
• Video Support: Extracts video tweet URLs with yt-dlp integration
• SEO-Friendly: Export media metadata in JSON format for integration with external systems
• Cloud-Ready: Works on local machines and Apify cloud deployments with proper authentication

Use Cases

• Content Curation: Build image or video collections around specific topics
• Market Research: Analyze trending visual content in your industry
• Social Media Analysis: Study engagement patterns on specific topics
• Data Collection: Gather labeled image datasets for machine learning projects
• Content Repurposing: Find source material for blogs, reports, or presentations
• Trend Monitoring: Track visual trends and viral content in real-time

Input Parameters

The actor accepts the following input parameters to customize your search.

Parameter Name	Type	Description	Default Value
searchQuery	String	The search term to find on Twitter/X. Example: dogs, nature photography, AI innovations	nature
limit	Number	Maximum number of media items to download. Range: 1-100	20
mediaType	String	Type of media to search for: images, videos, or all	all
downloadImages	Boolean	Whether to download and store image files	true
downloadVideos	Boolean	Whether to download video tweets (metadata only, not video files)	false
includeUsername	Boolean	Include the username in downloaded filenames	true
includeDescription	Boolean	Include tweet text in metadata	true

Example Inputs

Search for Dog Images

{
  "searchQuery": "dogs",
  "limit": 10,
  "mediaType": "images",
  "downloadImages": true,
  "downloadVideos": false
}

Search for Alaska Videos

{
  "searchQuery": "alaska",
  "limit": 15,
  "mediaType": "videos",
  "downloadImages": false,
  "downloadVideos": true
}

Mixed Media Search

{
  "searchQuery": "nature photography",
  "limit": 25,
  "mediaType": "all",
  "downloadImages": true,
  "downloadVideos": true
}

Output

The actor outputs media metadata in JSON format, stored in the Apify dataset.

Example Output Structure

Image Output

{
  "filename": "photographer_name_sunset_beach_001.jpg",
  "url": "https://pbs.twimg.com/media/abc123.jpg?format=jpg&name=large",
  "type": "image",
  "username": "photographer_name",
  "description": "Beautiful sunset at the beach",
  "search_query": "sunset beach",
  "status": "downloaded",
  "size_bytes": 245620,
  "content_type": "image/jpeg",
  "public_url": "https://api.apify.com/v2/key-value-stores/abc123/records/photographer_name_sunset_beach_001.jpg"
}

Video Output

{
  "filename": "JoelOsteen_video_19.mp4",
  "filepath": "/usr/src/app/videos/JoelOsteen_video_19.mp4",
  "url": "https://x.com/JoelOsteen/status/1984289641667694621/video/1",
  "type": "video",
  "username": "JoelOsteen",
  "description": "video_19",
  "file_size_bytes": 9277111,
  "file_extension": ".mp4",
  "search_query": "Apple",
  "status": "downloaded",
  "download_timestamp": 1762867628.8281474,
  "public_url": "https://api.apify.com/v2/key-value-stores/abc123/records/JoelOsteen_video_19.mp4"
}

Key Fields

• filename: The saved filename of the media item
• url: Direct URL to the media on Twitter/X (images) or tweet URL (videos)
• type: Either image or video
• username: Twitter/X username of the poster
• description: Text from the tweet or title
• search_query: The search term that found this item
• status: Download status (downloaded, pending, failed)
• size_bytes (images) / file_size_bytes (videos): File size in bytes
• content_type: MIME type (images only)
• file_extension: Video format e.g. .mp4 (videos only)
• public_url: Publicly accessible HTTPS URL for the file (both images and videos). Use this to download or embed media in external applications
• download_timestamp: Unix timestamp of when the video was downloaded (videos only)

Pricing

• Pay-As-You-Go: Only pay for the compute units you use
• Subscription Plans: Discounted rates for high-volume usage

Check the Apify Pricing Page for current rates.

FAQs

Q1: Does this actor work for all X.com/Twitter videos?

A: It works for public tweets with videos. The actor downloads actual video files (not just metadata) and stores them with public URLs accessible via the public_url field in the output.

Q2: Can I search for private account tweets?

A: No, only public tweets are accessible. Private or restricted accounts are not supported due to X.com's privacy policies.

Q3: Can I search multiple queries in one run?

A: This actor processes one search query per run. For multiple queries, schedule separate runs or use Apify's batch processing.

Q4: What's the difference between images, videos, and all?

A:

• images: Tweets with image attachments only
• videos: Tweets with video/GIF content only
• all: Both image and video tweets

Documentation

• Apify SDK Documentation