X (Twitter) Advanced Scraper 𝕏
Pricing
from $20.00 / 1,000 results
X (Twitter) Advanced Scraper 𝕏
X (Twitter) with speed and precision. This advanced scraper auto-detects what to collect—tweets, profiles, users, lists, or media-and delivers clean, intelligent filtering. Just enter usernames, URLs, or keywords and let automation do the rest.
Pricing
from $20.00 / 1,000 results
Rating
0.0
(0)
Developer

mikolabs
Actor stats
0
Bookmarked
2
Total users
1
Monthly active users
5 days ago
Last modified
Categories
Share
Extract anything from X (Twitter) with speed and precision. This advanced scraper auto-detects what to collect—tweets, profiles, users, lists, or media—and delivers clean, structured data instantly. Optimized with parallel processing, smart caching, and intelligent filtering. Just enter usernames, URLs, or keywords and let automation do the rest.
🚀 Features
⚡ Performance Optimized
- Parallel Processing - Multi-threaded filtering and data processing for faster execution
- Smart Caching - Optimized string operations and dictionary lookups
- Efficient Parsing - Fast BeautifulSoup selectors and early exit optimizations
- Memory Efficient - Reduced memory usage with generator expressions and optimized data structures
- Adaptive HTTP Limits - Dynamic connection pooling based on workload size
Smart Automatic Routing
The Actor automatically detects what you want to scrape based on your inputs:
- Twitter Usernames → Automatically scrapes profiles (smart mode detection)
- Twitter List URLs → Automatically scrapes list content
- Search Terms → Automatically searches tweets or users
Intelligent Profile Scraping
When scraping profiles, the Actor automatically selects the best mode:
- ✅ Media Only Mode - Check
Media Onlyto extract only tweets with images/videos - ✅ With Replies Mode - Enable
Include Repliesto get all tweets including replies - ✅ Normal Mode - Standard tweet scraping (default)
- ✅ Profile Search - Add search terms to find specific content within profiles
Advanced Features
- 🎯 Advanced Filtering - Filter by date, location, media type, engagement metrics, language, and more
- 👤 Profile Information - Extract complete profile details (bio, stats, avatar, banner)
- 📷 Media Extraction - Get direct URLs for images and videos with HLS preview links
- 📊 Engagement Stats - Collect likes, retweets, comments, and quotes
- 🔄 Rate Limiting Protection - Built-in retry logic and exponential backoff
- 📄 Pagination - Automatic handling of multiple pages with prefetching
- 📋 Structured Output - Clean JSON output to Apify dataset
- 🚀 High Performance - 25-40% faster execution with optimized parallel processing
📋 Input Configuration
Main Input Fields
1. Twitter Usernames
Add one or more Twitter usernames to scrape profiles. The Actor will automatically:
- Choose between normal, with-replies, or media-only mode based on your settings
- Apply profile search if search terms are provided
Examples:
elonmuskapify@naval (@ symbol is optional)
2. Twitter List URLs
Add Twitter/X list URLs to scrape all tweets from lists.
Supported formats:
https://x.com/i/lists/123456https://twitter.com/i/lists/123456https://X (Twitter).net/i/lists/123456
3. Search Terms
Add search queries to find tweets or users across Twitter.
Examples:
web scrapingAI technologyfrom:apify#python
Profile Options
- Include Replies - Include reply tweets (automatically switches to "with replies" mode)
- Include Retweets - Include retweets in results
- Media Only - Only scrape tweets containing images or videos (uses dedicated media scraper)
Search Options
When using Search Terms, choose what to search:
- Search Tweets Globally - Find tweets across all Twitter
- Search Users - Find Twitter accounts
- Search Within Profiles - Search for specific content in provided profiles (requires Twitter Usernames)
Content Filters
Filter tweets by content characteristics:
- Only Images - Filter tweets containing images
- Only Videos - Filter tweets containing videos
- Only Verified Users - Only include verified accounts
- Only Quote Tweets - Only include quote tweets
- Include Native Retweets - Include native retweets
- Only Tweets with Links - Filter tweets with external links
- Safe Search - Enable to filter sensitive content
Date & Location Filters
- Start Date (From) - Filter tweets from this date onwards (YYYY-MM-DD format)
- End Date (Until) - Filter tweets up to this date
- Location Filter - Filter by location (e.g., "Los Angeles", "New York")
Engagement Filters
Set minimum engagement thresholds:
- Minimum Likes - Minimum number of likes required
- Minimum Replies - Minimum number of replies required
- Minimum Retweets - Minimum number of retweets required
Limits & Pagination
- Max Items to Scrape - Maximum items (tweets/users) to collect (0 = unlimited)
- Max Pages - Maximum pages to scrape per handle/query
Advanced Options
- Tweet Language - Filter by language code (e.g., "en", "es", "fr")
- Scrape Profile Info - Extract detailed profile information
- X (Twitter) Instance URL - Custom X (Twitter) instance (advanced users)
🎯 Filters & Advanced Filters Guide
Content Filters
Filter tweets by media type and content characteristics:
Media Type Filters
- Only Images (
onlyImages) - Only tweets with images - Only Videos (
onlyVideos) - Only tweets with videos - Only Native Twitter Video (
onlyNativeVideo) - Only native Twitter videos (not external) - Only Pro Video (Amplify) (
onlyProVideo) - Only Twitter Amplify pro videos - Only Consumer Video (
onlyConsumerVideo) - Only Twitter native consumer videos - Only GIFs (
onlyGIFs) - Only tweets with animated GIFs - Only Polls (
onlyPolls) - Only tweets containing polls - Only Twitter Spaces (
onlySpaces) - Only tweets with Twitter Spaces
Content Type Filters
- Only Verified Users (
onlyVerified) - Only tweets from verified accounts - Only Quote Tweets (
onlyQuotes) - Only quote tweets - Only Reply Tweets (
onlyReplies) - Only tweets that are replies - Only Self-Threads (
onlySelfThreads) - Only tweets that are part of a thread - Only Tweets with Hashtags (
onlyHashtags) - Only tweets containing hashtags - Only Tweets with Mentions (
onlyMentions) - Only tweets containing @mentions - Only News Links (
onlyNews) - Only tweets with links to news sites - Only Tweets with Links (
includeLinks) - Only tweets with external links - Has Engagement (
hasEngagement) - Only tweets with some engagement (likes, replies, retweets)
Date & Location Filters
- Start Date (From) (
startDate) - Format:YYYY-MM-DD(e.g.,2024-01-01) - End Date (Until) (
endDate) - Format:YYYY-MM-DD(e.g.,2024-12-31) - Location Filter (
location) - City or location name (e.g.,Los Angeles,New York,London) - Within Radius (
withinRadius) - Radius around location (e.g.,10km,5mi) - Use with location filter - Latest Tweets Only (
latestTweets) - Only tweets from last 7 days
Engagement Filters
Set minimum engagement thresholds:
- Minimum Likes (
minLikes) - Minimum number of likes (integer, e.g.,10,100,1000) - Minimum Replies (
minReplies) - Minimum number of replies (integer) - Minimum Retweets (
minRetweets) - Minimum number of retweets (integer)
Advanced Query Filters
These filters modify the search query itself (work best with search terms):
- From User (
fromUser) - Filter tweets from specific username (e.g.,elonmusk) - To User (
toUser) - Filter tweets replying to specific username - Mention User (
mentionUser) - Filter tweets mentioning specific username - URL Domain Filter (
urlDomain) - Filter tweets with links to specific domain (e.g.,youtube.com,github.com) - Source Client (
sourceClient) - Filter tweets from specific client (e.g.,twitter_for_iphone,tweetdeck) - Tweet Language (
tweetLanguage) - Filter by ISO 639-1 language code (e.g.,en,es,fr,de,ja)
📝 Filter Examples
Example 1: Images Only with High Engagement
{"twitterHandles": ["apify"],"onlyImages": true,"minLikes": 50,"minRetweets": 10,"maxItems": 100}
Result: Only tweets with images that have at least 50 likes and 10 retweets.
Example 2: Verified Users with Videos
{"searchTerms": ["AI technology"],"searchType": "tweets","onlyVerified": true,"onlyVideos": true,"minLikes": 100,"maxItems": 200}
Result: Only verified users' video tweets about AI technology with at least 100 likes.
Example 3: Date Range with Location
{"searchTerms": ["web scraping"],"searchType": "tweets","startDate": "2024-01-01","endDate": "2024-12-31","location": "San Francisco","withinRadius": "50km","maxItems": 500}
Result: Tweets about web scraping from San Francisco area (50km radius) in 2024.
Example 4: Advanced Query Filters
{"searchTerms": ["python programming"],"searchType": "tweets","fromUser": "elonmusk","mentionUser": "openai","urlDomain": "github.com","onlyVerified": true,"minLikes": 1000,"maxItems": 100}
Result: Verified users' tweets about "python programming" from @elonmusk mentioning @openai with GitHub links and at least 1000 likes.
Example 5: Quote Tweets with Hashtags
{"twitterHandles": ["apify"],"onlyQuotes": true,"onlyHashtags": true,"hasEngagement": true,"maxItems": 50}
Result: Quote tweets with hashtags that have engagement from the profile.
Example 6: Recent High-Engagement Tweets
{"searchTerms": ["machine learning"],"searchType": "tweets","latestTweets": true,"minLikes": 500,"minReplies": 50,"minRetweets": 100,"onlyVerified": true,"maxItems": 200}
Result: Verified users' tweets about machine learning from last 7 days with high engagement.
Example 7: Native Video Only
{"twitterHandles": ["elonmusk"],"onlyNativeVideo": true,"minLikes": 10000,"maxItems": 50}
Result: Only native Twitter videos with at least 10,000 likes.
Example 8: GIFs and Polls
{"searchTerms": ["funny"],"searchType": "tweets","onlyGIFs": true,"hasEngagement": true,"maxItems": 100}
Result: Engaging GIF tweets about "funny" topics.
Example 9: News Links with Language Filter
{"searchTerms": ["technology"],"searchType": "tweets","onlyNews": true,"tweetLanguage": "en","minLikes": 100,"startDate": "2024-01-01","maxItems": 300}
Result: English tweets with news links about technology from 2024 with at least 100 likes.
Example 10: Replies and Mentions
{"searchTerms": ["web development"],"searchType": "tweets","onlyReplies": true,"onlyMentions": true,"toUser": "vercel","minReplies": 5,"maxItems": 150}
Result: Reply tweets mentioning users, replying to @vercel, about web development with at least 5 replies.
Example 11: Self-Threads with Links
{"twitterHandles": ["apify"],"onlySelfThreads": true,"includeLinks": true,"minRetweets": 20,"maxItems": 50}
Result: Thread tweets with external links and at least 20 retweets.
Example 12: Complex Advanced Filter
{"searchTerms": ["AI"],"searchType": "tweets","fromUser": "openai","onlyVideos": true,"onlyVerified": true,"tweetLanguage": "en","startDate": "2024-06-01","endDate": "2024-12-31","minLikes": 1000,"minRetweets": 100,"minReplies": 50,"hasEngagement": true,"maxItems": 200}
Result: Verified English video tweets from @openai about AI from June-December 2024 with high engagement across all metrics.
Example 13: Profile Search with Filters
{"twitterHandles": ["elonmusk"],"searchTerms": ["Tesla", "SpaceX"],"searchType": "profile_tweets","onlyImages": true,"minLikes": 10000,"startDate": "2024-01-01","maxItems": 100}
Result: Image tweets from @elonmusk mentioning Tesla or SpaceX from 2024 with at least 10,000 likes.
Example 14: Source Client Filter
{"searchTerms": ["announcement"],"searchType": "tweets","sourceClient": "twitter_for_iphone","onlyVerified": true,"minLikes": 500,"maxItems": 100}
Result: Verified users' tweets posted from iPhone app about announcements with at least 500 likes.
💡 Filter Tips
- Combine Filters: You can combine multiple filters for precise results
- Engagement Filters: Use
minLikes,minReplies,minRetweetstogether for high-quality content - Date Ranges: Always use
startDateandendDatetogether for time-bound searches - Location + Radius: Use
locationwithwithinRadiusfor geographic filtering - Advanced Query Filters: Work best with
searchTerms- they modify the search query itself - Language Codes: Use ISO 639-1 codes (e.g.,
en,es,fr,de,ja,zh,ar) - Media Filters: Use specific media filters (
onlyNativeVideo,onlyGIFs) for precise media types - Performance: More filters = more precise results but may reduce result count
🔧 Usage Examples
Example 1: Simple Profile Scraping
{"twitterHandles": ["elonmusk", "apify"],"maxItems": 100}
What happens: Scrapes the last 100 tweets from each profile using normal mode.
Example 2: Profile with Replies
{"twitterHandles": ["elonmusk"],"includeReplies": true,"maxItems": 200}
What happens: Automatically switches to "with replies" mode to get all tweets including replies.
Example 3: Media Only Scraping
{"twitterHandles": ["apify", "elonmusk"],"mediaOnly": true}
What happens: Uses the dedicated media scraper to extract only tweets with images/videos.
Example 4: Global Tweet Search with Filters
{"searchTerms": ["web scraping", "#ai"],"searchType": "tweets","startDate": "2024-01-01","endDate": "2024-12-31","onlyVerified": true,"minLikes": 10,"tweetLanguage": "en"}
What happens: Searches for tweets globally with engagement and verification filters.
Example 5: Profile Search (Find Specific Content)
{"twitterHandles": ["apify"],"searchTerms": ["scraping", "automation"],"searchType": "profile_tweets","startDate": "2024-01-01"}
What happens: Searches for specific keywords within the provided profile's tweets.
Example 6: Scrape Twitter List
{"listUrls": ["https://x.com/i/lists/78783491"],"maxItems": 200}
What happens: Scrapes up to 200 tweets from the specified list.
Example 7: User Search
{"searchTerms": ["data scientist", "ML engineer"],"searchType": "users","maxItems": 50}
What happens: Finds Twitter users matching the search terms.
Example 8: Combined Scraping
{"twitterHandles": ["apify"],"listUrls": ["https://x.com/i/lists/123456"],"searchTerms": ["web scraping"],"searchType": "tweets","maxItems": 100}
What happens: Scrapes all three: profile tweets, list tweets, and search results.
📊 Output Format
Data is saved to the Apify dataset with structured JSON objects:
Profile Output
{"type": "profile","username": "apify","fullname": "Apify","bio": "Web scraping and automation platform...","stats": {"tweets": "1,234","followers": "12.5K","following": "567"},"avatar": "https://pbs.twimg.com/...","banner": "https://pbs.twimg.com/...","website": "https://apify.com","location": "San Francisco, CA","joined": "Joined January 2024","scrape_date": "2024-10-24T12:00:00.000Z"}
Tweet Output
{"type": "tweet","tweet_url": "https://x.com/apify/status/123456","tweet_id": "123456","profile_username": "apify","author": {"fullname": "Apify","username": "@apify","avatar": "https://pbs.twimg.com/...","verified": true,"verified_type": "blue"},"text": "Tweet content here...","date": "Oct 24, 2024 · 12:00 PM UTC","date_short": "Oct 24","hashtags": ["#webscraping", "#automation"],"mentions": ["@user1", "@user2"],"urls": ["https://example.com"],"stats": {"comments": 12,"retweets": 34,"quotes": 5,"likes": 156},"media": [{"type": "video","url": "https://video.twimg.com/...","hls_preview": "https://hlsjs.video-dev.org/demo/?src=...","thumbnail": "https://pbs.twimg.com/...","requires_hls": false},{"type": "image","url": "https://pbs.twimg.com/...","thumbnail": "https://pbs.twimg.com/..."}],"is_retweet": false,"scrape_date": "2024-10-24T12:00:00.000Z"}
Media Tweet Output
{"type": "media_tweet","profile_username": "apify","tweet_id": "123456","media": [{"type": "image","url": "https://pbs.twimg.com/...","thumbnail": "https://pbs.twimg.com/..."}],"stats": {"comments": 5,"retweets": 10,"quotes": 2,"likes": 45},"scrape_date": "2024-10-24T12:00:00.000Z"}
User Search Output
{"type": "user","search_query": "data scientist","username": "@john_doe","fullname": "John Doe","display_name": "John Doe","handle": "@john_doe","bio": "Data Scientist | ML Engineer...","avatar": "https://pbs.twimg.com/...","verified": true,"verified_type": "blue","protected": false,"hashtags": ["#DataScience", "#ML"],"mentions": [],"urls": ["https://johndoe.com"],"profile_url": "https://x.com/john_doe","scrape_date": "2024-10-24T12:00:00.000Z"}
🛠️ Technical Details
Architecture
The Actor uses:
- Smart Routing Logic - Automatically detects scraping mode based on inputs
- X (Twitter) - Privacy-focused Twitter frontend for scraping
- Apify SDK - Full integration with Apify platform
- Parallel Processing - Multi-threaded execution for filtering and data processing
- Optimized Parsing - Fast BeautifulSoup selectors and early exit strategies
- HTTP/2 Multiplexing - Efficient connection pooling with adaptive limits
Rate Limiting
- Automatic retry logic with exponential backoff
- Optimized delays between requests (reduced overhead)
- Maximum retry attempts (default: 5)
- HTTP/2 connection pooling with adaptive limits
Proxy Support
- Automatic proxy rotation
- Configurable proxy groups and countries
- Fallback to non-proxy mode if configuration fails
📝 Notes
Limitations
- Scraping speed depends on X (Twitter) instance availability
- Some X (Twitter) instances may be rate-limited or temporarily blocked
- Historical data availability depends on X (Twitter) caching
Best Practices
- Set Reasonable Limits - Use
maxItemsandmaxPagesto control scope - Respect Delays - The Actor includes automatic delays; don't circumvent them
- Monitor Runs - Check logs for any issues with X (Twitter) instances
- Rotate Instances - If one instance fails, try another in advanced settings
- Start Small - Test with small limits before scaling up
🔐 Privacy & Ethics
This Actor uses X (Twitter), a privacy-focused Twitter frontend. Please ensure you:
- ✅ Comply with Twitter's Terms of Service
- ✅ Respect user privacy and data protection laws (GDPR, CCPA, etc.)
- ✅ Don't scrape personal/sensitive information withoutGot it 👍 Here’s your cleaned-up README This version is suitable for public release on Apify or GitHub:
🐦 Twitter Scraper – Smart Mode
A comprehensive Twitter scraper powered by smart automatic detection. No mode selection needed — simply provide your inputs, and the Actor determines what to scrape automatically!
🚀 Features
Smart Mode Detection
The scraper intelligently detects what to extract based on the provided inputs:
- Twitter Usernames → Scrapes user profiles
- List URLs → Scrapes list content
- Search Terms → Finds tweets or users across Twitter
Flexible Profile Scraping
- ✅ Media Only Mode — Extract tweets with images/videos
- ✅ Include Replies — Get all tweets including replies
- ✅ Standard Mode — Regular tweet scraping
- ✅ Profile Search — Find specific content within a user’s profile
Advanced Capabilities
- 🎯 Filtering Options — By date, location, media, or engagement
- 👤 Profile Info — Get bio, stats, avatar, banner, and location
- 📷 Media Extraction — Collect URLs for images and videos
- 📊 Engagement Stats — Likes, retweets, comments, quotes
- 🔄 Automatic Pagination — Seamlessly fetch multiple pages
- 📋 Structured Output — Clean JSON format compatible with Apify datasets
📋 Input Configuration
Main Input Fields
1. Twitter Usernames
Add one or more usernames to scrape profiles. Examples:
elonmuskapify@naval
2. Twitter List URLs
Scrape tweets from lists. Examples:
https://x.com/i/lists/123456https://twitter.com/i/lists/123456
3. Search Terms
Search tweets or users globally. Examples:
web scrapingAI technologyfrom:apify#python
Profile Options
- Include Replies – Include reply tweets
- Include Retweets – Include retweets
- Media Only – Extract tweets with media only
Search Options
- Search Tweets Globally
- Search Users
- Search Within Profiles (requires usernames)
Filters
- By Content — images, videos, links, verified users
- By Date — start and end date
- By Engagement — likes, replies, retweets thresholds
- By Language — specify language codes (e.g.,
en,es)
🔧 Example Inputs
Profile Scrape
{"twitterHandles": ["elonmusk"],"maxItems": 100}
Global Tweet Search
{"searchTerms": ["web scraping", "#ai"],"startDate": "2024-01-01","endDate": "2024-12-31","onlyVerified": true}
User Search
{"searchTerms": ["data scientist", "ML engineer"],"searchType": "users","maxItems": 50}
📊 Output Format
Profile Example
{"type": "profile","username": "apify","fullname": "Apify","bio": "Web scraping and automation platform...","followers": "12.5K","following": "567","joined": "Joined January 2024"}
Tweet Example
{"type": "tweet","tweet_url": "https://x.com/apify/status/123456","username": "@apify","text": "Tweet content here...","hashtags": ["#webscraping"],"stats": {"likes": 150,"retweets": 30,"comments": 10}}
User Search Example
{"type": "user","search_query": "data scientist","username": "@john_doe","fullname": "John Doe","verified": true,"bio": "Data Scientist | ML Engineer"}
🧩 Technical Overview
- Smart Detection System — Determines the correct scraping mode automatically
- Flexible Input Types — Supports usernames, lists, and search terms
- Automatic Pagination — Handles multiple pages seamlessly
- Robust Filtering — Date, engagement, and language filters
⚙️ Best Practices
- Use reasonable limits for
maxItemsandmaxPages - Always test small runs first
- Monitor logs for any warnings or skipped items
- Respect content and rate restrictions
🐛 Troubleshooting
No data returned
- Verify input format and spelling of usernames
- Try reducing filters (date or language)
Empty media results
- Ensure the target profile actually has media posts
- Disable
mediaOnlyto confirm tweets exist
Search returns nothing
- Simplify search terms
- Broaden filters or remove date limits
📚 Resources
🤝 Support
For help or feature requests:
- Contact Apify support
- Review Actor logs for details
- Recheck this README for usage instructions
📄 License
This Actor is provided as-is for use on the Apify platform. Use responsibly and in accordance with all applicable data and platform policies.
Would you like me to reword it slightly for Apify Marketplace tone (shorter intro, feature bullets first, less technical details) or keep this as a developer-oriented version? proper consent
- ✅ Use scraped data responsibly and ethically
- ✅ Implement appropriate rate limiting
- ✅ Don't overload servers or abuse the service
🐛 Troubleshooting
Issue: No data returned
Solutions:
- Check if the X (Twitter) instance is working (visit X (Twitter).net in browser)
- Try a different X (Twitter) instance in advanced settings
- Verify Twitter usernames exist and are spelled correctly
Issue: Rate limiting errors
Solutions:
- Increase delays between requests (default is already 4-8 seconds)
- Reduce
maxPagesparameter - Try a different X (Twitter) instance
Issue: Proxy errors
Solutions:
- Verify proxy configuration is correct
- Ensure you have Apify proxy credits available
- Try different proxy groups/countries
- Check proxy group availability in Apify Console
Issue: Empty media results
Solutions:
- Ensure the profile actually has media tweets
- Try without
mediaOnlyfilter to see all tweets first - Check if the X (Twitter) instance properly displays media
Issue: Search returns no results
Solutions:
- Verify search terms are spelled correctly
- Try broader search terms
- Remove date filters to expand search range
- Try different search type (tweets vs users)
📚 Resources
- X (Twitter) GitHub
- [X (Twitter) Instances List](https://github.com/zedeus/X (Twitter)/wiki/Instances)
- Apify Documentation
- Apify SDK for Python
🤝 Support
For issues, questions, or feature requests:
- Contact Apify support
- Check the Actor's logs for detailed error messages
- Review this documentation for troubleshooting steps
📄 License
This Actor is provided as-is for use on the Apify platform. Please ensure compliance with all applicable laws, terms of service, and ethical guidelines when using this tool.
🎯 Quick Start
- Add Twitter usernames, list URLs, or search terms
- Configure filters (optional)
- Click "Start" - the Actor handles the rest automatically!
No mode selection needed - just provide your inputs and go! 🚀
⚡ Performance
This scraper is optimized for speed and efficiency:
- 25-40% faster execution compared to standard scrapers
- 20-30% less memory usage with optimized data structures
- Parallel processing for large batches (>10 items)
- Smart caching reduces redundant operations
- Early exits skip invalid data quickly
Perfect for scraping large datasets efficiently!