Tweets-x-Scraper

Extract anything from X (Twitter) with speed and precision. This smart scraper auto-detects what to collect—tweets, profiles, users, lists, or media—and delivers clean, structured data instantly. Just enter usernames, URLs, or keywords and let automation do the rest.

Extract anything from X (Twitter) with speed and precision. This smart scraper auto-detects what to collect—tweets, profiles, users, lists, or media—and delivers clean, structured data instantly. Just enter usernames, URLs, or keywords and let automation do the rest.

🚀 Features

Smart Automatic Routing

The Actor automatically detects what you want to scrape based on your inputs:

• Twitter Usernames → Automatically scrapes profiles (smart mode detection)
• Twitter List URLs → Automatically scrapes list content
• Search Terms → Automatically searches tweets or users

Intelligent Profile Scraping

When scraping profiles, the Actor automatically selects the best mode:

• ✅ Media Only Mode - Check Media Only to extract only tweets with images/videos
• ✅ With Replies Mode - Enable Include Replies to get all tweets including replies
• ✅ Normal Mode - Standard tweet scraping (default)
• ✅ Profile Search - Add search terms to find specific content within profiles

Advanced Features

• 🎯 Advanced Filtering - Filter by date, location, media type, engagement metrics
• 👤 Profile Information - Extract complete profile details (bio, stats, avatar, banner)
• 📷 Media Extraction - Get direct URLs for images and videos with HLS preview links
• 📊 Engagement Stats - Collect likes, retweets, comments, and quotes
• 🔄 Rate Limiting Protection - Built-in retry logic and exponential backoff
• 📄 Pagination - Automatic handling of multiple pages
• 📋 Structured Output - Clean JSON output to Apify dataset

📋 Input Configuration

Main Input Fields

1. Twitter Usernames

Add one or more Twitter usernames to scrape profiles. The Actor will automatically:

• Choose between normal, with-replies, or media-only mode based on your settings No mode selection needed - just provide your inputs and go! 🚀by minimum likes (minLikes), minimum retweets (minRetweets), minimum replies (minReplies), verified authors only (onlyVerified), quotes only (onlyQuotes), or tweets with external links (includeLinks).
• Rich Media & Content: Extracts full-resolution images, original MP4 videos, animated GIFs, video poster thumbnails, and Twitter card previews.
• Anti-Blocking Built-In: Integrated support for Apify Residential proxies and intelligent session recycling.

---

📋 Input Parameters

Parameter	Type	Default	Description
twitterHandles	Array	["rish_w3b"]	Twitter handles to scrape (with or without @).
listUrls	Array	[]	Twitter list URLs (e.g. https://x.com/i/lists/123456).
searchTerms	Array	[]	Search terms, keywords, hashtags, or phrases.
searchType	String	"tweets"	tweets (Global Tweet Search), users (User Search), or profile_tweets (Search Within Profiles).
includeReplies	Boolean	false	Include replies when scraping profiles.
includeRetweets	Boolean	true	Include retweets in results.
mediaOnly	Boolean	false	Only scrape tweets containing images or videos.
maxItems	Integer	100	Maximum items to scrape across all targets (0 = unlimited for paid users; max 10 for free users).
startDate	String	""	Start date filter (YYYY-MM-DD).
endDate	String	""	End date filter (YYYY-MM-DD).
location	String	""	Location filter (e.g. 'New York', 'London').
onlyImages	Boolean	false	Filter tweets that contain images.
onlyVideos	Boolean	false	Filter tweets that contain videos.
onlyVerified	Boolean	false	Only tweets from verified accounts.
onlyQuotes	Boolean	false	Only quote tweets.
includeNativeRetweets	Boolean	true	Include native retweets in search.
includeLinks	Boolean	false	Only tweets containing external URLs.
latestTweets	Boolean	false	Only scrape tweets from the last 7 days.
safeSearch	Boolean	false	Enable safe search to filter sensitive content.
minLikes	Integer	0	Minimum likes threshold.
minReplies	Integer	0	Minimum replies threshold.
minRetweets	Integer	0	Minimum retweets threshold.
tweetLanguage	String	""	Language code (e.g. 'en', 'es', 'fr').
scrapeProfileInfo	Boolean	true	Extract full profile metadata header when scraping handles.
useResidentialProxy	Boolean	false	Use Apify Residential Proxy for maximum reliability.
maxRetries	Integer	5	Retry attempts per failed page request.

---

💡 Example Configurations

1. Scrape Profile Timelines with Media

{
  "twitterHandles": ["elonmusk", "OpenAI"],
  "scrapeProfileInfo": true,
  "mediaOnly": false,
  "maxItems": 100
}

2. Scrape a Twitter List

{
  "listUrls": [
    "https://x.com/i/lists/123456789"
  ],
  "maxItems": 200
}

3. High-Engagement Topic Search

{
  "searchTerms": ["artificial intelligence", "#LLM"],
  "searchType": "tweets",
  "minLikes": 50,
  "tweetLanguage": "en",
  "startDate": "2024-01-01",
  "maxItems": 500
}

4. Search Users by Topic

{
  "searchTerms": ["crypto founder", "AI researcher"],
  "searchType": "users",
  "maxItems": 50
}

---

📊 Output Dataset Format

Tweet Item

{
  "id": "1892837465928172635",
  "type": "tweet",
  "url": "https://x.com/elonmusk/status/1892837465928172635",
  "text": "Exciting advances in AI systems coming this year.",
  "timestamp": "2026-08-21T16:20:00Z",
  "username": "elonmusk",
  "fullname": "Elon Musk",
  "likes": 42500,
  "retweets": 8300,
  "replies": 3200,
  "quotes": 940,
  "has_media": true,
  "images": ["https://pbs.twimg.com/media/xyz123.jpg"],
  "videos": [],
  "urls": ["https://x.ai"],
  "hashtags": ["AI"],
  "mentions": [],
  "author": {
    "username": "elonmusk",
    "fullname": "Elon Musk",
    "verified": true,
    "verified_type": "blue",
    "profile_picture": "https://pbs.twimg.com/profile_images/.../photo.jpg"
  }
}

Profile Metadata Item

{
  "id": "elonmusk",
  "type": "profile",
  "username": "elonmusk",
  "fullname": "Elon Musk",
  "bio": "...",
  "location": "Texas",
  "website": "https://x.com",
  "joined": "June 2009",
  "followers": 180000000,
  "following": 500,
  "tweets_count": 45000,
  "banner": "https://pbs.twimg.com/profile_banners/.../banner.jpg",
  "profile_picture": "https://pbs.twimg.com/profile_images/.../photo.jpg",
  "verified": true,
  "verified_type": "blue"
}

---

⚖️ License

This project is licensed under the AGPL-3.0 License.