X (Twitter) Profile Scraper

Extract X (Twitter) profile data with advanced anti-detection measures. Provide your session cookies, enter usernames or URLs, and get structured data instantly.

Important: X/Twitter requires login to view profiles. You must provide your session cookies for this scraper to work.

Features

• Simple Input - Accepts usernames, @handles, full URLs, or mixed formats
• Profile Data - Display name, bio, location, website, join date, followers, following
• Engagement Metrics - Followers count, following count, tweets count, likes count
• Verification Status - Detects blue verified and legacy verified badges
• Media - Profile picture (high-res) and banner image URLs
• Anti-Detection - Browser fingerprinting, stealth headers, popup dismissal
• Graceful Degradation - Returns partial data when full access is blocked

Prerequisites: Getting Your Cookies

X/Twitter now requires authentication to view any profile data. You must provide your session cookies.

Step-by-Step Guide

1. Install Cookie-Editor extension

   • Chrome
   • Firefox

2. Log into X/Twitter

   • Go to x.com and log in with your account

3. Export your cookies

   • Click the Cookie-Editor extension icon
   • Click "Export" button
   • Select "Export as JSON"
   • Copy the entire JSON array

4. Paste into the actor

   • Paste the JSON into the cookies input field

Required Cookies

The scraper validates that you have these essential cookies:

Cookie	Description
auth_token	Your session authentication token (required)
ct0	CSRF token for API requests (recommended)

Example Cookie Format

[
  {"name": "auth_token", "value": "abc123def456...", "domain": ".x.com", "path": "/", ...},
  {"name": "ct0", "value": "xyz789...", "domain": ".x.com", "path": "/", ...},
  ...
]

Note: Cookie-Editor exports all cookies with the correct format. Just paste the entire output.

Usage

Input Formats

Enter X/Twitter profiles in any format (one per line or comma-separated):

elonmusk
@OpenAI
https://x.com/BillGates
https://twitter.com/satloops
x.com/apaborkar

Input Parameters

Parameter	Type	Description
profiles	String	X/Twitter usernames or URLs (required)
cookies	String	X session cookies as JSON array (required)
proxyType	String	Proxy group: RESIDENTIAL (default), BUYPROXIES94952 (Datacenter), or StaticUS3
maxConcurrency	Integer	Parallel pages (1-3, default: 1)
maxRequestsPerMinute	Integer	Rate limit (5-20, default: 10)

Output

Successful Scrape

{
  "inputUrl": "https://x.com/elonmusk",
  "scrapedUrl": "https://x.com/elonmusk",
  "username": "elonmusk",
  "displayName": "Elon Musk",
  "bio": "The people voted for major change...",
  "location": "Earth",
  "website": "https://x.com/America",
  "joinedDate": "June 2009",
  "followers": 215000000,
  "following": 1024,
  "tweetsCount": 65000,
  "likesCount": 52000,
  "isVerified": false,
  "isBlueVerified": true,
  "profilePicture": "https://pbs.twimg.com/profile_images/.../photo_400x400.jpg",
  "bannerImage": "https://pbs.twimg.com/profile_banners/.../1500x500",
  "recentPosts": [
    {
      "id": "1883847583920193820",
      "text": "This is the tweet content...",
      "timestamp": "2026-01-26T08:30:00.000Z",
      "timeAgo": "2h",
      "url": "https://x.com/elonmusk/status/1883847583920193820",
      "replies": 15420,
      "reposts": 32100,
      "likes": 245000,
      "views": 12500000,
      "bookmarks": 8200,
      "isRepost": false,
      "hasImage": true,
      "hasVideo": false
    }
  ],
  "scrapedAt": "2026-01-26T10:30:00.000Z",
  "dataSource": "devwithbobby/x-profile-scraper",
  "isPartialData": false
}

Recent Posts Structure

Each post in the recentPosts array contains:

Field	Type	Description
id	String	Unique tweet ID
text	String	Tweet content (max 500 chars)
timestamp	String	ISO timestamp of post
timeAgo	String	Relative time (e.g., "2h", "3d")
url	String	Direct link to the tweet
replies	Integer	Number of replies
reposts	Integer	Number of reposts/retweets
likes	Integer	Number of likes
views	Integer	Number of views
bookmarks	Integer	Number of bookmarks
isRepost	Boolean	Whether this is a repost
hasImage	Boolean	Contains an image
hasVideo	Boolean	Contains a video

Partial Data (Login Wall)

When X blocks full access, the scraper returns available meta data:

{
  "inputUrl": "https://x.com/someuser",
  "scrapedUrl": "https://x.com/someuser",
  "username": "someuser",
  "displayName": "Some User",
  "bio": "Bio extracted from meta tags",
  "isPartialData": true,
  "scrapedAt": "2026-01-26T10:30:00.000Z"
}

Error Cases

{
  "inputUrl": "https://x.com/deletedaccount",
  "username": "deletedaccount",
  "error": "Account not found",
  "scrapedAt": "2026-01-26T10:30:00.000Z"
}

Output Fields

Field	Type	Description
inputUrl	String	Original URL requested
scrapedUrl	String	Final URL after redirects
username	String	X handle without @
displayName	String	Profile display name
bio	String	Profile bio/description
location	String	Location from profile
website	String	Website URL from profile
joinedDate	String	When the account was created
followers	Integer	Number of followers
following	Integer	Number of accounts following
tweetsCount	Integer	Total number of tweets
likesCount	Integer	Total number of likes
isVerified	Boolean	Legacy verified status
isBlueVerified	Boolean	X Blue/Premium verified
profilePicture	String	High-res profile image URL
bannerImage	String	Banner image URL
recentPosts	Array	Up to 10 recent posts with engagement metrics
isPartialData	Boolean	True if some data couldn't be extracted
error	String	Error message if scrape failed

Technical Details

Anti-Detection Measures

• Browser Fingerprinting - Realistic Chrome/desktop fingerprints via useFingerprints
• Stealth Headers - Proper Sec-Ch-Ua, Sec-Fetch-* headers matching real Chrome
• WebDriver Masking - Overrides navigator.webdriver and other automation indicators
• Human-like Behavior - Random delays, scrolling, variable viewport sizes
• Popup Dismissal - Automatically closes X login modals and prompts

Block Detection

The scraper detects and handles:

• Login redirects
• Suspended accounts
• Deleted/non-existent accounts
• Rate limiting
• Login wall modals

Limitations

• Authentication required - X requires login to view profiles; cookies must be provided
• Rate limiting - X may block after many requests from the same IP
• Private accounts - Cannot scrape protected/private profiles
• Cookie expiration - Session cookies expire; you may need to re-export periodically

Best Practices

1. Use Residential Proxies for highest success rates (X heavily blocks datacenter IPs)
2. Keep concurrency at 1 to avoid triggering rate limits
3. Refresh cookies weekly - Export fresh cookies if you see authentication failures
4. Space out runs to avoid IP-based blocks
5. Use low request rates (10/min or less recommended)

Proxy Options

Option	Description	Best For
RESIDENTIAL	Residential proxies (default)	Highest success rate
BUYPROXIES94952	Datacenter proxies	Cost-effective, lower success
StaticUS3	Static US IPs	Consistent identity

Cost Estimation

• Residential proxy: ~$12.50 per 1000 requests (recommended)
• Datacenter proxy: ~$0.25 per 1000 requests (may have lower success)
• Compute: ~$0.15 per 100 profiles

Summary Statistics

After each run, a summary is saved to the key-value store:

{
  "totalRequested": 10,
  "totalScraped": 8,
  "totalErrors": 2,
  "averageFollowers": 1250000,
  "verifiedCount": 3,
  "proxyUsed": "RESIDENTIAL",
  "authenticated": true,
  "completedAt": "2026-01-26T10:35:00.000Z"
}

Troubleshooting

"No data could be extracted"

• Check your cookies - Make sure you exported all cookies, not just specific ones
• Refresh cookies - Log out of X, log back in, and export fresh cookies
• Verify auth_token - Ensure the auth_token cookie is present in your export

"Missing required cookie: auth_token"

• You need to be logged into X when exporting cookies
• Make sure you're on x.com (not twitter.com) when exporting

Rate limiting / blocked

• Reduce maxRequestsPerMinute to 5
• Use RESIDENTIAL proxy type
• Wait a few hours before retrying

Support

For issues or feature requests, contact the author or open an issue on the actor's page.