Twitter Follower Scraper

A Twitter Scraper that scrapes followers/following and verified follower/following data using authenticated requests. This scraper can extract detailed user information including verification status, profile data, and social metrics.

Features

• Multiple Scraping Types: Scrape followers, following, or filter for verified users only
• Batch Processing: Process multiple usernames in a single run
• Rich User Data: Extract comprehensive profile information including:
  • Basic profile data (name, username, description)
  • Social metrics (followers, following, tweets count)
  • Verification status and details
  • Profile images (standard and high-resolution)
  • Account creation date and location
• Verification Filtering: Option to scrape only verified users
• Rate Limit Handling: Built-in delays to respect Twitter's API limits
• Error Handling: Robust error handling with detailed logging

Input Parameters

Configure the scraper using these input parameters:

Required Parameters

Parameter	Type	Description
username	Array[String]	List of Twitter usernames to scrape (without @ symbol)
cookies	Array[Object]	Authentication cookies from your Twitter session

Optional Parameters

Parameter	Type	Default	Description
scrape_type	String	"followers"	Type of data to scrape
follower_limit	Number	100	Maximum number of followers/following to scrape per username

Scrape Types

• "followers" - Scrape followers of the specified users
• "following" - Scrape users that the specified users are following
• "verified_followers" - Scrape only verified followers
• "verified_following" - Scrape only verified users that the specified users are following

Authentication Setup

This scraper requires Twitter authentication cookies. Here's how to obtain them:

1. Login to Twitter in your browser
2. Open Developer Tools (F12)
3. Go to Application/Storage tab → Cookies → https://twitter.com
4. Export the following cookies:

   [
     {"name": "auth_token", "value": "your_auth_token_value"},
     {"name": "ct0", "value": "your_ct0_value"},
     {"name": "twid", "value": "your_twid_value"}
   ]
   

Cookie Format

Each cookie object should contain:

{
  "name": "cookie_name",
  "value": "cookie_value"
}

Input Example

{
  "username": ["elonmusk", "sundarpichai", "satyanadella"],
  "scrape_type": "followers",
  "follower_limit": 500,
  "cookies": [
    {"name": "auth_token", "value": "your_auth_token"},
    {"name": "ct0", "value": "your_csrf_token"},
    {"name": "twid", "value": "your_twitter_id"}
  ]
}

Output Data

The scraper extracts the following data for each user:

Basic Information

• id - Twitter user ID
• username - Twitter handle (without @)
• name - Display name
• description - Bio/description

Social Metrics

• followers_count - Number of followers
• following_count - Number of accounts following
• tweets_count - Total number of tweets

Verification Details

• verified - Overall verification status (boolean)
• verification_details - Detailed verification information:
  • verified_type - Type of verification
  • blue_verified - Twitter Blue verification
  • legacy_verified - Legacy blue checkmark
  • government_verified - Government account verification
  • business_verified - Business account verification

Profile Assets

• profile_image_url - Standard profile image URL
• profile_image_url_hd - High-resolution profile image URL
• profile_banner_url - Profile banner image URL
• profile_background_image_url - Background image URL

Additional Data

• created_at - Account creation date
• location - User's location (if provided)
• url - Website URL (if provided)
• protected - Whether the account is private
• scraped_from - Username this data was scraped from
• scrape_type - Type of scraping performed

Usage Tips

1. Start Small: Begin with a low follower_limit to test your setup
2. Monitor Rate Limits: The scraper includes delays, but be mindful of Twitter's limits
3. Verified Users: Use verified filtering to get higher-quality data
4. Batch Processing: Process multiple usernames efficiently in one run
5. Data Storage: Results are automatically saved to Apify dataset

Rate Limiting

The scraper implements several rate limiting strategies:

• 1-second delay between pagination requests
• 2-second delay between different usernames
• Automatic error handling for rate limit responses

Error Handling

The scraper handles various error scenarios:

• Invalid usernames
• Authentication failures
• Network connectivity issues
• Rate limiting
• Private/suspended accounts

All errors are logged with detailed information for debugging.

Requirements

• Apify Account: Required to run this actor
• Valid Twitter Cookies: Fresh authentication cookies from an active Twitter session
• Internet Connection: For API requests to Twitter

Limitations

• Requires valid Twitter authentication
• Subject to Twitter's rate limiting
• Cannot access private accounts unless you follow them
• Suspended or deleted accounts will be skipped

Troubleshooting

Authentication Issues

• Ensure cookies are fresh (less than 24 hours old)
• Check that all required cookies are provided
• Verify cookie values are complete and unmodified

Rate Limiting

• Reduce follower_limit if hitting rate limits
• Increase delays between requests if needed
• Use fewer usernames per run

No Data Returned

• Verify usernames exist and are public
• Check if accounts have followers/following to scrape
• Ensure proper scrape_type is selected

Legal and Ethical Considerations

• Respect Twitter's Terms of Service
• Use scraped data responsibly
• Consider privacy implications
• Don't overwhelm Twitter's servers
• Comply with applicable data protection laws