Twitter User Following Scraper

A powerful Apify actor that scrapes Twitter (X) user following lists efficiently. Extract comprehensive profile data for all users that a specific Twitter account follows, including profile information, follower counts, verification status, and more. The actor saves data in real-time to the dataset, ensuring you can monitor progress and access results immediately.

Why Choose Us?

• Real-Time Data Saving: Results are saved to the dataset as they're scraped, so you can access data immediately without waiting for the entire run to complete
• Intelligent Proxy Management: Automatic proxy fallback system ensures maximum success rate - starts with no proxy, automatically falls back to datacenter, then residential proxies if needed
• Bulk Processing: Process multiple usernames in a single run, saving time and resources
• Comprehensive Data: Extract complete profile information including follower counts, verification status, profile images, descriptions, and more
• Flexible Input: Accept usernames, full URLs, or keywords - the actor automatically extracts and processes them
• Reliable & Robust: Built with error handling and retry logic to ensure successful data collection

Key Features

• ✅ Real-Time Data Saving: Watch results appear in the dataset as scraping progresses
• ✅ Smart Proxy Fallback: Automatic proxy switching (no proxy → datacenter → residential) with 3 retries
• ✅ Bulk Username Support: Process multiple Twitter accounts in one run
• ✅ Flexible Input Formats: Accept usernames, URLs (x.com/twitter.com), or keywords
• ✅ Comprehensive Profile Data: Extract 20+ fields per user including:
  • User ID, name, screen name
  • Profile and banner images
  • Follower/following counts
  • Verification status
  • Bio/description
  • Location and website
  • Account creation date
  • And more
• ✅ Progress Tracking: Real-time logs show pages fetched and users saved
• ✅ Error Handling: Robust error handling ensures maximum data collection success

Input

The actor accepts the following input parameters:

Input Schema

{
  "startUrls": [
    "jordanbpeterson",
    "https://x.com/elonmusk",
    "twitter.com/billgates"
  ],
  "maxResults": 100,
  "authToken": "",
  "ct0": "",
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

Input Parameters

Parameter	Type	Required	Description
startUrls	Array	Yes	List of Twitter usernames, URLs, or keywords. Supports multiple formats: - Username: "jordanbpeterson" - Full URL: "https://x.com/elonmusk" - Twitter URL: "twitter.com/billgates"
maxResults	Integer	No	Maximum number of following users to scrape per username. Default: 200. Range: 1-10000
authToken	String	No	Optional X/Twitter auth_token cookie. If not provided, the actor uses its built-in defaults.
ct0	String	No	Optional X/Twitter ct0 (CSRF) cookie. If provided, it is also used as the x-csrf-token header. If not provided, the actor uses its built-in defaults.
proxyConfiguration	Object	No	Proxy settings. By default, no proxy is used. If Twitter blocks requests, the actor automatically falls back to datacenter proxy, then residential proxy with 3 retries

Input Examples

Single Username:

{
  "startUrls": ["jordanbpeterson"],
  "maxResults": 200
}

Multiple Usernames:

{
  "startUrls": [
    "jordanbpeterson",
    "elonmusk",
    "billgates"
  ],
  "maxResults": 500
}

With URLs:

{
  "startUrls": [
    "https://x.com/jordanbpeterson",
    "twitter.com/elonmusk",
    "billgates"
  ],
  "maxResults": 1000
}

Output

The actor saves data to the Apify dataset in real-time. Each record contains comprehensive profile information for a following user.

Output Schema

{
  "username": "jordanbpeterson",
  "id": "17995040",
  "name": "Ben Shapiro",
  "screen_name": "benshapiro",
  "profile_image_url_https": "https://pbs.twimg.com/profile_images/...",
  "profile_banner_url": "https://pbs.twimg.com/profile_banners/...",
  "description": "Daily Wire co-founder, host of @benshapiroshow...",
  "followers_count": 8010425,
  "friends_count": 486,
  "statuses_count": 204599,
  "is_blue_verified": true,
  "verified_type": null,
  "created_at": "Tue Dec 09 16:40:22 +0000 2008",
  "profile_url": "https://x.com/benshapiro",
  "location": "",
  "url": "https://t.co/kvSlznhfDe",
  "favourites_count": 50,
  "media_count": 8629,
  "listed_count": 18882,
  "fast_followers_count": 0,
  "normal_followers_count": 8010425,
  "profile_image_shape": "Circle"
}

Output Fields

Field	Type	Description
username	String	Source username whose following list was scraped
id	String	Twitter user ID
name	String	Display name
screen_name	String	Twitter handle (without @)
profile_image_url_https	String	Profile image URL
profile_banner_url	String	Profile banner image URL
description	String	Bio/description text
followers_count	Integer	Number of followers
friends_count	Integer	Number of accounts following
statuses_count	Integer	Total number of tweets
is_blue_verified	Boolean	Whether account has blue verification
verified_type	String/null	Verification type
created_at	String	Account creation date
profile_url	String	Full profile URL
location	String	Location from profile
url	String	Website URL from profile
favourites_count	Integer	Number of liked tweets
media_count	Integer	Number of media posts
listed_count	Integer	Number of lists containing user
fast_followers_count	Integer	Fast followers count
normal_followers_count	Integer	Normal followers count
profile_image_shape	String	Profile image shape (Circle/Square)

🚀 How to Use the Actor (via Apify Console)

1. Log in to Apify Console and navigate to Actors
2. Find the twitter-user-following-scraper actor and click on it
3. Configure inputs:
   • Enter Twitter usernames or URLs in the startUrls field (one per line or as array)
   • Set maxResults to limit how many following users to scrape per username
   • Optionally configure proxy settings (recommended for large-scale scraping)
4. Run the actor by clicking the "Start" button
5. Monitor progress in real-time through the logs - watch as users are saved to the dataset
6. Access results in the OUTPUT tab - data appears in real-time as scraping progresses
7. Export results to JSON, CSV, Excel, or other formats from the dataset

Example Workflow

1. Enter usernames: ["jordanbpeterson", "elonmusk"]
2. Set max results: 200
3. Start the actor
4. Watch logs: See "Fetching page 1...", "Extracted and saved 20 users..."
5. Check OUTPUT tab: Results appear immediately as they're scraped
6. Export: Download complete dataset when finished

Best Use Cases

• Social Media Research: Analyze who influential accounts follow to understand their network
• Competitor Analysis: Discover who your competitors follow to identify potential connections
• Influencer Marketing: Find accounts that follow specific influencers for targeted outreach
• Network Mapping: Build relationship graphs by analyzing following patterns
• Lead Generation: Extract contact information and profile data for business development
• Market Research: Understand audience interests by analyzing who they follow
• Content Strategy: Discover content creators and thought leaders in specific niches

Frequently Asked Questions

How many users can I scrape per run?

The actor supports up to 10,000 users per username (configurable via maxResults). For multiple usernames, the total will be the sum of all scraped users.

Does the actor work without a proxy?

Yes! The actor starts with no proxy by default. It automatically switches to proxies only if Twitter blocks or rejects requests.

How long does scraping take?

Scraping speed depends on:

• Number of usernames
• Max results per username
• Twitter's response time
• Network conditions

Typically, 100 users take 1-2 minutes. The actor processes 20 users per page with a 1-second delay between pages.

Can I scrape private accounts?

No, the actor only scrapes publicly available following lists. Private accounts' following lists are not accessible.

Is the data saved in real-time?

Yes! Data is saved to the dataset immediately as each user is extracted. You don't need to wait for the entire run to complete to access results.

What happens if a request is blocked?

The actor automatically handles blocks with intelligent proxy fallback:

1. First attempt: No proxy
2. If blocked: Falls back to datacenter proxy
3. If still blocked: Falls back to residential proxy (with 3 retries)

Can I use this for commercial purposes?

Yes, you can use the scraped data for commercial purposes, but ensure you comply with:

• Twitter's Terms of Service
• Data protection regulations (GDPR, CCPA, etc.)
• Local laws regarding data collection and usage

How accurate is the data?

The actor extracts data directly from Twitter's API, ensuring high accuracy. All data reflects the current state of profiles at the time of scraping.

Support and Feedback

For custom solutions, feature requests, or support, please contact us at:

📧 Email: dev.scraperengine@gmail.com

We're always happy to help and improve the actor based on your feedback!

Cautions

⚠️ Important Notes:

• Public Data Only: This actor collects data only from publicly available sources. No data is taken from private accounts or password-protected content
• Rate Limiting: The actor includes built-in rate limiting (1 second delay between requests) to respect Twitter's servers
• Legal Compliance: The end user is responsible for ensuring legal compliance with:
  • Twitter's Terms of Service
  • Data protection and privacy laws (GDPR, CCPA, etc.)
  • Anti-spam regulations
  • Local data collection laws
• Ethical Use: Please use this actor responsibly and ethically. Do not use scraped data for spam, harassment, or other malicious purposes
• Data Accuracy: Profile data reflects the state at the time of scraping. Accounts may be deleted, suspended, or modified after scraping

---

Version: 0.1
Last Updated: 2025
Maintained by: Scraper Engine Team