What does Twitter List Scraper do?

Twitter List Scraper will enable you to get more data from Twitter.com than the official Twitter.com API.

Twitter List Scraper can scrape:

• User information, such as name, Twitter handle (username), location, follower/following count, profile URL/image/banner, date of creation
• List of tweets
• Statistics for each tweet: number of favorites, replies, and retweets

Why scrape Twitter.com?

Twitter.com has 217 million active daily users and is a great source of data for marketing and research based on social media.

Here are just some of the ways you could use that data:

• Track discussions about your brand, products, country, or city.
• Monitor your competitors and their popularity, and get a competitive edge.
• Keep an eye on new trends and fashions as they emerge.
• Use the data to train AI models or for academic research.
• Track sentiment to make sure your investments are protected.
• Fight fake news by identifying the spreading pattern of misinformation.
• Explore discussions about travel destinations, services, amenities, and take advantage of local knowledge.
• Analyze consumer habits and develop new products or target underdeveloped niches.

If you would like more inspiration on how scraping Twitter.com could help your business or organization, check out our industry pages.

How to scrape Twitter.com

It's easy to scrape Twitter.com with Twitter List Scraper. Just follow these few steps and you'll get your data in a few minutes.

1. Click on Try for free.
2. Copy a list URL from Twitter.com.
3. Click on Start.
4. When Twitter List Scraper has finished, preview or download your data from the Dataset tab.

Results

The actor stores its results into the default dataset associated with the actor run. The data can be downloaded in machine-readable formats such as JSON, HTML, CSV or Excel.

Each item in the dataset will contain a separate tweet that follows this format:

1{
2    "user": {
3        "created_at": "2009-06-02T20:12:29.000Z",
4        "default_profile_image": false,
5        "description": "",
6        "fast_followers_count": 0,
7        "favourites_count": 16884,
8        "followers_count": 125195531,
9        "friends_count": 166,
10        "has_custom_timelines": true,
11        "is_translator": false,
12        "listed_count": 107060,
13        "location": "",
14        "media_count": 1358,
15        "name": "Elon Musk",
16        "normal_followers_count": 125195531,
17        "possibly_sensitive": false,
18        "profile_banner_url": "https://pbs.twimg.com/profile_banners/44196397/1576183471",
19        "profile_image_url_https": "https://pbs.twimg.com/profile_images/1590968738358079488/IY9Gx6Ok_normal.jpg",
20        "protected": false,
21        "screen_name": "elonmusk",
22        "statuses_count": 22020,
23        "translator_type": "none",
24        "verified": true,
25        "withheld_in_countries": [],
26        "id_str": "44196397"
27    },
28    "id": "1612699840571637760",
29    "conversation_id": "1612699840571637760",
30    "full_text": "All four orbital launch pads fully loaded with rockets for the first time!",
31    "reply_count": 2674,
32    "retweet_count": 3370,
33    "favorite_count": 38001,
34    "hashtags": [],
35    "symbols": [],
36    "user_mentions": [],
37    "urls": [],
38    "media": [],
39    "url": "https://twitter.com/elonmusk/status/1612699840571637760",
40    "created_at": "2023-01-10T06:36:19.000Z"
41}

How many results can you scrape with Twitter List scraper?

Twitter List scraper returns around 800 results on average (no matter the size of the list). However, you have to keep in mind that scraping twitter.com has many variables to it and may cause the results to fluctuate case by case. There’s no one-size-fits-all-use-cases number. The maximum number of results may vary depending on the complexity of the input, location, and other factors. Some of the most frequent cases are:

• website gives a different number of results depending on the type/value of the input
• website has an internal limit that no scraper can cross
• scraper has a limit that we are working on improving

Therefore, while we regularly run Actor tests to keep the benchmarks in check, the results may also fluctuate without our knowing. The best way to know for sure for your particular use case is to do a test run yourself.

How much will scraping Twitter List cost you?

When it comes to scraping, it can be challenging to estimate the resources needed to extract data as use cases may vary significantly. That's why the best course of action is to run a test scrape with a small sample of input data and limited output. You’ll get your price per scrape, which you’ll then multiply by the number of scrapes you intend to do.

Watch this video for a few helpful tips. And don't forget that choosing a higher plan will save you money in the long run.

Integrations and Twitter List Scraper

Last but not least, Twitter List Scraper can be connected with almost any cloud service or web app thanks to integrations on the Apify platform. You can integrate with Make, Zapier, Slack, Airbyte, GitHub, Google Sheets, Google Drive, and more. Or you can use webhooks to carry out an action whenever an event occurs, e.g. get a notification whenever Twitter List Scraper successfully finishes a run.

Using Twitter List Scraper with the Apify API

The Apify API gives you programmatic access to the Apify platform. The API is organized around RESTful HTTP endpoints that enable you to manage, schedule, and run Apify actors. The API also lets you access any datasets, monitor actor performance, fetch results, create and update versions, and more.

To access the API using Node.js, use the apify-client NPM package. To access the API using Python, use the apify-client PyPI package.

Check out the Apify API reference docs for full details or click on the API tab for code examples.

Need a more advanced Twitter scraper?

If you need more detailed data from Twitter or need to use different input, try our Twitter Scraper. It's a little more complicated to set up, but you can customize it as you want.

You can read our step-by-step tutorial on how to scrape Twitter if you need some guidance on how to run the scraper.

Want more tools for scraping Twitter?

🔎 Tweet Flash - Twitter Scraper	🔍 Twitter Search
🎥 Twitter Video Downloader	🔗 Twitter URL Scraper
🛰️ Twitter Spaces Scraper	🧭 Twitter Explorer

Not your cup of tea? Build your own scraper

Twitter List Scraper doesn’t exactly do what you need? You can always build your own! We have various scraper templates in Python, JavaScript, and TypeScript to get you started. Alternatively, you can write it from scratch using our open-source library Crawlee. You can keep the scraper to yourself or make it public by adding it to Apify Store (and find users for it).

Or let us know if you need a custom scraping solution.

Is it legal to scrape Twitter.com?

Note that personal data is protected by GDPR in the European Union and by other regulations around the world. You should not scrape personal data unless you have a legitimate reason to do so. If you're unsure whether your reason is legitimate, consult your lawyers. We also recommend that you read our blog post: is web scraping legal?.

Your feedback

We’re always working on improving the performance of our Actors. So if you’ve got any technical feedback for Twitter List Scraper or simply found a bug, please create an issue on the Actor’s Issues tab in Apify Console.