Twitter (X) Data Scraper - Fast and Efficient Tweet Extraction

This Actor provides a powerful and efficient way to scrape data from Twitter (now X). It allows you to extract tweets based on various criteria, making it ideal for researchers, entrepreneurs, data analysts, and anyone needing to gather Twitter data for analysis. It's designed for fetching historical data, not real-time monitoring.

Key Features

• Multiple Input Methods: Get the data you need, how you need it:

  • Start URLs: Provide direct URLs for tweets, user profiles, search results, or lists.
  • Search Terms: Use keywords and advanced search operators ( see Advanced Search Guide).
  • Twitter Handles: Extract tweets from specific users.

• Powerful Filtering: Refine your results with precise filtering:

  • Maximum Items: Control the number of tweets you retrieve.
  • Sorting: Get results sorted by "Top" or "Latest".
  • Language: Filter by language using ISO 639-1 codes (a full list of supported languages is available in the Actor's input settings).
  • User Verification: Target tweets from verified users or Twitter Blue subscribers.
  • Media Type: Extract only tweets containing images, videos, or quotes.
  • Date Range: Specify a start and end date to retrieve tweets within a specific timeframe.

• Advanced Options:

  • Include Search Terms: Add a field to each tweet showing the search term that matched it.
  • Custom Map Function: Transform the output data using a custom JavaScript function (for data shaping, not filtering).
  • Query Wizard: Easily create complex queries. (Available Tweet X.com Scraper)

• Lightning-Fast Performance:

  • Scrape tweets with incredible speed. (Tweet X.com Scraper: advertised as 30-80 tweets per second)

• Cost-Effective:

  • Competitive pricing. Tweet X.com Scraper: $0.50 per 1,000 tweets.

Use Cases

• Market Research: Analyze trends, sentiment, and brand perception.
• Academic Research: Gather data for social media studies, political science research, and more.
• Competitive Analysis: Track competitors' activities and public engagement.
• Data Journalism: Uncover insights and patterns within Twitter data.
• Historical Data Analysis: Explore past tweets for specific events or periods.

Quick Start Examples

Here are some example input configurations to get you started quickly:

1. Fetching Tweets from a Specific Profile (e.g., NASA) over Several Time Periods:

This example fetches English tweets from the NASA profile (@NASA) in multiple date ranges, sorted by latest.

1{
2    "includeSearchTerms": false,
3    "onlyImage": false,
4    "onlyQuote": false,
5    "onlyTwitterBlue": false,
6    "onlyVerifiedUsers": false,
7    "onlyVideo": false,
8    "searchTerms": [
9        "from:elonmusk AI since:2023-01-01 until:2023-03-01",
10        "from:elonmusk AI since:2023-03-01 until:2023-05-01",
11        "from:elonmusk AI since:2023-05-01 until:2023-07-01",
12        "from:elonmusk AI since:2023-07-01 until:2023-09-01",
13        "from:elonmusk AI since:2023-09-01 until:2023-12-01"
14    ],
15    "sort": "Latest",
16    "tweetLanguage": "en"
17}

2. Fetching Replies to a Specific Tweet with a Hashtag:

This example retrieves replies (in English, sorted by latest) to a specific tweet (replace tweet_id_here with the actual tweet ID) that include a specific hashtag (replace hashtag_here).

1{
2    "includeSearchTerms": false,
3    "onlyImage": false,
4    "onlyQuote": false,
5    "onlyTwitterBlue": false,
6    "onlyVerifiedUsers": false,
7    "onlyVideo": false,
8    "searchTerms": [
9        "conversation_id:tweet_id_here #hashtag_here"
10    ],
11    "sort": "Latest",
12    "tweetLanguage": "en"
13}

Explanation of Key Input Fields:

• searchTerms: This is the core of your query. You can use advanced search operators (as shown in the NASA example) or a conversation ID combined with a hashtag. Refer to the Advanced Search Guide for all the possibilities.
• sort: Choose "Latest" for chronological order or "Top" for the most relevant tweets.
• tweetLanguage: Specify the language code (e.g., "en" for English).
• only... (e.g., onlyImage, onlyVerifiedUsers): Boolean flags (true/false) to filter for specific tweet characteristics.
• includeSearchTerms: Set to true if you want an extra field in the output showing which search term matched each tweet.

Usage Guidelines (Important!)

• Historical Data Focus: This Actor is designed for retrieving historical data. Avoid using it for very frequent, real-time monitoring.
• Concurrency Limits: There are limits on concurrent runs (maximum 1, with up to 5 batched queries).
• Minimum Tweets Per Query: Aim to fetch at least 50 tweets per query (handle, keyword, etc.). This minimum increases proportionally when batching queries.
• Single Tweet Fetching Restrictions: Fetching a single tweet by URL is generally prohibited without explicit permission. Allow a few minutes interval time minimum between runs.
• Respect Twitter's Terms of Service: Always use this tool ethically and responsibly.

Output Example

1[
2    {
3        "id": "1630332507265589248",
4        "url": "https://x.com/NASA/status/1630332507265589248",
5        "twitterUrl": "https://twitter.com/NASA/status/1630332507265589248",
6        "full_text": "@yalhunzai We have internships open to international students from countries with whom we have agreements: https://t.co/Kl2Lz4qjpw",
7        "favorite_count": 14,
8        "retweet_count": 9,
9        "reply_count": 0,
10        "quote_count": 0,
11        "bookmark_count": 1,
12        "author": {
13            "id_str": "11348282",
14            "screen_name": "NASA",
15            "name": "NASA",
16            "advertiser_account_service_levels": [
17                "dso",
18                "media_studio"
19            ],
20            "advertiser_account_type": "promotable_user",
21            "analytics_type": "enabled",
22            "can_dm": false,
23            "can_media_tag": true,
24            "created_at": "Wed Dec 19 20:20:32 +0000 2007",
25            "description": "There's space for everybody. ✨\n\nVerification: https://t.co/8nok3NP4PW",
26            "entities": {
27                "description": {
28                    "hashtags": [],
29                    "symbols": [],
30                    "urls": [
31                        {
32                            "display_url": "nasa.gov/socialmedia",
33                            "expanded_url": "http://nasa.gov/socialmedia",
34                            "indices": [
35                                46,
36                                69
37                            ],
38                            "url": "https://t.co/8nok3NP4PW"
39                        }
40                    ],
41                    "user_mentions": []
42                },
43                "url": {
44                    "urls": [
45                        {
46                            "display_url": "nasa.gov",
47                            "expanded_url": "http://www.nasa.gov/",
48                            "indices": [
49                                0,
50                                23
51                            ],
52                            "url": "https://t.co/9NkQJKAnuU"
53                        }
54                    ]
55                }
56            },
57            "fast_followers_count": 0,
58            "favourites_count": 16401,
59            "followers_count": 85752559,
60            "friends_count": 170,
61            "geo_enabled": false,
62            "has_custom_timelines": true,
63            "has_extended_profile": true,
64            "is_translator": false,
65            "location": "Pale Blue Dot",
66            "media_count": 27374,
67            "normal_followers_count": 85752559,
68            "pinned_tweet_ids_str": [],
69            "profile_background_color": "000000",
70            "profile_banner_url": "https://pbs.twimg.com/profile_banners/11348282/1718393721",
71            "profile_image_url_https": "https://pbs.twimg.com/profile_images/1321163587679784960/0ZxKlEKB_normal.jpg",
72            "profile_interstitial_type": "",
73            "profile_link_color": "205BA7",
74            "protected": false,
75            "statuses_count": 72761,
76            "translator_type_enum": "Regular",
77            "url": "https://t.co/9NkQJKAnuU",
78            "verified": false,
79            "verified_type": "Government",
80            "withheld_in_countries": []
81        },
82        "bookmarked": false,
83        "conversation_id_str": "1630332507265589248",
84        "created_at": "Tue Feb 28 20:28:53 +0000 2023",
85        "display_text_range": [
86            11,
87            130
88        ],
89        "entities": {
90            "hashtags": [],
91            "symbols": [],
92            "timestamps": [],
93            "urls": [
94                {
95                    "display_url": "go.nasa.gov/3EKfKkN",
96                    "expanded_url": "https://go.nasa.gov/3EKfKkN",
97                    "indices": [
98                        107,
99                        130
100                    ],
101                    "url": "https://t.co/Kl2Lz4qjpw"
102                }
103            ],
104            "user_mentions": [
105                {
106                    "id_str": "1583456964453040129",
107                    "indices": [
108                        0,
109                        10
110                    ],
111                    "name": "Baki bùlen",
112                    "screen_name": "yalhunzai"
113                }
114            ]
115        },
116        "favorited": false,
117        "in_reply_to_screen_name": "yalhunzai",
118        "in_reply_to_status_id_str": "1630616311226048513",
119        "in_reply_to_user_id_str": "1583456964453040129",
120        "is_quote_status": false,
121        "lang": "en",
122        "possibly_sensitive": false,
123        "possibly_sensitive_editable": true,
124        "retweeted": false,
125        "user_id_str": "11348282"
126    }
127]

Troubleshooting

If you encounter issues like too few results, no results, or incomplete tweet collection, refer to the detailed troubleshooting section in the Actor's input schema or documentation. Common causes include overly restrictive filters or exceeding rate limits.

Get Started

Start scraping Twitter data today! This Actor provides a powerful, flexible, and cost-effective solution for all your Twitter data extraction needs.

🎹️ Fast TikTok API	📹️ TikTok Trend API	🔍️ TikTok Search API
🧛️ TikTok User API	🧛️ TikTok User Info API	#️ TikTok Hashtag API
🛍️ TikTok Shop API	👤️ TikTok Followers API