Twitter Scraper & X.com API

Unlock the full potential of Twitter (formerly X.com) data with our powerful, fast, and efficient Twitter Scraper. Whether you are a researcher, data analyst, marketer, or entrepreneur, this X.com scraping tool allows you to extract historical tweets, user profiles, hashtags, and more without complex setups. Built for large-scale data extraction, it serves as the perfect alternative to the official Twitter API.

🚀 Key Features of the Twitter Scraper

Our Twitter data extraction solution provides advanced capabilities for precise scraping:

• Flexible Input Methods: Define your exact data target:
  • Start URLs: Provide direct links to tweets, user profiles, search results, or curated lists.
  • Search Terms: Utilize keywords, hashtags, and advanced search operators (see Advanced Search Guide).
  • Twitter Handles: Directly scrape tweets from specific users.
• Granular Filtering & Targeting: Refine your Twitter scraping to get only the data you need:
  • Maximum Items: Control the exact pagination and number of tweets extracted.
  • Sorting: Organize scraped results by "Top" or "Latest" tweets.
  • Language Filtering: Extract tweets in specific languages using ISO 639-1 codes.
  • User Verification Status: Target tweets posted exclusively by verified users or Twitter Blue (X Premium) subscribers.
  • Media-Specific Scraping: Extract only tweets that contain images, videos, or quotes.
  • Date Range: Programmatically scrape historical Twitter data by setting a since and until date.
• Advanced Data Processing:
  • Include Search Terms: Track the origin by appending the matching search term to each scraped tweet.
  • Custom Map Function: Clean and shape your JSON output on the fly with custom JavaScript.
  • Query Wizard: Build complex queries easily via the Apify platform interface.
• High-Speed & Cost-Effective:
  • Lightning-Fast: Scrapes between 30-80 tweets per second.
  • Affordable Pricing: Only $0.50 per 1,000 tweets extracted.

Need a step-by-step guide? Check out the tutorial from our development team: Tutorial: Scrape Twitter Data with Apify's Tweet X.com Scraper

🎯 Primary Use Cases for Twitter Data Extraction

Our X.com bot acts as a versatile crawler for numerous professional applications:

• Market Research & Social Listening: Monitor brand mentions, track real-time trends, and conduct extensive sentiment analysis.
• Academic & Scientific Research: Gather robust datasets for political science, sociology, and media studies.
• Competitive Analysis: Keep a pulse on competitor strategies, announcements, and audience engagement on Twitter.
• Data Journalism: Extract and investigate large datasets to uncover hidden patterns and public narratives.
• Historical Data Archiving: Fetch and archive past events, viral moments, or corporate communications.

⚡ Quick Start Examples

Below are standard input configurations (schema parameters) to quickly launch your Twitter Scraper task.

1. Fetching Historical Tweets from a Specific Profile (e.g., NASA):

This JSON configuration extracts English tweets from the @NASA profile across multiple date ranges, sorted by latest engagement.

{
    "includeSearchTerms": false,
    "onlyImage": false,
    "onlyQuote": false,
    "onlyTwitterBlue": false,
    "onlyVerifiedUsers": false,
    "onlyVideo": false,
    "searchTerms": [
        "from:elonmusk AI since:2023-01-01 until:2023-03-01",
        "from:elonmusk AI since:2023-03-01 until:2023-05-01",
        "from:elonmusk AI since:2023-05-01 until:2023-07-01",
        "from:elonmusk AI since:2023-07-01 until:2023-09-01",
        "from:elonmusk AI since:2023-09-01 until:2023-12-01"
    ],
    "sort": "Top",
    "tweetLanguage": "en"
}

2. Extracting Replies to a Specific Tweet with a Hashtag:

Scrape mentions and replies efficiently. Replace tweet_id_here and hashtag_here for targeted scraping.

{
    "includeSearchTerms": false,
    "onlyImage": false,
    "onlyQuote": false,
    "onlyTwitterBlue": false,
    "onlyVerifiedUsers": false,
    "onlyVideo": false,
    "searchTerms": [
        "conversation_id:tweet_id_here #hashtag_here"
    ],
    "sort": "Latest",
    "tweetLanguage": "en"
}

⚙️ Explanation of Key Input Parameters:

• searchTerms: The core engine of your scrape. Utilize queries, user handles, advanced search operators, or a combination of conversation_id and keywords.
• sort: Determine output sequence—use "Latest" for chronological ingestion or "Top" for high-engagement ranking.
• tweetLanguage: Isolate geo-linguistic data using country codes (e.g., "en" for English, "es" for Spanish).
• Boolean Filters (onlyImage, onlyVerifiedUsers, etc.): Toggle specific content traits to minimize noise in your scraped dataset.
• includeSearchTerms: Essential for batch processing tracking; attaches the triggering keyword to the final results.

⚠️ Usage Guidelines & Best Practices

To maximize the efficiency and longevity of your scraping tasks:

• Target Historical Data: This crawler is optimized for historical extraction rather than real-time polling or dashboard monitoring.
• Batch Your Queries: Ensure a minimum yield of 50 tweets per query. Expand date ranges or broader keywords to maintain efficiency.
• Throttle Concurrency: Limit simultaneous runs (max 1 concurrent run, up to 5 batched queries) to avoid blocks.
• Single Tweet Lookups: Scraping single independent tweets explicitly via URLs is heavily restricted. Space out individual checks with several minutes of interval time.
• Compliance First: Always respect X.com / Twitter terms of service. Employ this scraper responsibly.

🛡️ Ethical Web Scraping & Data Practices

This actor operates as an unauthenticated Twitter crawler. It securely processes and extracts publicly accessible and visible user data without logging into X.com, ensuring that we do not inherently bypass authenticated platform barriers.

We champion privacy by design:

• No private messages, non-public user details, or sensitive personal fields are parsed.
• The infrastructure operates via Apify—an EU-based platform compliant with major data protection frameworks (including GDPR where applicable). For more legal insights, read Apify's guide on web scraping legality.

Our technical extraction layers are designed to tread lightly on X.com’s servers, enabling ethical academic, corporate, and public research without degrading platform performance or inciting harm.

📄 Output Schema Example

The scraper outputs rich JSON arrays containing complete, structured tweet metadata perfectly formatted for databases and data pipelines:

[
    {
        "id": "1630332507265589248",
        "url": "https://x.com/NASA/status/1630332507265589248",
        "twitterUrl": "https://twitter.com/NASA/status/1630332507265589248",
        "full_text": "@yalhunzai We have internships open to international students from countries with whom we have agreements: https://t.co/Kl2Lz4qjpw",
        "favorite_count": 14,
        "retweet_count": 9,
        "reply_count": 0,
        "quote_count": 0,
        "bookmark_count": 1,
        "author": {
            "id_str": "11348282",
            "screen_name": "NASA",
            "name": "NASA",
            "advertiser_account_service_levels": [
                "dso",
                "media_studio"
            ],
            "advertiser_account_type": "promotable_user",
            "analytics_type": "enabled",
            "can_dm": false,
            "can_media_tag": true,
            "created_at": "Wed Dec 19 20:20:32 +0000 2007",
            "description": "There's space for everybody. ✨\n\nVerification: https://t.co/8nok3NP4PW",
            "entities": {
                "description": {
                    "hashtags": [],
                    "symbols": [],
                    "urls": [
                        {
                            "display_url": "nasa.gov/socialmedia",
                            "expanded_url": "http://nasa.gov/socialmedia",
                            "indices": [
                                46,
                                69
                            ],
                            "url": "https://t.co/8nok3NP4PW"
                        }
                    ],
                    "user_mentions": []
                },
                "url": {
                    "urls": [
                        {
                            "display_url": "nasa.gov",
                            "expanded_url": "http://www.nasa.gov/",
                            "indices": [
                                0,
                                23
                            ],
                            "url": "https://t.co/9NkQJKAnuU"
                        }
                    ]
                }
            },
            "fast_followers_count": 0,
            "favourites_count": 16401,
            "followers_count": 85752559,
            "friends_count": 170,
            "geo_enabled": false,
            "has_custom_timelines": true,
            "has_extended_profile": true,
            "is_translator": false,
            "location": "Pale Blue Dot",
            "media_count": 27374,
            "normal_followers_count": 85752559,
            "pinned_tweet_ids_str": [],
            "profile_background_color": "000000",
            "profile_banner_url": "https://pbs.twimg.com/profile_banners/11348282/1718393721",
            "profile_image_url_https": "https://pbs.twimg.com/profile_images/1321163587679784960/0ZxKlEKB_normal.jpg",
            "profile_interstitial_type": "",
            "profile_link_color": "205BA7",
            "protected": false,
            "statuses_count": 72761,
            "translator_type_enum": "Regular",
            "url": "https://t.co/9NkQJKAnuU",
            "verified": false,
            "verified_type": "Government",
            "withheld_in_countries": []
        },
        "bookmarked": false,
        "conversation_id_str": "1630332507265589248",
        "created_at": "Tue Feb 28 20:28:53 +0000 2023",
        "display_text_range": [
            11,
            130
        ],
        "entities": {
            "hashtags": [],
            "symbols": [],
            "timestamps": [],
            "urls": [
                {
                    "display_url": "go.nasa.gov/3EKfKkN",
                    "expanded_url": "https://go.nasa.gov/3EKfKkN",
                    "indices": [
                        107,
                        130
                    ],
                    "url": "https://t.co/Kl2Lz4qjpw"
                }
            ],
            "user_mentions": [
                {
                    "id_str": "1583456964453040129",
                    "indices": [
                        0,
                        10
                    ],
                    "name": "Baki bùlen",
                    "screen_name": "yalhunzai"
                }
            ]
        },
        "favorited": false,
        "in_reply_to_screen_name": "yalhunzai",
        "in_reply_to_status_id_str": "1630616311226048513",
        "in_reply_to_user_id_str": "1583456964453040129",
        "is_quote_status": false,
        "lang": "en",
        "possibly_sensitive": false,
        "possibly_sensitive_editable": true,
        "retweeted": false,
        "user_id_str": "11348282"
    }
]

🛠️ Troubleshooting

Encountering "no results" or low tweet compilation volumes? Review the detailed troubleshooting schema inside the Actor's documentation. Common stumbling blocks involve hyper-restrictive queries or temporarily hitting platform rate thresholds.

🚀 Get Started Now

Unlock the most comprehensive and flexible Twitter API alternative. Launch your tasks and begin extracting meaningful Twitter data today!

🎹️ Fast TikTok API	📹️ TikTok Trend API	🔍️ TikTok Search API
🧛️ TikTok User API	🧛️ TikTok User Info API	#️ TikTok Hashtag API
🛍️ TikTok Shop API	👤️ TikTok Followers API	⚡️ TikTok Scraper (pay-per-result)
💬 TikTok Comment API	🎶 TikTok Music API	🎶 TikTok Music Trend API
🎶 Tiktok Scraper Ultimate (pay-per-event)