Twitter X Profile Scraper

What does Twitter X Profile Scraper do?

The Twitter (X) Profile & Tweets Scraper is an all-in-one data extraction tool designed to help you scrape complete X (formerly Twitter) profiles, extract individual tweets, and download media seamlessly. Whether you need a reliable Twitter profile scraper, a high-speed tweet scraper, or a dedicated X media scraper, this Actor delivers exactly what you need without requiring any official X API keys, login credentials, or authentication.

Built by MikoLabs, this powerful Apify Actor auto-detects your input and adapts its scraping mode. Simply provide usernames or profile URLs, and the scraper will instantly collect:

• Complete Profile Information (Profile Scraper) — Extract bio, followers, following count, location, website, join date, avatar, and banner images.
• Tweets and Posts (Tweet Scraper) — Scrape full text, timestamps, hashtags, mentions, URLs, and engagement metrics (likes, reposts, quotes).
• Media Content (Media Scraper) — Extract and download tweets containing only images, videos, and GIFs from any X account.
• Replies and Conversations — Scrape threaded reply tweets with full conversational context.
• Retweets and Quote Tweets — Collect shared posts including original author metadata.

Why scrape X (Twitter)?

X (Twitter) has over 500 million monthly active users and is one of the richest real-time data sources on the internet. As a top Twitter API alternative, this Apify Twitter scraper allows you to automatically scrape Twitter profiles and extract all their publicly available data.

By capturing details on user bios, tweet histories, follower counts, and engagement metrics, you can unlock vital insights for market research, sentiment analysis, and competitive intelligence.

Here are just some of the ways you could use that data:

• Brand monitoring — Track mentions, sentiment, and customer feedback in real-time.
• Competitive intelligence — Analyze competitor tweets, engagement strategies, and audience growth.
• Influencer research — Discover influencers by engagement rate, follower count, and content niche.
• Lead generation — Identify potential customers and prospects based on their public activity.
• Academic research — Build datasets for social media studies, NLP training, and public opinion analysis (often used alongside a Twitter scraper Python integration).
• Content strategy — Analyze what types of tweets drive the most engagement in your industry.
• News and trend tracking — Monitor breaking news, emerging trends, and viral content.
• Market sentiment — Track crypto projects, stock discussions, and financial market chatter.

If you would like more inspiration on how scraping X could help your business or organization, check out our industry pages.

How to scrape Twitter Profiles

It's incredibly easy to use this Twitter profile scraper app to scrape X data from any user profiles, such as tweets, reposts, images, replies, likes, views, followers, and following. Just follow these few steps and you'll get your data in minutes:

1. Click on Try for free.
2. Enter the Twitter handles you want to scrape (e.g. elonmusk, apify), or paste profile URLs.
3. Configure your scraping options — choose whether to include replies or filter by media only.
4. Set maximum number of tweets to collect (optional).
5. Click on Run.
6. When the scraper has finished, preview or download your data from the Dataset tab.

The Actor extracts profiles and tweets without any additional configuration needed.

Input

You can configure the scraper through the Apify Console input form or provide JSON input directly. Here are the key input fields:

Field	Type	Description
twitterHandles	string[]	Twitter usernames to scrape (with or without @)
twitterUrls	string[]	Direct profile URLs (e.g. https://x.com/elonmusk)
includeReplies	boolean	Include reply tweets in results
mediaOnly	boolean	Scrape only tweets with images or videos
maxItems	integer	Max tweets to collect (default: 20, max: 999)
proxyConfiguration	object	Proxy settings (residential recommended)
maxRetries	integer	Max retry attempts for failed requests (1-10, default: 5)

Example Input

{
    "twitterHandles": ["elonmusk", "apify"],
    "includeReplies": false,
    "mediaOnly": false,
    "maxItems": 999
}

Results

The scraper outputs clean, structured JSON data for each item. You can download the dataset in various formats such as JSON, CSV, Excel, XML, RSS, or HTML Table.

Profile Output

{
    "type": "profile",
    "username": "apify",
    "fullname": "Apify",
    "bio": "Web scraping and automation platform for data extraction",
    "followers": 12500,
    "following": 567,
    "tweets": 1234,
    "likes": 890,
    "avatar": "https://pbs.twimg.com/profile_images/...",
    "banner": "https://pbs.twimg.com/profile_banners/...",
    "website": "https://apify.com",
    "location": "San Francisco, CA",
    "joined": "January 2024",
    "scrape_date": "2024-10-24T12:00:00.000Z"
}

Tweet Output

{
    "type": "tweet",
    "username": "@apify",
    "fullname": "Apify",
    "text": "Excited to announce our new scraping features!",
    "tweet_url": "https://x.com/apify/status/123456",
    "tweet_id": "123456",
    "date": "Oct 24, 2024 · 12:00 PM UTC",
    "hashtags": ["#webscraping", "#automation"],
    "mentions": ["@user1", "@user2"],
    "urls": ["https://example.com"],
    "stats": {
        "comments": 12,
        "retweets": 34,
        "quotes": 5,
        "likes": 156
    },
    "media": [
        {
            "type": "image",
            "url": "https://pbs.twimg.com/media/...",
            "thumbnail": "https://pbs.twimg.com/media/..."
        }
    ],
    "is_retweet": false,
    "is_reply": false,
    "is_quoted": false,
    "author": {
        "username": "@apify",
        "fullname": "Apify",
        "avatar": "https://pbs.twimg.com/...",
        "verified": true
    },
    "scrape_date": "2024-10-24T12:00:00.000Z"
}

Data fields

Profile data

Field	Type	Description
type	string	Always "profile"
username	string	Twitter handle
fullname	string	Display name
bio	string	Profile biography
followers	integer	Follower count
following	integer	Following count
tweets	integer	Total tweet count
likes	integer	Total likes given
avatar	string	Profile image URL
banner	string	Banner image URL
website	string	Profile website link
location	string	Profile location
joined	string	Account join date
scrape_date	string	ISO timestamp of when data was scraped

Tweet data

Field	Type	Description
type	string	"tweet", "profile_search_tweet", etc.
username	string	Tweet author handle
text	string	Full tweet text
tweet_url	string	Direct link to tweet
date	string	Publication date
hashtags	string[]	Hashtags used
mentions	string[]	Users mentioned
stats.likes	integer	Like count
stats.retweets	integer	Retweet count
stats.comments	integer	Reply count
stats.quotes	integer	Quote tweet count
media	object[]	Attached images/videos
is_retweet	boolean	Whether it's a retweet
is_reply	boolean	Whether it's a reply

How much will it cost to scrape X (Twitter)?

Apify gives you $5 free usage credits every month on the Apify Free plan. You can scrape several hundred profiles or thousands of tweets per month with those credits, so you can get started completely free!

If you need to scrape more data regularly from X (Twitter), consider upgrading to a paid plan. We recommend our $49/month Starter plan - perfect for regular scraping tasks with substantial monthly allowances.

For larger-scale projects, the Scale plan offers even greater capacity and is ideal for enterprises with significant data collection needs.

Tips for scraping X (Twitter)

• Start small — Set maxItems to 50–100 on your first run to test the configuration before scaling up
• Media filtering — Use mediaOnly option to focus on tweets with images or videos
• Include/Exclude replies — Control whether to include reply tweets in your results
• Residential proxy recommended — Enable residential proxy in the proxy configuration for the most reliable results
• Adjust retry attempts — Increase maxRetries if you experience frequent request failures
• Parallel scraping — The Actor processes multiple handles in parallel for faster results
• Retry logic built-in — Failed requests are automatically retried with exponential backoff, so transient errors resolve themselves

Frequently asked questions

Can I scrape private Twitter profiles?

No. Twitter X Profile Scraper only works with public profiles. Private or protected accounts cannot be scraped.

Do I need a Twitter API key or account?

No. This Actor does not require any Twitter API credentials, cookies, or authentication. It works independently of the official Twitter API.

What export formats are available?

You can export your scraped data as JSON, CSV, Excel (XLSX), XML, RSS, or HTML Table directly from the Apify Console Dataset tab.

How many tweets can I scrape?

Set maxItems to control the number of tweets collected. Default is 20. Set it higher for more data, but be mindful of compute costs for very large collections.

Is it legal to scrape X (Twitter)?

Note that personal data is protected by GDPR in the European Union and by other regulations around the world. You should not scrape personal data unless you have a legitimate reason to do so. If you're unsure whether your reason is legitimate, consult your lawyers.

You should also review X (Twitter)'s Terms of Service before scraping. Always scrape responsibly, respect rate limits, and avoid collecting sensitive personal information.

We also recommend that you read our blog post: is web scraping legal?.

Resources and integrations

• Apify Documentation
• Python SDK Documentation
• Apify API Reference
• Integration with Make, Zapier, and other apps

Twitter X Profile Scraper integrates seamlessly with tools like Make (Integromat), Zapier, Google Sheets, Slack, and more through the Apify platform. Schedule recurring runs, set up webhooks, or access results via the REST API.

Telegram Notifications

Get instant alerts when your scraper encounters issues or returns zero items.

See ./TELEGRAM_SETUP.md for complete setup instructions including:

• Creating a Telegram bot
• Configuring credentials
• Receiving zero-items and error alerts
• Message examples and troubleshooting

Support and feedback

Having issues or have feedback? Report bugs or submit feature requests in the Issues tab on the Actor's page.

For custom scraping solutions or enterprise support, visit MikoLabs.