Twitter X Profile Scraper

What does Twitter X Profile Scraper do?

Twitter X Profile Scraper will enable you to get more data from X (Twitter) than the official X API — without any API key, authentication, or rate limit restrictions.

Built by MikoLabs, this Actor auto-detects what to collect — profiles, tweets, replies, or media — and delivers clean, structured data instantly. Simply provide usernames or profile URLs, and the scraper handles the rest.

Twitter X Profile Scraper can scrape:

• Complete profile information — bio, followers, following, location, website, join date, avatar, and banner
• Tweets and posts — full text, timestamps, hashtags, mentions, URLs, and engagement metrics
• Replies and conversations — threaded reply tweets with full context
• Media content — tweets containing images, videos, and GIFs
• Retweets and quote tweets — including original author metadata

Why scrape X (Twitter)?

X (Twitter) has over 500 million monthly active users and is one of the richest real-time data sources on the internet. It is a vital platform for market research, sentiment analysis, competitive intelligence, and trend tracking.

Here are just some of the ways you could use that data:

• Brand monitoring — Track mentions, sentiment, and customer feedback in real-time
• Competitive intelligence — Analyze competitor tweets, engagement strategies, and audience growth
• Influencer research — Discover influencers by engagement rate, follower count, and content niche
• Lead generation — Identify potential customers and prospects based on their public activity
• Academic research — Build datasets for social media studies, NLP training, and public opinion analysis
• Content strategy — Analyze what types of tweets drive the most engagement in your industry
• News and trend tracking — Monitor breaking news, emerging trends, and viral content
• Market sentiment — Track crypto projects, stock discussions, and financial market chatter

If you would like more inspiration on how scraping X could help your business or organization, check out our industry pages.

How to scrape X (Twitter)

It's easy to scrape X (Twitter) with Twitter X Profile Scraper. Just follow these few steps and you'll get your data in a few minutes.

1. Click on Try for free.
2. Enter the Twitter handles you want to scrape (e.g. elonmusk, apify), or paste profile URLs.
3. Configure your scraping options — choose whether to include replies or filter by media only.
4. Set maximum number of tweets to collect (optional).
5. Click on Run.
6. When Twitter X Profile Scraper has finished, preview or download your data from the Dataset tab.

The Actor extracts profiles and tweets without any additional configuration needed.

Input

You can configure the scraper through the Apify Console input form or provide JSON input directly. Here are the key input fields:

Field	Type	Description
twitterHandles	string[]	Twitter usernames to scrape (with or without @)
twitterUrls	string[]	Direct profile URLs (e.g. https://x.com/elonmusk)
includeReplies	boolean	Include reply tweets in results
mediaOnly	boolean	Scrape only tweets with images or videos
maxItems	integer	Max tweets to collect (Maximium: 999 per user)
proxyConfiguration	object	Proxy settings (residential recommended)
maxRetries	integer	Max retry attempts for failed requests (1-10, default: 5)

Example Input

{
    "twitterHandles": ["elonmusk", "apify"],
    "includeReplies": false,
    "mediaOnly": false,
    "maxItems": 999
}

Output

The scraper outputs clean, structured JSON data for each item. You can download the dataset in various formats such as JSON, CSV, Excel, XML, RSS, or HTML Table.

Profile Output

{
    "type": "profile",
    "username": "apify",
    "fullname": "Apify",
    "bio": "Web scraping and automation platform for data extraction",
    "followers": 12500,
    "following": 567,
    "tweets": 1234,
    "likes": 890,
    "avatar": "https://pbs.twimg.com/profile_images/...",
    "banner": "https://pbs.twimg.com/profile_banners/...",
    "website": "https://apify.com",
    "location": "San Francisco, CA",
    "joined": "January 2024",
    "scrape_date": "2024-10-24T12:00:00.000Z"
}

Tweet Output

{
    "type": "tweet",
    "username": "@apify",
    "fullname": "Apify",
    "text": "Excited to announce our new scraping features!",
    "tweet_url": "https://x.com/apify/status/123456",
    "tweet_id": "123456",
    "date": "Oct 24, 2024 · 12:00 PM UTC",
    "hashtags": ["#webscraping", "#automation"],
    "mentions": ["@user1", "@user2"],
    "urls": ["https://example.com"],
    "stats": {
        "comments": 12,
        "retweets": 34,
        "quotes": 5,
        "likes": 156
    },
    "media": [
        {
            "type": "image",
            "url": "https://pbs.twimg.com/media/...",
            "thumbnail": "https://pbs.twimg.com/media/..."
        }
    ],
    "is_retweet": false,
    "is_reply": false,
    "is_quoted": false,
    "author": {
        "username": "@apify",
        "fullname": "Apify",
        "avatar": "https://pbs.twimg.com/...",
        "verified": true
    },
    "scrape_date": "2024-10-24T12:00:00.000Z"
}

Data fields

Profile data

Field	Type	Description
type	string	Always "profile"
username	string	Twitter handle
fullname	string	Display name
bio	string	Profile biography
followers	integer	Follower count
following	integer	Following count
tweets	integer	Total tweet count
likes	integer	Total likes given
avatar	string	Profile image URL
banner	string	Banner image URL
website	string	Profile website link
location	string	Profile location
joined	string	Account join date
scrape_date	string	ISO timestamp of when data was scraped

Tweet data

Field	Type	Description
type	string	"tweet", "profile_search_tweet", etc.
username	string	Tweet author handle
text	string	Full tweet text
tweet_url	string	Direct link to tweet
date	string	Publication date
hashtags	string[]	Hashtags used
mentions	string[]	Users mentioned
stats.likes	integer	Like count
stats.retweets	integer	Retweet count
stats.comments	integer	Reply count
stats.quotes	integer	Quote tweet count
media	object[]	Attached images/videos
is_retweet	boolean	Whether it's a retweet
is_reply	boolean	Whether it's a reply

How much will it cost to scrape X (Twitter)?

Apify gives you $5 free usage credits every month on the Apify Free plan. You can scrape hundreds of tweets and profiles for free each month — more than enough for testing and small-scale projects.

But if you need to get more data regularly from X (Twitter), you should grab an Apify subscription. We recommend our $49/month Starter plan — perfect for regular monitoring, competitive research, and medium-scale data collection.

For enterprise-level needs with thousands of profiles and millions of tweets, the Scale plan at $499/month delivers the power and throughput you need.

Tips for scraping X (Twitter)

• Start small — Set maxItems to 50–100 on your first run to test the configuration before scaling up
• Media filtering — Use mediaOnly option to focus on tweets with images or videos
• Include/Exclude replies — Control whether to include reply tweets in your results
• Residential proxy recommended — Enable residential proxy in the proxy configuration for the most reliable results
• Adjust retry attempts — Increase maxRetries if you experience frequent request failures
• Parallel scraping — The Actor processes multiple handles in parallel for faster results
• Retry logic built-in — Failed requests are automatically retried with exponential backoff, so transient errors resolve themselves

Frequently asked questions

Can I scrape private Twitter profiles?

No. Twitter X Profile Scraper only works with public profiles. Private or protected accounts cannot be scraped.

Do I need a Twitter API key or account?

No. This Actor does not require any Twitter API credentials, cookies, or authentication. It works independently of the official Twitter API.

What export formats are available?

You can export your scraped data as JSON, CSV, Excel (XLSX), XML, RSS, or HTML Table directly from the Apify Console Dataset tab.

How many tweets can I scrape?

Set maxItems to control the number of tweets collected. Default is 20. Set it higher for more data, but be mindful of compute costs for very large collections.

Is it legal to scrape X (Twitter)?

Note that personal data is protected by GDPR in the European Union and by other regulations around the world. You should not scrape personal data unless you have a legitimate reason to do so. If you're unsure whether your reason is legitimate, consult your lawyers.

You should also review X (Twitter)'s Terms of Service before scraping. Always scrape responsibly, respect rate limits, and avoid collecting sensitive personal information.

We also recommend that you read our blog post: is web scraping legal?.

Resources and integrations

• Apify Documentation
• Python SDK Documentation
• Apify API Reference
• Integration with Make, Zapier, and other apps

Twitter X Profile Scraper integrates seamlessly with tools like Make (Integromat), Zapier, Google Sheets, Slack, and more through the Apify platform. Schedule recurring runs, set up webhooks, or access results via the REST API.

Support and feedback

Having issues or have feedback? Report bugs or submit feature requests in the Issues tab on the Actor's page.

For custom scraping solutions or enterprise support, visit MikoLabs.