X (Twitter) Profile Scraper

What does X (Twitter) Profile Scraper do?

X (Twitter) Profile Scraper is an X.com data extractor and Twitter scraper for collecting public profile-based data from X / Twitter accounts. You can scrape tweets, followers, following, affiliates, or user profile info from one or more accounts in a single run.

The input is simple: add profile URLs, usernames, or @handles, choose one category, and run the Actor. X (Twitter) Profile Scraper normalizes each profile identifier, paginates when needed, and saves the results to the Apify dataset. For tweet collection, it uses a latest-style profile query like from:elonmusk, paginates with max_id, and removes duplicates before saving.

Why use X (Twitter) Profile Scraper on Apify?

X (Twitter) Profile Scraper is useful when you need account-centric X data rather than keyword search results. Typical use cases include influencer research, creator discovery, follower audits, profile enrichment, lead generation, media monitoring, audience analysis, and tracking account activity over time.

It is also practical for repeatable workflows. You can point the same automation at different profile lists and switch only the category input depending on whether you want profile metadata, recent content, or network data such as followers and following.

Apify adds the platform capabilities that turn this scraper into a reusable data product:

• Run X (Twitter) Profile Scraper on demand or on a schedule
• Access results through API endpoints and webhooks
• Monitor runs, logs, and failures in one place
• Connect the dataset to automations and downstream tools
• Scale from quick tests to larger recurring jobs

What can this X (Twitter) scraper extract?

The Actor supports these categories:

• tweets
• followers
• following
• affiliates
• user_info

When category is tweets, you can optionally narrow results with profile tweet filters:

• tweetSubtype: all, replies, self_threads, quotes
• includeNativeRetweets
• mediaSubtype: any, media, images, videos
• requireLinks
• requireHashtags
• requireMentions

Accepted profile inputs:

• Full URLs such as https://x.com/elonmusk
• Plain usernames such as elonmusk
• Handles such as @elonmusk

The Actor also appends run metadata to each dataset item so you can trace where each record came from:

• metadata.category
• metadata.screenname
• metadata.sourceInput

What data can you extract from X / Twitter profiles?

The exact fields depend on the selected category and the returned data shape, but these are the main fields commonly available in the dataset:

Field	Description
tweet_id	Tweet identifier for tweet-like categories
text	Tweet or reply text
created_at	Original creation timestamp
screen_name	Account handle
name	Display name
user_id	Account ID
followers_count	Follower count when provided
friends_count	Following count when provided
statuses_count	Number of posts on the account
verified	Verification flag
blue_verified	Premium verification flag
profile_image / avatar	Profile image URL
header_image	Header image URL
description / desc	Bio text
website	Website URL from profile
location	Profile location
views, retweets, replies, favorites	Engagement fields for tweet-like results

How to scrape X / Twitter profile data

1. Open the Actor in Apify Console.
2. Go to the Input tab.
3. Add one or more X profile URLs, usernames, or @handles to urls.
4. Select a single category for the whole run.
5. If category is tweets, optionally configure tweet subtype and content filters.
6. Set maxResults if you want to cap paginated outputs.
7. Enable parseAllResults only if you want the Actor to keep going until no more pages are available.
8. Start the run.
9. Open the Output tab or dataset API to inspect and download the results.

If you are looking for terms like Twitter profile scraper, X profile scraper, download tweets, or X.com profile data extractor, this Actor is designed for exactly that profile-centric workflow.

Input

The Actor is configured from the Apify Input tab. Example input:

{
    "urls": ["https://x.com/elonmusk", "nasa", "@apify"],
    "category": "tweets",
    "tweetSubtype": "all",
    "includeNativeRetweets": false,
    "mediaSubtype": "any",
    "requireLinks": false,
    "requireHashtags": false,
    "requireMentions": false,
    "maxResults": 100,
    "parseAllResults": false
}

Input fields:

• urls: Array of X / Twitter profile URLs, usernames, or @handles
• category: One category to scrape for all inputs in the run
• tweetSubtype: For tweet runs, narrow results to all posts, replies, self-threads, or quote tweets
• includeNativeRetweets: Include native retweets in tweet results
• mediaSubtype: For tweet runs, restrict results to media, images, or videos
• requireLinks: Only keep tweets with links
• requireHashtags: Only keep tweets with hashtags
• requireMentions: Only keep tweets with mentions
• maxResults: Maximum number of items to save per profile for paginated categories
• parseAllResults: If enabled, the Actor ignores maxResults and keeps paginating until no more results are returned

If your deployment uses API credentials via environment variables, configure them before running the Actor.

Output

The Actor stores results in the default dataset. You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

Example output:

[
    {
        "tweet_id": "1771234567890123456",
        "created_at": "Wed Mar 20 12:34:56 +0000 2024",
        "text": "Example post text",
        "screen_name": "elonmusk",
        "name": "Elon Musk",
        "views": 125034,
        "retweets": 1820,
        "replies": 490,
        "favorites": 21455,
        "metadata": {
            "category": "tweets",
            "screenname": "elonmusk",
            "sourceInput": "https://x.com/elonmusk"
        }
    },
    {
        "screen_name": "apify",
        "name": "Apify",
        "followers_count": 19234,
        "friends_count": 321,
        "verified": false,
        "description": "Web scraping and browser automation platform.",
        "metadata": {
            "category": "user_info",
            "screenname": "apify",
            "sourceInput": "@apify"
        }
    }
]

How much does it cost to scrape X / Twitter profiles?

Cost depends mainly on three things: how many profiles you process, which category you select, and how deep pagination goes.

• user_info is usually the cheapest category because it typically needs only one request per profile.
• tweets can require multiple pages depending on account activity and how narrow or broad your tweet filters are.
• followers, following, and affiliates can become the most expensive categories because they may paginate deeply.

To keep costs predictable:

• Leave parseAllResults disabled unless you truly need complete coverage.
• Set a realistic maxResults.
• Split large profile lists into smaller runs so they are easier to monitor and retry.

For larger runs, total cost depends on both Actor runtime and how much data you choose to collect.

Tips and advanced options

• Use plain usernames when your source data is already normalized.
• Invalid profile inputs are skipped instead of crashing the whole run.
• Failed API calls are retried up to 4 times with a short delay.
• If one profile fails, the Actor logs the error and continues with the remaining inputs.
• Tweet pagination uses max_id and removes duplicates by tweet_id before saving.
• parseAllResults is convenient, but it should be used carefully on large accounts.

Related resources

• Related Actor: X/Twitter Advanced Search Scraper
• Related Actor: X/Twitter Followers Scraper
• Learn more about Apify integrations
• See the Apify API reference

FAQ, disclaimers, and support

Is it legal to scrape X / Twitter?

This Actor is intended for public web data workflows. You are responsible for making sure your use complies with X terms, local law, privacy rules, and your internal compliance requirements. Do not use scraped data in ways that violate platform rules or applicable regulations.

Why do fields vary between runs or categories?

Different categories return different objects, and some accounts may expose more or fewer fields than others.

Why did my run return fewer items than expected?

Common reasons are invalid input handles, temporary request failures, API-side limits, strict tweet filters, or stopping at maxResults. If you need deeper coverage, increase maxResults, loosen tweet filters, or enable parseAllResults.

Where can I get help or request changes?

Use the Actor Issues tab on Apify for bug reports, field requests, or workflow-specific adjustments. If you need a custom version for your pipeline, that is the right place to start the discussion.