Twitter (X) Profile Posts Scraper

Paste an X/Twitter profile URL and get that account's posts — for any period you choose. No official API needed. The actor opens the profile, auto-paginates, and exports every matching post as structured JSON: text, author, likes, retweets, replies, quotes, views, media, hashtags and timestamps. Download as JSON, CSV or Excel, or read it from the Apify API.

The output is built for pipelines: flat items with id_str, url, text, created_at, engagement counts and a full user author object.

Features

• 🔗 Just paste a profile URL — https://x.com/nasa, @nasa or nasa all work, and so does any link on the profile (a post URL resolves to its author). Scrape many profiles per run.
• 📅 Date-range filtering — From/To dates, both inclusive. Give a range and the actor automatically switches to X advanced search (from:handle since: until:), which jumps straight to the period instead of scrolling the whole timeline back to it.
• 🗂 Posts / Posts & replies / Media tabs — pick which part of the profile to read.
• 🎛 Content filters — replies, retweets, pinned post, language, has-media, has-links, questions, and minimum likes / retweets / replies.
• ♾ Auto-pagination — scrolls until your target count is reached, the date range is passed, or the feed runs dry (up to 5,000 posts).
• 💾 Incremental results — posts are pushed as each profile finishes, so long runs never lose collected data to a timeout.
• 🔐 Flexible authentication — session cookies, secret env vars, or fully automated login with session caching.
• 🛠 Debuggable failures — on empty runs the actor saves a screenshot and the page HTML to the run's key-value store.

Use cases

• Tracking a competitor's, executive's or brand's posting activity over a period
• Building a dated archive of an account's posts
• Social listening, sentiment analysis and engagement benchmarking
• Market and academic research

Quick start

1. Paste the profile URL (e.g. https://x.com/nasa).
2. Set From date / To date if you want a specific period.
3. Provide a session (see Authentication) — a throwaway X account's cookies or credentials.
4. Run. Posts appear in the dataset as each profile completes.

Input

Field	Type	Description
profileUrl	string	The profile to scrape. Full URL, @handle or bare handle. Several can be pasted one per line.
profileUrls	array	Multiple profiles — each scraped separately. numberOfTweets applies per profile.
tab	string	posts, replies (Posts & replies) or media. Default posts.
numberOfTweets	integer	Target posts per profile, after filtering (max 5000). Default 100.
scrapeAll	boolean	Auto-paginate until the target is reached. When false only the first page is returned. Default true.
timeSince / timeUntil	string	yyyy-mm-dd date bounds. Both inclusive, UTC.
scrapeMode	string	auto (search when dates are set, timeline otherwise), search, or timeline. Default auto.
includeReplies	boolean	Include the account's replies. Forced on for the replies tab. Default false.
includeRetweets	boolean	Include retweets; the original is exposed in retweeted_status. Default true.
includePinned	boolean	Include the pinned post (which can be far older than the rest). Default true.
onlyProfileAuthor	boolean	Drop posts written by other accounts (the replies tab pulls in parent posts as context). Default true.
language	string	ISO 639-1 code, e.g. en.
minFaves / minRetweets / minReplies	integer	Engagement floors. 0 disables.
mediaHasMedia / mediaHasLinks	boolean	Only posts with media / with links.
contentQuestionMarks	boolean	Questions only (search mode).
cookies	array	Cookies of a logged-in X session (Cookie-Editor / EditThisCookie JSON export).
twUsername / twPassword / twEmail	string	Credentials for automated login (2FA must be disabled).
proxyConfiguration	object	Proxy settings. Residential strongly recommended.
headless	boolean	Uncheck only for local debugging.

Scrape modes — which one to use

A profile timeline cannot be filtered by date: reaching a range from two years ago means scrolling past every post in between. X advanced search can, so:

	Advanced search (from:handle since: until:)	Profile timeline
Date ranges	✅ jumps straight to the period	⚠️ must scroll back to it
Engagement floors (min_faves…)	✅ applied server-side	filtered client-side
Pinned post	not marked	✅ detected (is_pinned)
Best for	any dated query	"just give me the newest posts"

auto picks search whenever a date is set, and the timeline otherwise. Both modes go through the same filters afterwards, so they return the same set.

Authentication

X login-walls profiles for anonymous visitors, so the actor needs a session. It resolves one in this priority order:

1. the cookies input (per-run),
2. the TW_COOKIES secret env var (JSON cookie array or a name=value; name2=value2 header string),
3. the session cached in the twitter-session key-value store by a previous automated login,
4. automated login with twUsername/twPassword (or the TW_USERNAME/TW_PASSWORD/TW_EMAIL secret env vars) — the fresh session is cached for future runs.

Always use a throwaway account, never a personal one. Accounts used for scraping can be restricted by X.

Output

One dataset item per post:

{
    "id": "1234567890123456789",
    "id_str": "1234567890123456789",
    "url": "https://x.com/nasa/status/1234567890123456789",
    "twitterUrl": "https://twitter.com/nasa/status/1234567890123456789",
    "text": "Post text...",
    "full_text": "Post text...",
    "lang": "en",
    "created_at": "2026-01-15T09:30:00.000Z",
    "createdTime": "2026-01-15T09:30:00.000Z",
    "favorites": 12,
    "retweets": 3,
    "replies": 4,
    "quotes": 1,
    "bookmarks": 2,
    "views": 4520,
    "hashtags": ["space"],
    "mentions": ["esa"],
    "urls": ["https://nasa.gov/article"],
    "media": [{ "type": "photo", "url": "https://pbs.twimg.com/media/...", "expanded_url": "...", "video_url": null }],
    "is_retweet": false,
    "is_quote": false,
    "is_reply": false,
    "is_pinned": false,
    "retweeted_status": null,
    "quoted_status": null,
    "in_reply_to_status_id_str": null,
    "in_reply_to_screen_name": null,
    "conversation_id_str": "1234567890123456789",
    "possibly_sensitive": false,
    "user": {
        "id_str": "987654321",
        "name": "NASA",
        "screen_name": "nasa",
        "url": "https://x.com/nasa",
        "description": "Explore the universe",
        "followers": 1500,
        "following": 300,
        "tweets": 8200,
        "verified": false,
        "blue_verified": true,
        "avatar": "https://pbs.twimg.com/profile_images/...",
        "created_at": "2015-03-01T00:00:00.000Z",
        "location": "Houston"
    },
    "username": "nasa",
    "authorName": "NASA",
    "author": { "id": "987654321", "name": "NASA", "username": "nasa", "profileUrl": "https://x.com/nasa" },
    "profileHandle": "nasa",
    "profileUrl": "https://x.com/nasa",
    "tab": "posts",
    "scrapeMode": "search",
    "scrapedAt": "2026-08-13T12:00:00.000Z"
}

Posts are pushed newest-first per profile.

How it works

The actor opens the profile (or its advanced-search equivalent) in a Playwright-driven Chrome with an authenticated session, intercepts the UserTweets / UserTweetsAndReplies / UserMedia / SearchTimeline GraphQL responses X streams to the page, and scrolls to auto-paginate. Parsing walks the GraphQL JSON generically instead of hardcoding one payload path, so it survives X's frequent payload-shape changes. Both feeds are reverse-chronological, so in timeline mode the scroll stops as soon as it drops below timeSince rather than reading the account's whole history.

FAQ & troubleshooting

Why do I get 0 results? Check the run log first — it names the cause. Most common: an expired/missing session, a blocked proxy IP, a date range in which the account posted nothing, or filters (includeReplies, includeRetweets, min*) excluding everything. DEBUG_SCREENSHOT and DEBUG_HTML in the run's key-value store show exactly what the browser saw.

A date range returns fewer posts than the profile shows. By default replies are excluded (includeReplies: false). Turn it on, or pick the "Posts & replies" tab.

Timeline mode with an old date range is slow. That's inherent — the timeline has no date filter, so it must be scrolled back to the range. Leave scrapeMode on auto and the actor uses search instead.

Login fails — what now? The error message says which check X raised (identity confirmation, confirmation code/2FA, Arkose CAPTCHA, suspension) and what to do about it. Logging the account in manually once from the proxy's region usually clears security checks. When in doubt, export cookies from a logged-in browser session and pass those instead.

Navigation timeouts? Use Apify Proxy with the RESIDENTIAL group; datacenter IPs are widely blocked by X.

Private/suspended profiles are skipped with a warning; the rest of the run continues.

Is this legal? The actor extracts only publicly visible data. You are responsible for complying with X's Terms of Service and applicable laws (e.g. GDPR) for your use case — consult a lawyer if unsure.