Twitter X Tweets Scraper & Post Replies

Twitter X Scraper — Extract Posts, Replies and Profile Data

Twitter X Scraper reads public X (Twitter) profile timelines and single posts, then opens the conversation under each post and attaches the replies to that same row as a post_replies array — one post, one row, full thread included. Every response is structured JSON: post text, engagement counts, author profile fields, media, polls, quoted posts, and reply objects with their own author and engagement data, ready to pass directly to an LLM, index into a vector store, or feed a monitoring pipeline. Point it at a handle, a numeric user ID, or a single post URL and start collecting.

What is Twitter X Scraper?

Twitter X Scraper is an X (Twitter) scraper that collects public post data from profile timelines, numeric user IDs, or single post URLs, and returns it as structured JSON rows. What sets it apart from a plain post scraper: it also opens the TweetDetail conversation under each collected post and attaches the replies people actually wrote — the text, the replier's profile, and how the reply itself performed — to that same post's row as post_replies, so one post stays one row with the whole thread riding on it.

No X account is needed to collect posts, profiles, media, polls or quoted posts — that surface is keyless (a guest token). An X session (auth_token + ct0 cookies) is required whenever reply collection is on (the default) or when pulling a profile's own "Posts and replies" tab, because X serves both of those surfaces only to logged-in callers.

Key capabilities:

• 🔎 Profile timelines by handle, @handle, X/Twitter URL, or numeric user ID — up to maxTweets posts each
• 🔗 Single post URLs — one row per post, ignoring the post limit, the fastest way to read one conversation
• 💬 The reply conversation under each post, ranked by Likes, Relevance, or Recency, or unioned across all three with deepReplySweep
• 👥 A follower-count floor (minReplierFollowers) to screen replies by the size of the account that wrote them
• ↩️ A profile's own replies to other people, collected as posts in their own right (includeReplies)
• ↕️ Row order control (sortOrder) and a "keep only posts with replies" filter (onlyRepliedPosts)

What data can you get with Twitter X Scraper?

Every run returns one JSON row per post, plus an uncharged accounting row for any target that produced nothing.

Result type	Extracted fields	Primary use case
Post	type, id, url, user_posted, name, description, date_posted, likes, replies, reposts, quotes, bookmarks, views, lang, conversation_id, is_repost, reposted_by, is_quote, is_reply, in_reply_to_user, in_reply_to_post_id, is_pinned, hashtags, tagged_users	Timeline export, engagement tracking, content archiving
Author profile (embedded on every post row)	is_verified, is_blue_verified, verified_type, followers, following, posts_count, profile_image_link, biography, external_url	Audience sizing, bio-link extraction, verification checks
Conversation (post_replies[])	reply_id, text, date_posted, url, username, name, user_id, followers, is_blue_verified, biography, profile_image_link, likes, replies, reposts, views, photos, replied_to_post_id, is_reply_to_author, is_direct_reply	Customer-service audits, thread sentiment, reply-guy screening
Reply completeness (on the post row)	has_replies, replies_declared, replies_returned, replies_recovery_ratio, replies_truncated, replies_dropped_by_filter, reply_pages_fetched, reply_ranking_used, reply_fetch_status	Knowing exactly how much of a conversation you actually got
Media	photos, videos, media[] (type, image_url, alt_text, width, height, video_url, duration_ms, expanded_url)	Creative audits, video archiving
Quoted post	quoted_post (data_posted, description, post_id, profile_id, profile_name, profile_username, url, videos)	Quote-post tracking, narrative spread
Poll	poll (card_type, choices[], total_votes, counts_are_final, duration_minutes, end_datetime_utc, last_updated_datetime_utc)	Live poll results and sentiment
Link card	link_preview (card_type, title, description, domain, url)	Outbound-link and referral tracking
Accounting row	type, id, target, errorReason, message, input	Debugging a partial run — never charged

💬 The conversation under each post

This is the capability the base X posts scraper doesn't have: every collected post's conversation is opened through X's TweetDetail surface and the replies are attached to that post's own row, so a spreadsheet built on posts alone simply gains columns rather than needing a second dataset joined by post ID. Each reply carries its own text, its author's handle, display name, user ID, follower count, bio and avatar, the reply's own likes/replies/reposts/views, any photos, which post or reply it answers (replied_to_post_id), and two flags — is_reply_to_author (the post's own author replying in-thread) and is_direct_reply (answers the focal post, not a branch). Completeness is reported rather than assumed: replies_declared is X's own count for the post, replies_returned is how many made it into post_replies after the cap and the follower floor, and replies_recovery_ratio is the ratio between them.

{
  "post_replies": [
    {
      "reply_id": "1888888888888888901",
      "text": "This is huge, been waiting for this.",
      "username": "replierone",
      "followers": 3200,
      "is_direct_reply": true,
      "is_reply_to_author": false
    }
  ]
}

👤 Author profile fields

Every row — post or reply — carries the author's profile fields as they stood at fetch time: followers, following, posts_count, biography, external_url (the expanded destination, not the t.co shortener), profile_image_link, and is_blue_verified. This is what powers minReplierFollowers — screening a conversation by the size of the accounts that wrote into it, which matters now that X Premium made the blue badge purchasable and no longer a reliable signal of an established account on its own.

Why not build an X (Twitter) scraper yourself?

X's GraphQL operation and query IDs are undocumented and rotate without notice, so a hand-rolled scraper hardcoding them breaks the next time X ships a frontend build. This Actor discovers them live from X's own web client bundle at runtime — with a full webpack-chunk crawl as a fallback and a 6-hour cache — and refuses to start a run rather than call a stale ID. It also handles the parts that are easy to get wrong silently: each GraphQL operation has its own required HTTP verb (a wrong-verb call returns a 404 and still burns your rate-limit bucket), the reply surface (TweetDetail) returns a 0-byte 404 body to a logged-out caller on both GET and POST rather than a helpful error, and a 429 needs different handling depending on whether the caller is a rotatable guest token or a fixed authenticated session. Reproducing all of that — plus browser TLS fingerprinting to avoid an immediate block — is a maintenance burden most teams don't want to own for what should be a one-line input.

What is the difference between a post scraper and a reply (conversation) scraper?

A post scraper returns a post's own text and its summary counts — how many likes, reposts, and replies it has. A reply (conversation) scraper goes one step further and opens the thread underneath, returning what those replies actually say and who wrote them. X's own timeline surfaces never expose reply content — only the reply count — so reading the conversation itself requires a different, login-gated GraphQL surface (TweetDetail) that a plain post scraper never calls.

The distinction matters for anything that depends on what people actually said back: is a spike in replies genuine customer complaints or bot noise, is a launch post landing well or getting dragged, which accounts are driving the conversation. A post-only scraper can tell you a number changed; it can't tell you why.

Twitter X Scraper returns both on the same row: the post's own fields plus its conversation, arriving as the post_replies array alongside replies_declared / replies_returned so you can see exactly how much of the conversation was captured.

How to scrape X (Twitter) with Twitter X Scraper?

1. Open Twitter X Scraper on the Apify Store and click Try for free (or Run, if you already have it).
2. Paste one or more targets into startUrls — a profile URL, @handle, numeric user ID, or a single post URL.
3. Set maxTweets (posts per profile, 1–1000) and, if you want the conversation, leave collectPostReplies on and add auth_token + ct0 from an x.com browser session.
4. Adjust the optional controls — sortOrder, replyRanking, deepReplySweep, minReplierFollowers, onlyRepliedPosts, includeReplies — and start the run.
5. Download the dataset as JSON, CSV, or Excel from the Apify Console, or pull it with the API / apify_client once the run finishes.

{
  "startUrls": ["https://x.com/NASA", "https://x.com/NASA/status/2082511887757881648"],
  "maxTweets": 10,
  "collectPostReplies": true,
  "maxRepliesPerPost": 20,
  "replyRanking": "Likes",
  "auth_token": "your_auth_token_cookie_value",
  "ct0": "your_ct0_cookie_value"
}

How to run multiple targets in one job

startUrls is a list — mix profile URLs, handles, numeric user IDs, and single post URLs freely in one run; each is classified and processed independently. Targets are worked through one at a time, in the order given — there is no concurrency setting in this Actor's input, so a large batch of profiles with reply collection on will take proportionally longer, not shorter, than fewer targets.

⬇️ Input

Parameter	Required	Type	Description	Example value
startUrls	Yes	array	Profile URLs, @handles, numeric user IDs, or single post URLs — one per line	["https://x.com/NASA"]
maxTweets	No	integer	Posts to collect per profile (1–1000, default 10). Single post URLs ignore this	10
sortOrder	No	string (enum: recent, oldest, popular)	Re-orders the posts already collected in this run (default recent)	recent
collectPostReplies	No	boolean	Open each post's conversation and attach the replies (default true)	true
maxRepliesPerPost	No	integer	Replies kept per post (1–200, default 20)	20
replyRanking	No	string (enum: Likes, Relevance, Recency)	Which ranking X should serve first (default Likes)	Likes
deepReplySweep	No	boolean	Sweep all three rankings and merge by reply ID (default false)	false
minReplierFollowers	No	integer	Keep only replies whose author has at least this many followers (default 0)	0
onlyRepliedPosts	No	boolean	Drop posts whose conversation returned no replies (default false)	false
includeReplies	No	boolean	Also collect the profile's own replies to others, as posts (default false)	false
auth_token	No	string (secret)	The auth_token cookie from an x.com browser session	—
ct0	No	string (secret)	The ct0 cookie from the same x.com session	—
proxyConfiguration	No	object	Apify Proxy configuration (default {"useApifyProxy": false})	{"useApifyProxy": false}

Common pitfall: collectPostReplies defaults to true, so a run started with only startUrls filled in and no auth_token / ct0 fails immediately with a status message naming TweetDetail — it does not quietly return posts with empty post_replies arrays. Either add the session cookies, or switch collectPostReplies off to run the public timeline scrape with no login at all.

⬆️ Output

Results are typed, normalized JSON — one row per post, with the same 40 base fields and 10 reply-tracking fields on every row regardless of which targets or options you used. Export from the Apify Console as JSON, CSV, Excel, or an RSS/HTML table, or fetch the dataset through the API or apify_client.

Scraped results

[
  {
    "type": "tweet",
    "id": "1888888888888888888",
    "url": "https://x.com/examplehandle/status/1888888888888888888",
    "user_posted": "examplehandle",
    "name": "Example Handle",
    "description": "Our new feature just shipped. Tell us what you think!",
    "date_posted": "2026-08-01T14:32:00.000Z",
    "likes": 412,
    "replies": 96,
    "reposts": 58,
    "quotes": 3,
    "bookmarks": 21,
    "views": 18420,
    "lang": "en",
    "conversation_id": "1888888888888888888",
    "is_repost": false,
    "reposted_by": null,
    "is_quote": false,
    "is_reply": false,
    "in_reply_to_user": null,
    "in_reply_to_post_id": null,
    "is_pinned": false,
    "is_verified": true,
    "is_blue_verified": true,
    "verified_type": null,
    "followers": 54210,
    "following": 812,
    "posts_count": 3390,
    "profile_image_link": "https://pbs.twimg.com/profile_images/000/example.jpg",
    "biography": "Building things in public.",
    "external_url": "https://example.com",
    "hashtags": ["launch"],
    "tagged_users": null,
    "photos": null,
    "videos": null,
    "media": null,
    "quoted_post": null,
    "poll": null,
    "link_preview": null,
    "input": { "url": "https://x.com/examplehandle/status/1888888888888888888" },
    "has_replies": true,
    "replies_declared": 96,
    "replies_returned": 3,
    "replies_recovery_ratio": 0.0313,
    "replies_truncated": true,
    "replies_dropped_by_filter": 0,
    "reply_pages_fetched": 12,
    "reply_ranking_used": "Likes",
    "reply_fetch_status": "capped",
    "post_replies": [
      {
        "reply_id": "1888888888888888901",
        "text": "This is huge, been waiting for this.",
        "date_posted": "2026-08-01T14:40:00.000Z",
        "url": "https://x.com/replierone/status/1888888888888888901",
        "username": "replierone",
        "name": "Replier One",
        "user_id": "9001",
        "followers": 3200,
        "is_blue_verified": false,
        "biography": "Product nerd.",
        "profile_image_link": "https://pbs.twimg.com/profile_images/000/one.jpg",
        "likes": 44,
        "replies": 1,
        "reposts": 0,
        "views": 980,
        "photos": null,
        "replied_to_post_id": "1888888888888888888",
        "is_reply_to_author": false,
        "is_direct_reply": true
      },
      {
        "reply_id": "1888888888888888902",
        "text": "Any timeline for the EU rollout?",
        "date_posted": "2026-08-01T14:45:00.000Z",
        "url": "https://x.com/replier_two/status/1888888888888888902",
        "username": "replier_two",
        "name": "Replier Two",
        "user_id": "9002",
        "followers": 812,
        "is_blue_verified": true,
        "biography": null,
        "profile_image_link": "https://pbs.twimg.com/profile_images/000/two.jpg",
        "likes": 9,
        "replies": 1,
        "reposts": 0,
        "views": 210,
        "photos": null,
        "replied_to_post_id": "1888888888888888888",
        "is_reply_to_author": false,
        "is_direct_reply": true
      },
      {
        "reply_id": "1888888888888888903",
        "text": "Next month — working through the app store review.",
        "date_posted": "2026-08-01T15:02:00.000Z",
        "url": "https://x.com/examplehandle/status/1888888888888888903",
        "username": "examplehandle",
        "name": "Example Handle",
        "user_id": "500",
        "followers": 54210,
        "is_blue_verified": true,
        "biography": "Building things in public.",
        "profile_image_link": "https://pbs.twimg.com/profile_images/000/example.jpg",
        "likes": 61,
        "replies": 2,
        "reposts": 0,
        "views": 1500,
        "photos": null,
        "replied_to_post_id": "1888888888888888902",
        "is_reply_to_author": true,
        "is_direct_reply": false
      }
    ]
  },
  {
    "type": "error",
    "id": null,
    "target": "https://x.com/protectedaccount",
    "errorReason": "protected_account",
    "message": "This account is protected - X publishes its posts only to approved followers.",
    "input": { "url": "https://x.com/protectedaccount" }
  }
]

Two things worth knowing before you build a pipeline on this shape:

• is_verified currently mirrors is_blue_verified value-for-value on every row — it is not the legacy manual-verification flag. Use verified_type if you need to distinguish verification categories.
• quoted_post.data_posted is spelled that way on purpose (not date_posted) — it is the shipped field name and renaming it downstream would be the mistake, not this README.

Error/accounting rows (type: "error") are pushed without the row_result charged event, so they never bill. Filter them out of billing math with type != "error", or filter them into a review queue with type == "error".

How can I use the data extracted with Twitter X Scraper?

• 📣 Social and community teams — pull post_replies under your own brand's posts, filter by minReplierFollowers to separate real customer signal from low-follower noise, and route the ones with negative-leaning text into a support queue.
• 🤖 AI engineers and LLM developers — an agent issues a query with startUrls, receives structured JSON back (post + full conversation), and passes it to the model as grounded context for a "what's the reaction to X" question.
• 📊 Market and brand researchers — track replies_returned and replies_recovery_ratio across a set of competitor accounts to see where conversations are actually happening versus where a post just racked up passive likes.
• 🔬 Trust-and-safety / thread researchers — use replied_to_post_id and is_direct_reply to reconstruct which replies branch off other replies versus answer the focal post directly, without needing X's own (undocumented) thread UI.

How do you monitor a conversation over time?

Monitoring a conversation means re-running the same post or profile targets on a schedule and diffing what came back against the last run, rather than treating one snapshot as the final word — replies keep arriving after a post is published, and replies_returned on run 1 is rarely the same as run 3. The fields worth diffing between runs: the set of reply_id values inside post_replies (new IDs are new replies), replies_declared versus replies_returned (a widening gap can mean the conversation grew faster than your maxRepliesPerPost cap), and followers on newly-seen repliers (a sudden reply from a high-follower account is often the thing you actually want alerted on).

A concrete loop: schedule a run across a fixed list of post URLs or profile handles → on each run, diff the current post_replies reply-ID set against the previous run's dataset → alert when a new reply's author clears a follower threshold, or when replies_dropped_by_filter spikes (a sign the conversation just got noisier). Apify Console Schedules runs this on a cron interval without any code; the dataset from each run is available through the API or apify_client immediately after it finishes for whatever diffing logic you build on top.

Integrate Twitter X Scraper and automate your workflow

Twitter X Scraper works with any language or tool that can send an HTTP request — it's an Apify Actor, so it's callable through the Apify API or any of Apify's official client libraries.

REST API with Python

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_API_TOKEN>")

run = client.actor("scrapier/twitter-x-tweets-scraper-scraper-and-post-replies").call(
    run_input={
        "startUrls": ["https://x.com/NASA/status/2082511887757881648"],
        "collectPostReplies": True,
        "auth_token": "your_auth_token_cookie_value",
        "ct0": "your_ct0_cookie_value",
    }
)

for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    if row.get("type") == "tweet":
        print(row["url"], row["replies_returned"], "replies collected")

Confirm the exact Actor ID on the API tab of this Actor's Console page if it differs from the slug above.

Scheduled monitoring and delivery

Set up a recurring run from Apify Console Schedules (cron-based) to re-run a fixed target list at an interval you choose. There is no built-in webhook payload specific to this Actor, but Apify's platform-level Webhooks can trigger on any run event (e.g. ACTOR.RUN.SUCCEEDED) to push a notification or call another service once a scheduled run finishes.

⚖️ Is it legal to scrape X (Twitter) posts and replies?

Yes — scraping publicly accessible content is generally lawful; this Actor returns only what any logged-out or logged-in visitor can already see on x.com, never content from protected or suspended accounts. In the US, hiQ Labs, Inc. v. LinkedIn Corp., 938 F.3d 985 (9th Cir. 2019), is widely cited for the principle that accessing publicly available web data does not violate the Computer Fraud and Abuse Act — it concerned a different platform, so treat it as persuasive context, not a guarantee for every jurisdiction.

What this Actor collects is personal data about identifiable people: post authors and, when reply collection is on, every replier's handle, name, bio, and follower count. That puts GDPR (EU/UK) and CCPA (California) squarely in scope for how you store and process it, regardless of the data being public. Scraping for aggregate research or short-lived monitoring generally carries a different risk profile than scraping for AI training or long-term storage of identifiable people's data. Consult your legal team for commercial use cases involving bulk data storage.

❓ Frequently asked questions

Does Twitter X Scraper need an X/Twitter account to run?

Not for posts, profiles, media, polls, or quoted posts — that surface is keyless. You do need an X session (auth_token + ct0 cookies) if collectPostReplies is on (the default) or if includeReplies is on, because both surfaces return an empty response to a logged-out caller.

How many posts and replies can Twitter X Scraper return per run?

Up to maxTweets posts per profile (1–1000, default 10; single post URLs return exactly one row regardless), and up to maxRepliesPerPost replies per post (1–200, default 20). A single post's conversation is also capped internally at 40 pagination pages per ranking mode, and profile timelines at 60 pages per target.

Does Twitter X Scraper return every reply on a post?

No, and this is the Actor's most important limitation: X's TweetDetail surface plateaus at a partial set of top-level replies regardless of how many the post declares. On a post X reported as having 337 replies, pagination to natural termination returned 182–228 unique replies depending on the ranking chosen; on a larger post, unioning all three rankings still recovered less than the platform's declared count. replies_declared and replies_returned both ship on every row precisely so this gap is visible rather than hidden.

Which reply ranking should I use?

Likes (the default) is engagement-weighted and paginates deepest in most cases; Relevance is X's own scoring and tends to terminate soonest; Recency is strictly newest-first. No single ranking recovers the most replies on every post, so deepReplySweep sweeps all three and merges by reply ID when maximum recovery matters more than request count — at up to three times the requests of a single-ranking run.

How does minReplierFollowers work?

Set it above 0 and a reply is kept only if X published a follower count for its author and that count meets the floor. Replies from accounts with no published follower count are not assumed to fail the floor and are not assumed to pass it either — they are excluded from post_replies but counted separately in replies_dropped_by_filter, so the filtering is visible rather than silent.

Can I scrape a single post's conversation without collecting a whole profile?

Yes — paste the post's URL directly into startUrls (e.g. https://x.com/NASA/status/2082511887757881648). A post URL returns exactly one row for that post, with its replies attached, and ignores maxTweets entirely.

How does Twitter X Scraper handle X's rate limits and anti-bot measures?

It activates a fresh guest token for keyless requests and rotates to a new one on a 429 (keyless rate-limit buckets are per guest token, not per IP); on an authenticated session, a 429 with a short reset window is waited out, otherwise the run reports reply_fetch_status: "rate_limited" on the affected post rather than retrying indefinitely. GraphQL operation IDs are discovered live from X's own web client bundle rather than hardcoded, and requests are sent with browser TLS impersonation (Chrome 131) rather than a bare HTTP client.

Can I use Twitter X Scraper without managing proxies?

Yes for the public timeline surfaces — X was measured serving that data to a bare Apify datacenter exit, so proxyConfiguration is off by default. You do still need to supply your own X session cookies (auth_token + ct0) for reply collection; a proxy does not raise or bypass the TweetDetail rate limit, which is tied to the session, not the IP.

Does Twitter X Scraper work with Claude, ChatGPT, and AI agent frameworks?

Yes, as an HTTP-callable Actor rather than a dedicated MCP server: any agent framework that can call the Apify API or apify_client can trigger a run with a query, poll for completion, and read back structured JSON — including the full reply conversation — to ground its answer before responding.

How do I monitor a conversation for new replies over time?

Schedule the same target list with Apify Console Schedules, then diff each run's post_replies reply-ID set and followers values on new repliers against the previous run's dataset, and alert on the delta that matters to you — a new high-follower reply, or a jump in replies_dropped_by_filter.

💬 Your feedback

Found a bug, or a field that isn't behaving the way this README says it should? Open an issue from the Issues tab on this Actor's Apify Store page, or reach the Scrapier team through the contact options on that same page — bug reports with a reproducing startUrls value get fixed fastest.