X Twitter Posts Search & Post Replies

X (Twitter) Posts Scraper — Search Posts, Replies & Threads

Search X (Twitter) by profile, handle, keyword, hashtag URL, search URL, or direct post link, and get back typed JSON for every matching post plus the full reply thread underneath it — replier text, author profile, engagement counts and views. Every response is normalized JSON, not HTML — no selectors, no parsing step, ready to drop into a pipeline or an LLM context window. Two entity types come out of one run: posts (with 38 top-level fields each) and replies (with 22 top-level fields each, including nested replies-to-replies). After reading this you'll know exactly which input drives which output field, what X blocks without a login, and what a single row_result charge actually includes.

🐦 What is X (Twitter) Posts Scraper?

X (Twitter) Posts Scraper is an Apify Actor that combines two things most X scrapers split across separate products: post search and reply-thread collection. Point it at a profile, an @handle, a keyword or hashtag, an x.com/search URL, or a single /status/… link, and it returns the matching posts plus — when enabled — every reply underneath each one, including replies that X hides behind a "Show more replies" cursor. Profile timelines work without any login; keyword search and reply threads require your own X session cookies, because X answers both of those surfaces with HTTP 404 for logged-out callers.

• Scrape posts from profiles, keyword/search syntax, hashtag URLs, and single post links
• Scrape the reply thread under every post, including hidden and nested (depth-2) replies
• Filter replies by keyword and get the matched terms back per reply and per post
• Export as JSON, CSV, Excel, or any format the Apify dataset supports
• No proxy management required — Apify Proxy is wired in and off by default unless you enable it

How a target string turns into a query

Each entry in startUrls is classified automatically, so a single list can mix profiles, keywords and post links freely:

You enter	Detected as	Example
Profile URL	Profile (user)	https://x.com/elonmusk
@handle or bare handle	Profile (user)	@NASA, BarackObama
Multi-word text	Search (search)	climate change, world cup final
search: prefix	Search (search)	search: python from:NASA min_retweets:50
x.com/search?q=… URL	Search (search), query URL-decoded	https://x.com/search?q=nasa&f=live
x.com/hashtag/NAME URL	Search (search) rewritten to #NAME	https://x.com/hashtag/NASA
x.com/<user>/status/<id> URL, or a bare 15+ digit ID	Single post (post) — post + reply thread from one call	https://x.com/NASA/status/1234567890123456789

Reserved x.com paths (search, hashtag, explore, i, home, notifications, messages, settings, compose, intent, share, login) are recognized and routed to a search query instead of being misread as a username — a link like x.com/i/lists/… won't be treated as a nonexistent profile called i.

📊 What data does X (Twitter) Posts Scraper collect?

Every run can return three distinct kinds of structured data, all from the same target list:

Data Type	Key Fields	JSON Field Names
Posts	text, engagement counts, media, hashtags, mentions, retweet flag	postText, postUrl, favouriteCount, repostCount, viewCount, hashtags, isRetweet
Replies	reply text, nesting depth, matched keywords, engagement counts	replyText, replyDepth, isNested, matchedKeywords, favouriteCount, viewCount
Author profiles (embedded in posts & replies)	name, handle, follower count, bio, verification	author.screenName, author.followersCount, author.description, author.isBlueVerified

Need more X (Twitter) data?

If you need trending topics instead of search results, API Empire also publishes Twitter Trends Scraper & Country List Per Trend for per-country trend lists. For media-focused extraction from posts you already have URLs for, there's Twitter X Posts Scraper (Media & Link Extraction). If you only need replies filtered by verification badge and engagement thresholds without running a full search, Twitter X Reply Scraper By Verified & Engagement Filter does that in isolation.

🛠️ Why not build this yourself?

X has no documented public endpoint for keyword search or reply threads — this Actor talks to the same internal GraphQL surface the x.com web client uses (SearchTimeline, UserTweets, TweetDetail), which is undocumented and moves without notice. Three things make a hand-rolled version expensive to keep working:

• Query IDs and the bearer token live inside X's rotating JS bundle. They're embedded in main.<hash>.js and change when X ships a new build. This Actor re-discovers them from the live bundle on each cold start and only falls back to pinned values if discovery fails — one of the actor's previously pinned SearchTimeline query IDs is already retired (X answers it with HTTP 404 / 0 bytes).
• SearchTimeline is POST-only. A GET request — the natural first attempt — gets a silent HTTP 404 with an empty body, which is easy to mistake for "no results" instead of "wrong HTTP verb."
• Reply threads use four different cursor types (Bottom, ShowMoreThreads, ShowMore, ShowMoreThreadsPrompt), and on low-engagement posts the only cursor served is the hidden one — a scraper that only follows the Bottom cursor will report zero replies on posts that actually have them. Promoted replies (ads rendered inline in the thread) also need to be filtered out by their promotedMetadata field or they get counted as organic conversation.
• The tweet payload itself has two co-existing schemas. X serves author data under a modern core.* block on some responses and the pre-2025 legacy.* block on others, and a retweet arrives wrapped as TweetWithVisibilityResults around the original tweet. Reading only one shape silently drops fields depending on which response format X happened to return that call.

Building and maintaining that discovery-and-cursor-and-dual-schema logic is the ongoing cost this Actor absorbs so you don't have to.

🎯 Why do developers and teams scrape X posts and replies?

For AI engineers and agent builders

Reply threads are one of the few public sources of natural, contextual conversational data tied to a known topic. Pull postText plus the full replies[] array for a keyword search, and you have topic-labeled dialogue turns — question, reply, and the matched term in matchedKeywords if you filtered — ready to index into a vector store or use as few-shot examples for a support or moderation agent. Because every field is typed JSON rather than HTML, there's no scraping/parsing layer sitting between the data source and the agent's tool call, which removes one of the most common failure points in agent pipelines that depend on live web data.

For marketers and brand teams

Search your brand or product name with searchType: "latest" and a timeWindowHours window, and you get the reaction as it happens, not just the mention. replyKeywords: ["refund", "broken", "cancel"] surfaces complaint threads directly inside a large conversation, with the matched term attached to each individual reply and rolled up per post in matchedReplyKeywords for fast triage. Because followHiddenReplies defaults to on, low-engagement threads that a naive scraper would report as "no replies" still surface their hidden conversation.

For researchers and analysts

Every reply carries the replier's follower count, bio, verification badge (isBlueVerified), profile location, and account-age signals (statusesCount, friendsCount), all sourced from public profile data returned alongside the reply itself. That is enough to study who engages with a topic and how, not just what was said — useful for discourse analysis, network mapping, or misinformation-spread research scoped strictly to publicly accessible accounts, with replyRankingMode letting you compare X's own "relevance" ranking against a plain recency ordering of the same conversation.

For developers building data products

postStatus and replyFetchStatus are populated on every single row, including targets that returned nothing at all — an "empty" target (surface answered normally, no matches) and a "blocked" or "login_required" target look structurally different in the data, with the cause in errorReason. That makes it safe to build a scheduled monitoring pipeline or a customer-facing data product on top of this Actor without writing special-case handling for silent failures — a downstream job can filter on postStatus the same way it filters on any other field.

How to scrape X (Twitter) posts and replies (step by step)

1. Open the Actor on its Apify Store listing and click Try for free (or Run, if already added to your account).
2. Provide startUrls — at least one profile URL, @handle, keyword, hashtag/search URL, or post link (this is the one required field alongside searchType).
3. Set searchType to "top" or "latest", and turn includeReplies on or off depending on whether you need the conversation or just the posts.
4. If you need keyword search or replies, paste your X auth_token and ct0 cookie values into the Login section, then start the run.
5. Download results as JSON, CSV, or Excel from the run's dataset, or pull them programmatically with the Apify API.

While it runs, the log narrates progress target by target, so you can see exactly what's happening without opening the dataset:

🚀 X (Twitter) Posts Search & Post Replies — starting up
🎯 Targets queued: 2
⚙️  Mode: 🆕 Latest  ·  📦 Max posts/target: 25  ·  ⏱️  Window: last 24h (applied to every target)
💬 Replies: ON  ·  max 50/post  ·  ranking relevance  ·  nested off  ·  hidden-thread cursor on
🔐 X session: ✅ supplied
🌐 Proxy: ✅ Apify Proxy enabled
👤 [1/2] Fetching profile @NASA …
  ✨ [1/25] @NASA  ❤️ 15.3K  🔁 1.9K  💬 412  👁️ 812.0K  🧵 50 replies (ok)  — NASA's Artemis II crew begins final training milestone…
  📦 Target done — 25 posts saved (running total: 25)
🔍 [2/2] Searching: 'climate change' …
  📦 Target done — 25 posts saved (running total: 50)
🎉 Run completed!  Posts saved: 50 ✅  ·  Replies collected: 1180 💬  ·  TweetDetail calls left in budget: 62

What to do when X changes its structure

The Actor is maintained, and the output schema — field names, types, and nesting — stays stable on your end even when X rotates its GraphQL query IDs or JS bundle. If a specific surface does get walled off, the run reports it explicitly through postStatus / replyFetchStatus rather than returning silently empty rows.

⬇️ Input

The full input schema, read from .actor/actor.json. Only startUrls and searchType are required — everything else has a working default.

According to the Actor's own published input description, X allows roughly 187 search calls and roughly 150 reply-thread calls per 15 minutes, per X account — a ceiling shared by every run that authenticates with the same authToken/ct0 pair, including runs started in parallel.

Parameter	Required	Type	Default	Constraints	Description
startUrls	Yes	array (stringList)	—	one entry per target	Profile URLs, @handles, keywords, X search syntax, x.com/search URLs, hashtag URLs, or single /status/… links. Multi-word input is treated as a search; a single word or URL is treated as an account/target.
searchType	Yes	string (enum)	"top"	"top" | "latest"	"top" returns popular/relevant posts; "latest" returns newest first. Only affects keyword/search targets — profile timelines expose no sort parameter, so this setting changes nothing on a profile target.
maxTweets	No	integer	10	min 1, max 10000	Cap on posts collected per target. Every saved post also spends reply-thread calls when includeReplies is on.
timeWindow	No	integer	—	min 1	Keep only posts newer than N days (UTC). Applies to every target type (profiles, searches, and post links). Leave empty to collect up to maxTweets regardless of date.
timeWindowHours	No	integer	—	min 1	Keep only posts newer than N hours (UTC). Applies to every target type. Takes precedence over timeWindow if both are set.
includeReplies	No	boolean	true	—	On: each saved post carries a replies[] array and every reply is also written to a per-run x-replies-<runId> dataset. Off: posts only. Requires X session cookies — without them, each row reports replyFetchStatus: "login_required" rather than an empty array.
maxRepliesPerPost	No	integer	25	min 0, max 1000	Hard cap on replies collected per post. X serves roughly 36 replies per call, so 25 ≈ 1 call, 200 ≈ 6 calls, 400 ≈ 12 calls. X itself stops serving new replies at roughly 350–410 per post no matter how deep you page, so values well above that measured ceiling don't buy more data.
replyRankingMode	No	string (enum)	"relevance"	"relevance" | "recency"	"relevance" returns X's own ranked conversation (what a visitor sees first); "recency" returns newest replies first. The two return genuinely different reply sets on the same post.
followHiddenReplies	No	boolean	true	—	X hides low-ranked replies behind a separate "Show more replies" cursor. On low-engagement posts, that cursor is the only one served — without following it, the post looks like it has zero replies.
includeNestedReplies	No	boolean	false	—	On: after direct replies, the Actor re-queries each reply that itself has replies and pulls its children, marked isNested: true, replyDepth: 2. Costs one extra call per expanded parent, capped at 8 parents per post.
replyKeywords	No	array (stringList)	—	case-insensitive substrings	Keeps only replies whose text contains one of these words (X's auto-inserted @mention prefixes are stripped before matching). Matching replies carry the hits in matchedKeywords[]; the post row rolls them up in matchedReplyKeywords[]. Leave empty to keep every reply.
authToken	No	string (secret)	—	40-char hex cookie value	The auth_token cookie from a logged-in x.com session. Stored encrypted by Apify. Combined with ct0, unlocks keyword search and reply threads. Rate limits are shared across every run using the same X account.
ct0	No	string (secret)	—	CSRF cookie value	The ct0 cookie from the same x.com session, sent as the x-csrf-token header. Must come from the same browser session as authToken.
proxyConfiguration	No	object	{"useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"]}	Apify proxy editor	Off = direct connection. On = Apify Proxy. Both RESIDENTIAL and datacenter proxy groups work against X — X session cookies tolerate IP changes well — but RESIDENTIAL is the safer default.

Both authToken and ct0 are marked isSecret in the input schema — they're encrypted at rest and never appear in the run log or the dataset.

Example input

{
  "startUrls": [
    "@NASA",
    "climate change",
    "https://x.com/search?q=nasa&f=live",
    "https://x.com/hashtag/NASA",
    "https://x.com/NASA/status/1234567890123456789"
  ],
  "maxTweets": 25,
  "searchType": "latest",
  "timeWindowHours": 24,
  "includeReplies": true,
  "maxRepliesPerPost": 50,
  "replyRankingMode": "relevance",
  "followHiddenReplies": true,
  "includeNestedReplies": false,
  "replyKeywords": ["launch", "delay"],
  "authToken": "<your 40-character auth_token cookie value>",
  "ct0": "<your ct0 cookie value>",
  "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] }
}

Common pitfall: leaving authToken / ct0 blank and expecting keyword search to work. Profile timelines run fine without cookies, but X answers SearchTimeline and TweetDetail (search and reply threads) with HTTP 404 for every logged-out request — the run doesn't error out, it reports postStatus: "login_required" per target instead.

⬆️ Output

Every row is typed, normalized JSON with a stable, flat set of top-level keys — the dataset's default table view shows all 38 of them, nothing is held back in a "raw" field. Nested structures (author, media[], replies[]) carry their own field set, documented below. Download as JSON, CSV, or Excel directly from the Apify Console, or pull it through the API.

Which rows are billed

This Actor charges the row_result pay-per-event only on rows with rowType: "post" — one charge per saved post, regardless of how many replies are attached to it. rowType: "accounting" rows (a target that returned nothing, was blocked, rate-limited, or needed a login it didn't have) are written with a plain dataset push and carry no charge at all. To count or export only the billable rows, filter on rowType == "post"; to see the free diagnostic rows, filter on rowType == "accounting". Replies mirrored into the secondary x-replies-<runId> dataset are not billed a second time either — they're the same objects already included, uncharged, inside their parent post's replies[] array.

Scraped post (rowType: "post")

{
  "postText": "NASA's Artemis II crew begins final training milestone ahead of launch.",
  "postUrl": "https://x.com/NASA/status/1234567890123456789",
  "timestamp": 1758123456000,
  "conversationId": "1234567890123456789",
  "postId": "1234567890123456789",
  "media": [
    { "mediaUrlHttps": "https://pbs.twimg.com/media/example.jpg", "type": "photo", "id": "1234567890123456780" }
  ],
  "author": {
    "name": "NASA",
    "screenName": "NASA",
    "followersCount": 96200000,
    "favouritesCount": 4200,
    "friendsCount": 320,
    "description": "Explore the universe and discover our home planet."
  },
  "replyCount": 412,
  "quoteCount": 88,
  "repostCount": 1900,
  "favouriteCount": 15300,
  "postedAt": "2025-09-17T14:30:56Z",
  "viewCount": 812000,
  "bookmarkCount": 2100,
  "lang": "en",
  "hashtags": ["Artemis"],
  "mentions": [],
  "links": ["https://www.nasa.gov/artemis"],
  "source": "Twitter Web App",
  "isRetweet": false,
  "retweetedBy": null,
  "inReplyToPostId": null,
  "inReplyToScreenName": null,
  "rowType": "post",
  "targetType": "user",
  "targetInput": "@NASA",
  "postStatus": "ok",
  "repliesClaimed": 412,
  "repliesReturned": 50,
  "replyRecoveryRatio": 0.1214,
  "replyFetchStatus": "ok",
  "replyPagesFetched": 3,
  "repliesTruncated": true,
  "promotedRepliesSkipped": 1,
  "ancestorPostsSkipped": 0,
  "matchedReplyKeywords": ["launch"],
  "replies": [ /* array of reply objects — see below */ ],
  "errorReason": null
}

isRetweet: true rows are saved once using the original tweet's text, media and counts, with retweetedBy set to the reposting account's handle — X's own retweet-collapsing behavior, preserved rather than duplicated.

Targets that fail or return nothing still produce a row (rowType: "accounting") with the same 38 keys — base fields null, postStatus set to one of login_required, rate_limited, blocked, error, or empty, and errorReason holding the human-readable cause. This is also how a target with zero matching posts is recorded, distinct from a target that was blocked.

Accounting row (rowType: "accounting") — a target that needed login

{
  "postText": null,
  "postUrl": null,
  "timestamp": null,
  "conversationId": null,
  "postId": null,
  "media": [],
  "author": {
    "name": null, "screenName": null, "followersCount": null,
    "favouritesCount": null, "friendsCount": null, "description": null
  },
  "replyCount": null,
  "quoteCount": null,
  "repostCount": null,
  "favouriteCount": null,
  "postedAt": null,
  "viewCount": null,
  "bookmarkCount": null,
  "lang": null,
  "hashtags": [],
  "mentions": [],
  "links": [],
  "source": null,
  "isRetweet": null,
  "retweetedBy": null,
  "inReplyToPostId": null,
  "inReplyToScreenName": null,
  "rowType": "accounting",
  "targetType": "search",
  "targetInput": "breaking news",
  "postStatus": "login_required",
  "repliesClaimed": null,
  "repliesReturned": 0,
  "replyRecoveryRatio": null,
  "replyFetchStatus": "not_attempted",
  "replyPagesFetched": 0,
  "repliesTruncated": false,
  "promotedRepliesSkipped": 0,
  "ancestorPostsSkipped": 0,
  "matchedReplyKeywords": [],
  "replies": [],
  "errorReason": "login_required: keyword search needs an X session (auth_token + ct0). No charge was made for this row."
}

Every accounting row shares this exact shape whether the cause was a missing login, a rate limit, an unexpected error, or a target that was simply empty — only postStatus, targetType, targetInput and errorReason change, which makes it possible to filter and count failure reasons with one field instead of parsing free-text log lines.

Reply (embedded in replies[], and mirrored to the x-replies-<runId> dataset)

{
  "replyId": "1234567890123456790",
  "replyUrl": "https://x.com/spacefan22/status/1234567890123456790",
  "replyText": "Incredible milestone — can't wait to see the crew launch.",
  "isLongForm": false,
  "postedAt": "2025-09-17T15:02:11Z",
  "timestamp": 1758124931000,
  "lang": "en",
  "viewCount": 4200,
  "favouriteCount": 88,
  "replyCount": 2,
  "repostCount": 3,
  "quoteCount": 0,
  "bookmarkCount": 1,
  "media": [],
  "inReplyToPostId": "1234567890123456789",
  "inReplyToScreenName": "NASA",
  "conversationId": "1234567890123456789",
  "isNested": false,
  "replyDepth": 1,
  "parentPostId": "1234567890123456789",
  "parentPostUrl": "https://x.com/NASA/status/1234567890123456789",
  "author": {
    "screenName": "spacefan22",
    "name": "Space Fan",
    "userId": "998877665544",
    "followersCount": 1230,
    "friendsCount": 540,
    "statusesCount": 8900,
    "favouritesCount": 12000,
    "isBlueVerified": false,
    "profileImageUrl": "https://pbs.twimg.com/profile_images/example_normal.jpg",
    "description": "Amateur astronomer. Views my own.",
    "location": "Houston, TX"
  },
  "matchedKeywords": ["launch"]
}

Note the reply-level author object carries 11 fields (including userId, statusesCount, isBlueVerified, profileImageUrl, location) versus 6 fields on the post-level author object — the reply payload X serves is simply richer than the timeline payload, and both are passed through as-is rather than trimmed to match.

How does X (Twitter) Posts Scraper compare to other X scrapers?

Feature	X (Twitter) Posts Scraper	Generic alternative
Reply threads in the same run	✅ built in, same charged event as the post	❌ scraper_one/x-posts-search requires a second Actor ("X Post Replies Scraper") to fetch replies (observed on its listing, checked 2026-07-25)
Hidden / low-engagement reply cursor	✅ explicitly followed and documented	Not documented on either competitor listing checked 2026-07-25
Target types accepted	Profile, @handle, keyword, X search syntax, search URL, hashtag URL, single post link — all auto-detected	scraping_solutions' listing exposes a single TypeScraper enum (timeline_users/search/hashtags/replies) chosen per run (checked 2026-07-25)
Empty/blocked-target transparency	✅ every target produces a status row (postStatus, errorReason) even at zero results	Not documented on the competitor listings checked 2026-07-25
Output schema	Flat, typed JSON, documented field-by-field	scraper_one documents an example output but no field-by-field schema (checked 2026-07-25)

If you're building an AI agent or RAG pipeline, the output format row is the decision-maker — parsing HTML or loosely-typed fields inside an agent loop is a reliability failure mode, not a feature.

How many results can you scrape with X (Twitter) Posts Scraper?

maxTweets caps posts per target at 1–10,000 (default 10), and maxRepliesPerPost caps replies per post at 0–1,000 (default 25) — those are the input-schema ceilings, not platform ceilings. In practice, X itself stops serving new replies on a single post at roughly 350–410 regardless of how many pages you fetch, so a maxRepliesPerPost set far above that won't return more data. Pagination is cursor-based throughout: profile timelines page in batches of up to 40, search results in batches of 20, and reply threads in batches of roughly 36 per call. All TweetDetail calls (single-post fetches, reply pages, and nested-reply expansions) draw from one shared budget of 140 calls per run — a deep multi-target run with replies enabled on every post can exhaust that budget before reaching the last target, at which point remaining replies report replyFetchStatus: "budget_exhausted" rather than failing the run.

Internally, each profile or search target actually requests up to maxTweets × 2 raw posts from X before stopping — that headroom absorbs duplicate IDs and posts that fall outside a timeWindow/timeWindowHours cutoff, so the run still lands on exactly maxTweets saved posts per target rather than under-shooting whenever a date filter is active.

Integrate X (Twitter) Posts Scraper and automate your workflow

X (Twitter) Posts Scraper works with any language or tool that can send an HTTP request — it runs on the Apify platform and is reachable through the Apify API and official SDKs.

REST API integration

from apify_client import ApifyClient

client = ApifyClient("<YOUR_API_TOKEN>")
run = client.actor("<YOUR_USERNAME>/x-twitter-posts-search-post-replies").call(
    run_input={
        "startUrls": ["@NASA", "climate change"],
        "searchType": "latest",
        "maxTweets": 25,
        "includeReplies": True,
        "maxRepliesPerPost": 50,
    }
)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["postUrl"], item["repliesReturned"], "replies")

Works in Python, Node.js, Go, Ruby, or cURL — anything that can call the Apify API with your account token.

Automation platforms (n8n, Make, LangChain)

In n8n, the Apify node's "Run Actor and get dataset" operation can call this Actor by name and pass startUrls/searchType as JSON, feeding results straight into a downstream node. In Make, the Apify app's "Run an Actor" module does the same, with the dataset items available to the next module in the scenario. In LangChain, the ApifyWrapper (langchain_community.utilities.ApifyWrapper) can call this Actor and load its dataset directly into a document loader for indexing.

Is it legal to scrape X (Twitter) posts and replies?

Yes — X (Twitter) Posts Scraper only returns data that is already publicly visible on x.com, without logging into or accessing private, protected, or restricted accounts, and without bypassing any account-level access controls. Because post authors and repliers are individuals, and the output includes personal data such as names, handles, bios, locations and follower counts, GDPR and CCPA considerations apply to how you store, retain and use that data — not to the act of reading a public profile itself. Using your own X session cookies to unlock search and replies means you are still bound by X's Terms of Service for your account while the Actor runs. You are responsible for having a lawful basis if you retain or process personal data belonging to EU or California residents. Consult legal counsel for commercial use cases involving bulk personal data.

❓ Frequently asked questions

Does X (Twitter) Posts Scraper work without an X account?

Partially. Profile-timeline targets (@handle or a profile URL) work with no login at all — the Actor falls back to an anonymous X guest token. Keyword search, x.com/search and hashtag URLs, and every reply thread require your own authToken + ct0 cookies, because X answers those two GraphQL surfaces with HTTP 404 for logged-out callers.

How often is the scraped data updated?

Every run fetches live from X at the moment it executes — nothing is cached or served from a stored snapshot. Run it on a schedule if you need repeated, up-to-date pulls.

What happens if a search returns zero matching posts, or a linked post was deleted?

The target still produces one row with rowType: "accounting" and postStatus: "empty" (surface responded normally, nothing matched) or the relevant failure status (login_required, blocked, rate_limited, error) with the reason in errorReason — the run never silently returns nothing with no explanation.

Can I scrape private or protected X accounts?

No. Only publicly accessible profiles, posts and replies are returned — protected accounts and posts visible only to approved followers are out of scope.

Why did my run stop collecting replies partway through a post?

Reply and post-detail calls to X's TweetDetail endpoint share one budget of 140 calls per run. Once it's exhausted, remaining replies on later posts report replyFetchStatus: "budget_exhausted" and repliesTruncated: true instead of failing the whole run — split large jobs across multiple runs if you need deep reply coverage on many posts.

Does X (Twitter) Posts Scraper work for AI agent workflows and LLM pipelines?

Yes. It's callable as an HTTP endpoint by any agent framework through the Apify API — every response is typed JSON with stable field names, so there's no HTML-parsing step between the scraper and your LLM context window or vector store.

How does X (Twitter) Posts Scraper handle X's anti-bot and login requirements?

Profile timelines authenticate with an anonymous guest token issued per run. Search and replies require your own session cookies (auth_token + ct0), sent with the matching x-csrf-token header. Query IDs and the bearer token used for every GraphQL call are re-discovered from X's live JS bundle at the start of each run, with pinned fallbacks if discovery fails, and network/5xx errors are retried up to 3 times with exponential backoff — genuine blocks (401/403/404/429) are not retried, they're reported immediately with the reason.

How does X (Twitter) Posts Scraper compare to other X scrapers?

The main structural difference, per the competitor listings checked 2026-07-25, is that reply threads are built into this Actor and charged as part of the same post row, where scraper_one/x-posts-search requires running a second, separate Actor for replies. See the comparison table above for the full breakdown.

Does X (Twitter) Posts Scraper return data in a format LLMs can use directly?

Yes. Every field is typed, normalized JSON with stable names across posts and replies — pass it straight into an LLM prompt, a RAG index, or an agent tool without a parsing step.

Can I use X (Twitter) Posts Scraper without managing proxies?

Yes. Apify Proxy is wired in through the proxyConfiguration input and defaults to RESIDENTIAL in the prefilled example — you pick a group in the UI, or leave it off entirely for a direct connection; there's no separate proxy service to provision.

What happens when X changes its structure or blocks the scraper?

The Actor is maintained, and the output schema — field names and types — stays the same on your end even after X changes its internal GraphQL layer. There's no numeric turnaround time promised for fixes.

💬 Your feedback

Found a bug, or missing a field you need in a post or reply row? We want to know. Open an issue through the Actor's Apify Store page or reach out via the Issues tab on the Actor's page so it can be tracked and fixed.