X (Twitter) Posts Search

X (Twitter) Posts Search — Text, Likes, Views and Author Data

X (Twitter) Posts Search turns a keyword, hashtag, x.com search URL or @handle into a structured post dataset: full post text including long-form notes, canonical post URL, UTC timestamp, likes, reposts, replies, quotes, views, bookmarks, language, posting client, media URLs and the author's handle, bio and follower counts. One typed JSON row per post, pushed live as it is collected, exportable to JSON, CSV, Excel or XML. Pick Top or Latest, add your targets, press Start.

⚠️ Read the cookie requirement below before your first run. X answers keyword, hashtag and search-URL requests from logged-out clients with an empty HTTP 404, so search targets need your own auth_token and ct0 cookies. Profile and @handle targets work fully logged out. Without cookies, a keyword target is skipped and the run writes one unbilled accounting row explaining why.

What is X (Twitter) Posts Search?

X (Twitter) Posts Search is an Apify Actor that reads X's own GraphQL timeline responses — SearchTimeline for queries, UserTweets for profiles — and flattens each result into a fixed 23-key post record. It accepts keywords, #hashtags, x.com/search?q=… URLs, x.com/hashtag/<tag> URLs, profile URLs and bare handles in the same input list, and writes every post to the dataset the moment it is normalised.

Whether a login is needed depends entirely on the target type. Profile timelines are fetched with a guest token the Actor obtains itself, so no credentials are required. Keyword search is different: X serves logged-out search requests an HTTP 404 with a zero-byte body, so you supply two cookies from your own X session. That is a platform behaviour, not an Actor limitation.

It is built for social listening and PR teams tracking a topic, research teams building post-level datasets, and developers piping X posts into a dashboard, a warehouse or an LLM pipeline.

What X post data is publicly available to scrape?

Everything this Actor returns is what a visitor sees in X's own search results and on a public profile timeline: the post text, its engagement counters, the attached media and the author's public profile numbers. What X does not put in a timeline payload — who individually liked or reposted something, anything from a protected account — is not available at any price.

Data Category	Available without a login	Needs your own logged-in X session
Public profile timeline posts (@handle, profile URL)	✅ Yes — guest token, no cookies	—
Keyword, hashtag and x.com/search results	❌ HTTP 404 with an empty body	✅ auth_token + ct0 cookies
Post text, canonical URL, post and conversation IDs, UTC timestamp	✅ Yes	—
Likes, reposts, replies, quotes, views, bookmarks	✅ Yes	—
Author display name, handle, bio, followers, following, total likes	✅ Yes	—
Media URLs and media type per attachment	✅ Yes	—
Reply threads under a post	❌ Not fetched by this Actor	Covered by Twitter X Reply Scraper
Individual liker and reposter lists	❌ Not in the timeline payload	Not available
Protected accounts, DMs, deleted posts	❌ Never	Not available
Author email or phone number	❌ Never in a post payload	Not available

X (Twitter) Posts Search only returns publicly visible posts — what any visitor sees in search results or on a public timeline. Nothing from a protected account, and nothing behind a follow request.

⚠️ Keyword search needs your own X session cookies

This is the one prerequisite, and it belongs up front rather than buried in the input table.

X's SearchTimeline endpoint answers an authenticated session only. A logged-out POST comes back as HTTP 404 with a zero-byte body — there is no guest tier for search. The Actor detects that exact response and reports it rather than pretending the query matched nothing. Profile lookups, profile timelines and the runtime query-id discovery all still work logged out, and no credential is baked into the Actor.

Supply two cookie values from a browser where you are already signed in to X:

• xAuthToken — your auth_token cookie, a roughly 40-character hex string
• xCt0 — your ct0 cookie, sent onward as the x-csrf-token header

Get both from DevTools → Application → Cookies → https://x.com, or from a cookie-export extension. Both are needed: the Actor only switches into authenticated mode when xAuthToken and xCt0 are non-empty, so one alone does nothing. Both fields are declared isSecret in the input schema, so the values are stored encrypted and masked in the Console and in run history, and the code never writes them to the dataset or the log — the log only states whether cookies were supplied. The Actor also reads X_AUTH_TOKEN and X_CT0 environment variables as a fallback when the input fields are empty.

Neither field is marked required in the schema, deliberately. A run without them still succeeds: profile targets are collected normally, and every keyword, hashtag or search-URL target is skipped with an unbilled accounting row carrying status set to login_required.

One thing to weigh before you paste anything in: auth_token is a login credential for the account it came from, and automation replaying a live session is exactly the pattern platforms challenge. Use a secondary account you can afford to lose rather than your main one, rotate the value when you are done, and expect that X may require re-authentication on that account at any point.

What data can I extract with X (Twitter) Posts Search?

Every row carries the same 23 keys in the same order: what the post says, when it was posted, how it performed, who wrote it, what was attached, and which target produced it.

Field Name	Description
postText	Full post text — long-form note text when X sends one, otherwise the standard full text
postUrl	Canonical post URL, built as https://x.com/<screenName>/status/<postId>; null if either part is missing
timestamp	Post creation time as Unix milliseconds; null if X's date string cannot be parsed
conversationId	ID of the thread the post belongs to — the same value as postId on a thread's first post
postId	Numeric post ID as a string
media	Array of attachment objects: mediaUrlHttps, type, id; [] when the post has no media
author	Author object: name, screenName, followersCount, favouritesCount, friendsCount, description
replyCount	Replies on the post
quoteCount	Quote posts
repostCount	Reposts (X's retweet_count)
favouriteCount	Likes (X's favorite_count)
postedAt	The same instant as timestamp, rendered as a UTC ISO string YYYY-MM-DDTHH:MM:SS.000Z
lang	X's own detected language code for the post, e.g. en, es, und
viewCount	View count as an integer; null when X omits it for that post
bookmarkCount	Bookmark count
source	Posting client with HTML tags stripped, e.g. Twitter Web App; null when empty
isRetweet	true when the collected entry was a repost
retweetedBy	Handle of the account that reposted it, on repost rows only; null otherwise
rowType	post on real post rows, run_status on unbilled accounting rows — the one reliable filter
target	The normalised target that produced the row: the search query, or the handle without @
targetType	search or profile
status	ok on post rows; the failure or empty reason on run_status rows
errorReason	null on post rows; the same string as status on run_status rows

Post, timestamp and media fields

postText reads X's note_tweet result first, so long-form posts arrive complete instead of truncated at the classic character limit. postId, conversationId and postUrl are the join keys: conversationId groups a thread, and postUrl is safe to hand to a human or to another Actor.

Time is given twice on purpose. timestamp is Unix milliseconds for sorting and diffing; postedAt is the same moment as a UTC ISO string, so the value does not shift with your container's time zone.

media is an array of three-key objects. mediaUrlHttps is X's own media_url_https value — for a video entry that is the thumbnail image, not the video file — type is X's label for the attachment, and id is the media ID. Video variant URLs, playback duration, hashtag and mention arrays, verified badges and author profile images are not in this output; the Actor emits what X already ships in the timeline payload and makes no extra call per post.

Reposts are handled explicitly. The row's post data and author describe the original post, while isRetweet is true and retweetedBy names the account whose timeline or search hit surfaced it. Two naming details worth knowing before you map fields: likes are favouriteCount with the British spelling, and the author's own total likes given are author.favouritesCount — different fields, similar names. And while the count key is repostCount, the repost flags kept X's original wording as isRetweet / retweetedBy. Every output key is camelCase, including the ones read from snake_case X fields.

Author and engagement fields

author is a nested object with six keys, always present. name and screenName identify the account, description is the profile bio, and followersCount, friendsCount (accounts followed) and favouritesCount (likes given) are the profile-level numbers X includes alongside the post. Any of them can be null when X omits the user block.

replyCount, quoteCount, repostCount, favouriteCount, viewCount and bookmarkCount are the six engagement integers. viewCount depends on X returning the view block for that post and comes back null when it does not — plan for that rather than assuming zero.

🤖 Add-on: Need additional X (Twitter) data?

Search is one entry point into X, and three sibling Scraper-Engine Actors cover the rest. Pick by what you have and what you are willing to supply:

Actor	What it returns	X session cookies
X (Twitter) Posts Search (this one)	Posts from keywords, hashtags, search URLs and profiles, with Top/Latest and a time window	✅ Needed for keyword, hashtag and search-URL targets
Twitter X Posts Scraper	Posts from profile targets — startUrls, sortOrder, maxTweets	❌ No auth parameters in its schema at all
Twitter X Reply Scraper	Replies under posts — startUrls, maxSearchResults, maxReplies	✅ Takes authToken + ct0
Twitter Trends Scraper	Trending topics with board rank, by country and hour	❌ Its listing states no login, cookie or API key is required

If all you need is a named account's recent posts, Twitter X Posts Scraper does that with no credential at all. Choose this Actor when the job starts with a query rather than a handle — that is what searchType and the time window exist for, and it is why search here costs a cookie.

How does X (Twitter) Posts Search differ from the official X API?

X publishes an official API (v2) with documented search endpoints, and if your use case fits a paid tier, use it — it is the supported route with terms you can rely on. This Actor exists for the case where the paid tier, the approval process or the tier-dependent access to recent search does not fit the job.

Feature	Official X API v2	X (Twitter) Posts Search
Credential you provide	X developer account with a paid tier and app keys	Nothing for profile targets; your own auth_token + ct0 for search
Access to recent-search endpoints	Tier-dependent — verify against X's current published API documentation	Search runs through the same web endpoint your browser uses
Setup before the first result	Developer account, app, project, plan selection	Paste a target, press Start
Input flexibility	Query syntax per the API reference, per endpoint	Keywords, #hashtags, x.com/search URLs, profile URLs and handles mixed in one list
Output shape	X's documented API response objects	23 flat, camelCase keys, identical on every row
Failed or empty targets	Handled by your own code	Unbilled run_status row naming the reason
Rate limiting	Plan-based quotas per X's documentation	Whatever the quota is on the account whose cookies you supply, read live from X's response headers

No price, quota or rate-limit figure from X's official plans is reproduced here — check X's current published API documentation, because those terms change. Use the official API for contractual guarantees and production workloads on an approved tier. Use this Actor for ad-hoc research, monitoring and one-off datasets where standing up a developer project is more work than the question is worth.

How to use X (Twitter) Posts Search

The Actor runs on Apify. Start it from the Apify Console or call it through the Apify API.

1. Open X (Twitter) Posts Search on Apify and click Try for free
2. Add your targets to Targets to scrape (startUrls) — one row per keyword, hashtag, search URL, profile URL or handle. This is one of the two schema-required inputs
3. Pick Search mode (searchType) — top or latest. This is the other required input, and it defaults to top
4. Paste your auth_token and ct0 cookies if any of your targets is a keyword, hashtag or search URL — required in practice, see the cookie section above
5. Set Max posts per target (maxTweets) — per target, not per run
6. Optionally set last N hours (timeWindowHours) or last N days (timeWindow), and turn on Apify Proxy if X blocks the direct connection
7. Click Start, watch posts stream into the Output tab live, then export as JSON, CSV, Excel or XML

Both startUrls and searchType are declared required in the schema, so a run that omits either is rejected by Apify's input validation before the Actor starts — no run, no rows, no charge. If startUrls is present but resolves to nothing usable (an empty list, or only entries the Actor skips), the run logs No targets provided and finishes with zero dataset rows and no accounting row. A caller that reaches the code with a missing or unrecognised searchType falls back to top.

The log prints one line per saved post with the author, likes, reposts, replies, views and a text snippet, plus the number of X search calls left in the current window, so you can see whether a query is worth a bigger run before committing to one.

How to scale to bulk post extraction

startUrls is a list, and every entry is processed in turn into the same dataset. Mix types freely — @NASA, climate change, #Artemis, https://x.com/search?q=james%20webb&f=live and https://x.com/elonmusk in one run is a normal input. Entries may be plain strings or objects; an object is read from its url, query or value key, which makes it easy to pipe another Actor's output straight in.

maxTweets applies per target, so ten targets at maxTweets: 100 is a target of 1,000 rows, not 100. Deduplication is run-wide: the Actor tracks every postId it has already saved, so a post matched by two different queries is written once, and the log reports how many were dropped as already saved. A target that fails or matches nothing does not stop the run — the loop moves to the next one.

What can you do with X post data?

• 📡 A social listening analyst tracking a product launch runs the same keyword daily with searchType: "latest" and timeWindowHours: 24, then ranks new rows by favouriteCount and viewCount to see which mentions actually travelled.
• 📰 A newsroom researcher verifying a claim pulls a hashtag plus the accounts involved in one run and uses postedAt, conversationId and postUrl to reconstruct the order in which a story spread.
• 📊 A brand manager benchmarking share of voice runs several competitor keywords in one pass and compares median repostCount and replyCount per target to see which message is being carried rather than just seen.
• 🌍 A market researcher studying a region filters rows on lang and reads author.followersCount alongside favouriteCount to separate genuine local reach from one large account's amplification.
• 🤖 An AI engineer building a monitoring agent indexes postText, author.screenName and postedAt into a vector store and keeps the six engagement counters plus lang as metadata filters, so the agent answers "what is being said about this topic today" against live posts rather than a stale export.

Every one of these is callable from an agent framework over the Apify API, since the Actor is a standard HTTP-triggered run.

How does X (Twitter) Posts Search handle rate limits and blocking?

No CAPTCHA solving is involved and no browser is launched. The Actor speaks HTTP directly to X's GraphQL endpoints with a desktop Chrome header set — POST for SearchTimeline, GET for the profile operations — which is what those endpoints expect.

Two things are discovered at runtime instead of being hardcoded, because both rot. The public web bearer token and the GraphQL query IDs are harvested from x.com's live JavaScript bundle at the start of every run; pinned fallbacks exist for the two profile operations, deliberately not for search. If a search call comes back 404 with a zero-byte body, the Actor re-harvests the query ID once and retries with the new one before giving up. If the harvest yields no search query ID at all, affected targets end with status set to qid_unavailable rather than a silent zero.

Quota handling is read from X, not guessed. Every response's x-rate-limit-limit, x-rate-limit-remaining and x-rate-limit-reset headers are recorded for two separate buckets — the POST search bucket and the GET bucket the profile operations use — and the remaining count is logged as the run progresses. When X reports two or fewer calls left, the Actor stops that target early instead of burning your last calls. X sets these limits per account; the input schema records a measured figure of roughly 187 search calls per 15-minute window on the search bucket, but treat that as an observation from one account's headers rather than a promise, and note that failed calls consume quota too and concurrent runs share it. Because X returns 20 posts per search call regardless of the count requested, a 100-post target costs at least five calls.

Failures are typed and never fail the run. Network errors and HTTP 5xx are retried up to three times with exponential backoff; HTTP 429 stops the target immediately as rate_limited rather than retrying into a deeper block; 401 and 403 become login_required; an unexpected redirect, a non-JSON body or a missing response container each get their own reason. Proxy use is optional — the Actor holds one Apify Proxy session URL for the whole run when you enable it, and if proxy setup fails it logs a warning and continues on a direct connection rather than downgrading silently.

⬇️ Input

Eight parameters, two of them required. Everything below matches the Actor's input schema exactly.

Parameter	Required	Type	Description	Example Value
startUrls	Yes	array	Targets, one per row: keyword, #hashtag, x.com/search?q=… URL, x.com/hashtag/<tag> URL, profile URL, @handle, bare handle, or an explicit search: <query> string. Strings, or objects with a url / query / value key. Prefilled with ["@NASA", "climate change"].	["@NASA", "climate change", "#Artemis"]
maxTweets	No	integer	Maximum posts collected per target. Minimum 1, maximum 10000, default 10. Clamped into range at runtime.	100
searchType	Yes	string	top (X's popularity and relevance ranking) or latest (newest first). Default "top".	"latest"
timeWindow	No	integer	Keep only posts newer than N days, UTC. Minimum 1, no default — leave empty for no date cut-off. Enforced on both sort modes.	7
timeWindowHours	No	integer	Keep only posts newer than N hours, UTC. Minimum 1, no default. Takes precedence over timeWindow when both are set.	24
proxyConfiguration	No	object	Apify Proxy settings. Prefilled with {"useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"]}. Leave it off for a direct connection.	{"useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"]}
xAuthToken	No	string	Your X auth_token cookie. Declared isSecret, so it is stored encrypted and masked. Not schema-required, but required in practice for every keyword, hashtag and search-URL target. Falls back to the X_AUTH_TOKEN environment variable.	"a1b2c3d4e5f60718293a4b5c6d7e8f9012345678"
xCt0	No	string	Your X ct0 cookie, sent as the x-csrf-token header. Declared isSecret. Must be supplied together with xAuthToken — one alone does nothing. Falls back to the X_CT0 environment variable.	"7f3c9ab1e5d24680bc13fe57a90d2c48"

Five honest notes on how these behave at runtime:

• searchType means something different on profile targets. X's profile timeline API has no server-side sort. For a @handle or profile URL the Actor collects up to twice your maxTweets from the account's recent posts and then keeps either the most-liked (top) or the newest (latest) of what it collected. That ranking is this Actor's, over a bounded window — not X's ranking of the whole account. On search targets searchType maps straight to X's own Top / Latest product.
• The time window is a post-fetch filter, not part of the query. No since: or until: operator is added. Posts are fetched and then dropped if they are older than the cut-off, which means filtered-out posts still consume pages and X quota, and the scanned pool is capped at roughly twice maxTweets. A narrow window on a busy query can therefore return fewer rows than maxTweets even when older matches exist. The log reports how many rows were dropped for being outside the window.
• timeWindowHours wins. If both time fields are set, hours are used and timeWindow is ignored entirely. Both cut-offs are computed from the run's own clock in UTC.
• Some entries are skipped with no row at all. Single-post links containing /status/<id>, X's own system routes (/explore/…, /i/…, /home, /messages, /notifications and similar) and empty rows are logged and dropped — no post row, no accounting row, no charge. A non-x.com URL is not skipped: it is treated as a search for that URL string, which is rarely what you want.
• What counts as a search versus a profile. A multi-word entry, a #hashtag or anything containing a space is a search; a single word matching X's handle pattern, an @handle or an x.com/<name> URL is that account's timeline. Prefix an entry with search: to force search interpretation, for example search: python from:NASA.

Example input

{
  "startUrls": [
    "@NASA",
    "james webb telescope",
    "#Artemis",
    "https://x.com/search?q=mars%20sample%20return&f=live",
    "https://x.com/elonmusk"
  ],
  "maxTweets": 100,
  "searchType": "latest",
  "timeWindowHours": 24,
  "timeWindow": 7,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  },
  "xAuthToken": "a1b2c3d4e5f60718293a4b5c6d7e8f9012345678",
  "xCt0": "7f3c9ab1e5d24680bc13fe57a90d2c4891bd6e07f2a3c5d81904e6b7a2c3d4f56"
}

With both time fields present as above, the run uses the last 24 hours and ignores the 7-day value.

⬆️ Output

Every row has the same 23 keys in the same order, run after run — no key is ever dropped, and unavailable values arrive as null (or [] for media) rather than going missing. Rows are pushed live as each post is normalised, so a run you stop early still leaves you everything collected up to that point. Export as JSON, CSV, Excel or XML, or read the dataset through the Apify API.

Billing is per row: each post row is pushed with the row_result charged event, one event per post. Accounting rows are pushed with no charged event at all — they are free. Because every key is always present, status is not the marker here; rowType is:

posts = [row for row in items if row["rowType"] == "post"]
diagnostics = [row for row in items if row["rowType"] == "run_status"]

An accounting row is written for any target that ends with zero saved posts, and its status names the reason. empty means X answered normally and nothing matched. login_required means either the target was a search and no cookies were supplied, or X answered 401/403 or a zero-byte 404. The rest come straight from the failure that stopped the target: rate_limited, not_found, unavailable (a handle that could not be resolved — missing, suspended or renamed), redirected, server_error, http_error, network_error, unparseable, api_error, container_missing, qid_unavailable, or the catch-all error. errorReason repeats the same string; the longer detail — the HTTP status, the body size, the resolved URL — goes to the run log, not the row.

Two limits of that mechanism, stated plainly. A target that saved at least one post gets no accounting row, so a run cut short mid-target by a 429 or by the stall guard shows the reason only in the log. And targets skipped at parse time produce no row of any kind.

Example output

A post row from a search target:

{
  "postText": "Webb just resolved the inner disk of Beta Pictoris in the mid-infrared. Three separate dust belts, and the innermost one is not where the models put it. Full data release and images in the thread below.",
  "postUrl": "https://x.com/NASAWebb/status/1948273615902847109",
  "timestamp": 1784989362000,
  "conversationId": "1948273615902847109",
  "postId": "1948273615902847109",
  "media": [
    {
      "mediaUrlHttps": "https://pbs.twimg.com/media/Gx7QpL2XwAAf3Rk.jpg",
      "type": "photo",
      "id": "1948273544012931072"
    }
  ],
  "author": {
    "name": "NASA Webb Telescope",
    "screenName": "NASAWebb",
    "followersCount": 2184903,
    "favouritesCount": 4127,
    "friendsCount": 118,
    "description": "The largest, most powerful space telescope ever built."
  },
  "replyCount": 412,
  "quoteCount": 296,
  "repostCount": 3841,
  "favouriteCount": 21507,
  "postedAt": "2026-07-25T08:22:42.000Z",
  "lang": "en",
  "viewCount": 1284617,
  "bookmarkCount": 1902,
  "source": "Twitter Web App",
  "isRetweet": false,
  "retweetedBy": null,
  "rowType": "post",
  "target": "james webb telescope",
  "targetType": "search",
  "status": "ok",
  "errorReason": null
}

An unbilled accounting row, written here for a keyword target on a run with no cookies. It carries the same 23 keys with empty defaults:

{
  "postText": null,
  "postUrl": null,
  "timestamp": null,
  "conversationId": null,
  "postId": null,
  "media": [],
  "author": {
    "name": null,
    "screenName": null,
    "followersCount": null,
    "favouritesCount": null,
    "friendsCount": null,
    "description": null
  },
  "replyCount": null,
  "quoteCount": null,
  "repostCount": null,
  "favouriteCount": null,
  "postedAt": null,
  "lang": null,
  "viewCount": null,
  "bookmarkCount": null,
  "source": null,
  "isRetweet": false,
  "retweetedBy": null,
  "rowType": "run_status",
  "target": "james webb telescope",
  "targetType": "search",
  "status": "login_required",
  "errorReason": "login_required"
}

How does it work?

Each run starts by loading x.com's own search page and reading its JavaScript bundles to harvest the public web bearer token and the current GraphQL query IDs for SearchTimeline, UserByScreenName and UserTweets. Those IDs change with X's build, which is why they are discovered per run rather than pinned.

Then each target is routed. A search target goes to SearchTimeline as a POST with product set to Top or Latest, and the Actor walks the bottom cursor page by page, deduplicating within the query and stopping when your limit is met, when the cursor stops advancing, after three consecutive pages arrive with no new posts, or when X's remaining-call header drops to two. A profile target resolves the handle to a numeric ID with UserByScreenName, then pages UserTweets, collecting standalone posts and self-thread items.

Every entry is normalised into the same 23 keys: reposts are unwrapped to the original post and flagged, long-form note text is preferred over truncated text, X's date string becomes both Unix milliseconds and a UTC ISO string, and the posting client is stripped of its HTML anchor. Only publicly visible posts are collected. Because the Actor reads X's structured GraphQL payload rather than rendered HTML, an X front-end redesign does not change your field names.

Integrations

X (Twitter) Posts Search is an Apify Actor, so it works with anything that can call the Apify API or consume a dataset.

Calling X (Twitter) Posts Search from Python

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run = client.actor("<YOUR_USERNAME>/x-twitter-posts-search").call(run_input={
    "startUrls": ["james webb telescope", "@NASAWebb", "#Artemis"],
    "searchType": "latest",
    "maxTweets": 100,
    "timeWindowHours": 24,
    "xAuthToken": "<YOUR_X_AUTH_TOKEN_COOKIE>",
    "xCt0": "<YOUR_X_CT0_COOKIE>",
})

for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    if row["rowType"] != "post":
        print("skipped:", row["target"], row["status"])
        continue
    print(row["postedAt"], row["author"]["screenName"], row["favouriteCount"], row["postUrl"])

Works in Go, Ruby, Node.js, cURL — any language that can make an HTTP request. Branch on rowType to keep accounting rows out of your post pipeline.

Scheduling and webhooks

Because session cookies expire, keyword monitoring works best on a schedule you can see failing. Use an Apify schedule to run the same startUrls with searchType: "latest" and a timeWindowHours matching the interval, and attach an Actor webhook on run success to POST the dataset to your endpoint. A dataset whose only rows carry status set to login_required is your signal to refresh the two cookie values — worth alerting on explicitly.

No-code tools (n8n, Make, LangChain)

In n8n, use the Apify node — or an HTTP Request node pointed at the Apify run endpoint with your token — and pass the same JSON input shown above; an IF node on rowType splits real posts from accounting rows. In Make, the Apify module supports run-and-wait, so a daily keyword sweep can feed a Google Sheets or Airtable step directly. In LangChain, wrap the run endpoint as a tool and hand the typed rows to the agent as-is; no parsing step is needed.

Is it legal to scrape X posts?

Scraping publicly visible posts is broadly treated as permissible where the content is accessible without circumventing an access restriction, and this Actor returns only public posts — nothing from a protected account, nothing from DMs. But this output is personal data, and that changes your obligations rather than removing them.

The fields that carry personal data are specific and worth naming: author.name, author.screenName and author.description identify and describe a real person, author.followersCount, author.friendsCount and author.favouritesCount are attributes of that person's account, postText is content they authored and can itself contain other people's names and details, retweetedBy names a second identifiable account, postId, conversationId and postUrl link straight back to the individual, and media[].mediaUrlHttps can point at images of identifiable people. Under GDPR and the UK GDPR you need a lawful basis before you store or process any of it. Legitimate interests is the usual route for research, monitoring and journalism, and it is not a formality: document the balancing test — your purpose, why the processing is necessary, the effect on the individual, and the safeguards you applied — before the first bulk run, not after. CCPA/CPRA treats the same fields as personal information about California residents.

Data minimisation is the practical control, and this output makes it easy. Drop author.description first — it is a free-text profile bio and the least likely field to be necessary — then author.name and the three author counters, then postText itself if you only need engagement patterns. What is left, the six counters plus timestamp, lang, target and postId, supports most trend analysis with far less exposure. Set a retention period and delete on it, keep a record of what you collected and why so you can answer a transparency request, and have a workable process for access, objection and deletion requests — including deleting rows for a post that has since been removed from X.

Separately, keyword search here uses your own X session cookies, which is governed by X's Terms of Service — read them and decide for yourself. Consult legal counsel if your use case involves bulk storage of personal data, profiling of identified individuals, or moving these rows outside your jurisdiction.

❓ Frequently asked questions

What X post fields does X (Twitter) Posts Search return?

The five most used are postText, postUrl, favouriteCount, viewCount and author. Every row carries the same 23 keys — including postedAt, lang, bookmarkCount, source, isRetweet and the rowType / target / status accounting trio. See the data fields table above for all of them, plus the nested media and author shapes.

Does X (Twitter) Posts Search require an X account or login?

It depends on the target. Profile URLs and @handles need nothing — the Actor obtains a guest token itself. Keyword, hashtag and search-URL targets need your own X session, supplied as xAuthToken (auth_token) and xCt0 (ct0), because X answers logged-out search requests with an empty HTTP 404. Both cookie fields are declared isSecret, so they are stored encrypted and masked in the Console and run history, and the Actor never writes them to the dataset or the log. Neither is schema-required: without them, profile targets still run and search targets get an unbilled login_required row.

How many posts can I extract per target?

maxTweets accepts 1 to 10000 and applies per target, so ten targets at 100 is a 1,000-row run. The real ceilings come from X, not the schema: X returns 20 posts per search call, your account's search quota is finite per 15-minute window, and a query ends when X's cursor stops producing new posts — the Actor gives up after three consecutive pages with nothing new. For volume, add more targets rather than pushing maxTweets very high on one.

What happens if a search returns zero results?

The run still succeeds, and the empty target is visible in your dataset instead of being an absence you have to notice. The Actor writes one unbilled accounting row with rowType set to run_status and status set to empty, meaning X answered normally and the query matched nothing — usually a too-narrow keyword or too tight a time window. If the target failed rather than matched nothing, the same row carries the real reason instead: login_required, rate_limited, unavailable, network_error, container_missing, qid_unavailable and so on. In code, filter with row["rowType"] == "post" and branch on row["status"] for the rest. If every target came back empty, the final log line says so explicitly and points at those rows.

Can I search multiple keywords and profiles at once?

Yes. startUrls is a list and each entry is processed in turn into the same dataset, with keywords, hashtags, search URLs, profile URLs and handles freely mixed. Deduplication is run-wide by postId, so a post that two queries both match is saved once, and the log reports how many duplicates were dropped. target and targetType on every row tell you which entry produced it.

Does the time window filter on X's side or after collection?

After collection. No since: or until: operator is added to the query — posts are fetched and then discarded if they are older than the cut-off, which means the discarded ones still cost pages and X quota, and the pool scanned per target is bounded at roughly twice maxTweets. Expect fewer rows than maxTweets on a narrow window over a busy query. timeWindowHours takes precedence when both time fields are set, and both cut-offs are enforced on top and latest alike.

Does X (Twitter) Posts Search work with Claude, ChatGPT and other AI agent tools?

Yes. It is callable as a standard HTTP endpoint through the Apify API, so LangChain, CrewAI, n8n or a hand-written tool definition can invoke it and receive typed JSON with no parsing step. Give the agent the same run input shown above and have it skip rows where rowType is not post.

How does X (Twitter) Posts Search compare to other X post search scrapers?

Checked on the Apify Store on 25 July 2026: scraper_one/x-posts-search is the most used of the alternatives and returns the same 11 core keys this Actor returns — postText, postUrl, timestamp, conversationId, postId, media, author, replyCount, quoteCount, repostCount, favouriteCount. Its README documents a single query string plus resultsCount, timeWindow, timeWindowHours and searchType, states that its time windows work only with the Latest search type, and notes that free users can make only a few requests per day; lang, viewCount, bookmarkCount, source, repost flags, profile targets and cookie authentication are not documented on that listing. scrapebase/x-twitter-posts-search documents a startUrls / sortOrder / maxTweets / maxComments input and a snake_case output shape (id, url, user_posted, likes, views as a string, tagged_users, quoted_post), and no cookie requirement is documented there either. What this Actor documents that neither does: mixed keyword, hashtag, search-URL and profile targets in one list, a time window enforced on both sort modes, the auth_token / ct0 requirement stated as a prerequisite rather than omitted, unbilled accounting rows with a named status vocabulary, and run-wide deduplication across targets.

Does X (Twitter) Posts Search return data in a format LLMs can use directly?

Yes. Every row is typed, normalized JSON with the same 23 field names on every run. No HTML parsing, no selectors. Pass a row straight into an LLM context window, index it into a vector store, or hand it to an agent tool. postText already contains the full long-form note text where X sends one, so nothing needs re-assembling.

What happens when X changes its API or anti-bot system?

The scraper is maintained, and it is built so that X's most frequent change breaks nothing: the bearer token and GraphQL query IDs are harvested from x.com's live bundle on every run rather than hardcoded, and a stale search query ID triggers a single automatic re-harvest and retry. Because it reads X's GraphQL payloads rather than rendered markup, a front-end redesign does not affect your field names or types.

Can I use X (Twitter) Posts Search without managing proxies or browser infrastructure?

Yes. There is no browser to run and no proxy account to create — enable Apify Proxy in proxyConfiguration and the Actor holds one session for the whole run; leave it off and it connects directly. It does not solve CAPTCHAs, because the endpoints it uses do not present one. What it handles for you is typed block detection, retries with backoff on network and 5xx errors, an immediate stop on HTTP 429, and reading X's rate-limit headers so it stops before your account's quota is gone. The one thing you manage yourself is your X cookies, which expire.

Which X post fields work best for AI training data and RAG indexing?

For RAG indexing: postText is the high-information field and chunks cleanly on its own, with author.screenName, target and lang as metadata filters and postedAt for recency ranking. For training data and analytics: favouriteCount, repostCount, replyCount, quoteCount, bookmarkCount and timestamp are the most structurally consistent fields across records, with viewCount the one to null-check because X does not always send it. All values are returned as typed primitives, arrays or plain nested objects, so no normalization pass is needed before indexing.

🔗 Related scrapers

Scraper Name	What it extracts
Twitter X Posts Scraper	Posts from X profile targets, with no session cookies required
Twitter X Reply Scraper	Replies under X posts, using your authToken and ct0
Twitter Trends Scraper	X trending topics by country and hour, with board rank
Threads User Posts Scraper	Posts from Threads profiles with keyword and date filters
Reddit Posts Scraper	Reddit posts by keyword with score filtering
Facebook Hashtag Search Scraper	Facebook posts published under a hashtag

💬 Your feedback

Found a bug, or need a field that is in X's timeline payload but not in the output? Open an issue on the Actor's Issues tab and it will be looked at. Reports that include the exact startUrls entry, the searchType you used and the run ID are the fastest to reproduce and fix.