Twitter (X.com) Tweets & Profiles Scraper

Twitter (X) Post Scraper — Text, Likes, Views and Author Data

Twitter (X.com) Tweets & Profiles Scraper pulls a public X (Twitter) profile's recent posts — full text, exact like/reply/repost/quote/view counts, media URLs, outbound links and the author's profile stats — into structured JSON, no login required. Pinned posts are flagged, and repost rows resolve to the original post's text and author instead of a truncated stub. Keyword and hashtag search works too, with an optional X session cookie. Give it a profile URL, a username or a search term and run it on Apify.

🧭 What is Twitter (X.com) Tweets & Profiles Scraper?

Twitter (X.com) Tweets & Profiles Scraper is an Apify Actor that reads a public X profile's post timeline directly from X's own GraphQL API and returns each post as a normalized JSON row, with the author's profile object attached. No X account or login is required for profile scraping — a guest session is enough. Keyword search, hashtag search and X's dedicated replies timeline are gated by X itself and need an optional session (authToken + ct0 cookies) supplied as input. It's built for developers, social-media analysts, researchers and AI engineers who want X post data as clean JSON rather than screen-scraped HTML.

🔓 What X (Twitter) post data is publicly available to scrape?

Anyone who opens a public X profile in a browser without logging in can already see its recent posts, engagement counts and basic profile stats — that is exactly what this Actor returns.

Data category	Publicly available	Restricted (needs an X login)
Post text, timestamp, links, media	✅
Like, reply, repost, quote and view counts	✅
Pinned-post flag	✅
Author bio, location, website, join date, follower/following/post counts	✅
Keyword and hashtag search (SearchTimeline)		✅ needs authToken + ct0
Dedicated replies timeline (UserTweetsAndReplies)		✅ needs a session; some replies still surface on the public timeline without one
Single-post pages (/status/...), follower lists, DMs		✅ not returned by this Actor at all

Twitter (X.com) Tweets & Profiles Scraper only returns publicly visible data — what any visitor sees. Nothing behind a login wall.

📋 What data can I extract with Twitter (X.com) Tweets & Profiles Scraper?

Every dataset row is one post, carrying the post's own content and metrics plus a nested author-profile object.

Post identity and content fields

Field	Description
id	Post's unique numeric ID as a string. On a repost row this is the ID of the repost entry itself, not the original post.
url	Canonical https://x.com/<user>/status/<id> URL. On a repost row it points at the original post.
username	Author handle, prefixed with @ (e.g. @NASA).
fullname	Author's display name.
verified	Whether the post's author has X's blue-check flag (is_blue_verified). True for Premium, Business and Government accounts alike.
timestamp	Post creation time in X's own format (e.g. Fri Jul 31 20:00:46 +0000 2026), UTC.
text	Full post text. Long-form ("Article"/note-tweet) posts return the untruncated body, not a 140-character stub.
links	De-duplicated list of outbound URLs found in the post's link entities, card and media, and its quoted post if any.
media	List of {type, url} objects — photo, video or animated_gif, with the highest-bitrate MP4 variant for video.

Engagement, type and repost fields

Field	Description
likes	Like count (favorite_count) — integer, or null when X does not publish it.
replies	Reply count — integer or null.
retweets	Repost count — integer or null.
quotes	Quote count — integer or null.
views	View count — integer or null. Some older posts and replies have no view counter and return null, never 0.
isPinned	true if the post is pinned to the profile.
isQuote	true if the post quotes another post.
isRetweet	true if this row is a repost. A repost row carries the original post's text, author, timestamp, counts and media.
isReply	true if the post is a reply to another post.
tweetUserId	The post author's numeric X user ID.
retweetedFrom	The reposting account's handle (@handle) — set only on repost rows.
originalTweetId	The original post's numeric ID — set only on repost rows.

Author profile fields (user object)

Attached to every row when includeUserInfo is true; omitted entirely when it is false.

Field	Description
avatar	Profile image URL (the _normal, 48px size X serves).
username	Author's handle, without @.
userFullName	Author's display name.
description	Bio text, or null if the account left it empty.
location	Free-text profile location, or null.
website	Expanded (unwound) website URL from the bio link, or null.
joinDate	Account creation date, in X's own format.
verified	Blue-check flag — same meaning as the top-level verified field.
totalLikes	Lifetime like count for the account.
totalTweets	Lifetime post count.
totalFollowing	Number of accounts followed.
totalFollowers	Follower count.
url	The bio link exactly as X shortens it (e.g. a t.co link), or null.
userId	Author's numeric X user ID — same value as tweetUserId.
totalMediaCount	Lifetime media-post count.

🤖 Add-on: Need additional X (Twitter) data?

For single-post URLs, poll results and bookmark counts, the same account also publishes Twitter X Tweets Scraper Scraper, which scrapes individual /status/ links directly. For other platforms, see Related scrapers below for profile-and-post scrapers on LinkedIn, Threads, TikTok and Pinterest.

🆚 How does Twitter (X.com) Tweets & Profiles Scraper differ from the official X API?

X's official API (X API v2) requires a registered developer account and either per-request billing or a paid tier. As of February 2026, new developer access defaults to pay-per-use pricing — $0.015 per post created, $0.005 per post read, capped at 2,000,000 reads per month — with the old free tier discontinued and legacy Basic/Pro/Enterprise tiers grandfathered for existing subscribers only (checked at docs.x.com/x-api and current X API pricing pages, 2026-08-04). Twitter (X.com) Tweets & Profiles Scraper needs none of that: point it at a profile URL or username and it returns post data without a developer application.

Feature	X API v2	Twitter (X.com) Tweets & Profiles Scraper
Access setup	Developer account + app registration; approval for legacy tiers	Run directly on Apify, no application
Cost model	Pay-per-use, or legacy tiers from ~$200/mo to Enterprise pricing	Apify pay-per-event, billed per post returned
Rate limits	Per-endpoint 15-minute windows (e.g. 450 requests/15 min per app for recent search), independent of spend	maxTweets capped at 100 per target; also bound by X's own guest/session rate limit
Keyword / full-text search	Recent search on paid tiers; full-archive search restricted to Pro/Enterprise	Supported with an optional session cookie, no separate tier
Profile timeline access	Requires a paid tier's users/:id/tweets endpoint	Included by default, keyless
Author profile data	Separate users endpoint call	Attached to every row automatically

Use the official X API when you need write access (posting, DMs) or a contractual support relationship with X. Use Twitter (X.com) Tweets & Profiles Scraper when you only need to read public post and profile data without a developer application or per-tier billing.

▶️ How to use Twitter (X.com) Tweets & Profiles Scraper

No coding is required to run it from the Apify Console.

1. Open Twitter (X.com) Tweets & Profiles Scraper on the Apify Store and click Run, or call it via the Apify API / apify-client.
2. Fill in startUrls — the only required field — with one target per line.
3. Optionally set maxTweets (1-100), withReplies, includeUserInfo, and add authToken + ct0 if you need keyword search or the full replies timeline.
4. Start the run.
5. Download the results as JSON, CSV, Excel or any other format the Apify dataset export supports.

Accepted startUrls targets:

You enter	The Actor does
https://x.com/nasa, twitter.com/nasa, nasa, @nasa	Scrapes that profile's timeline
11348282 (all digits)	Treated as a numeric user id
search: mars rover, #space, https://x.com/search?q=...	Keyword/hashtag search — needs authToken + ct0
https://x.com/nasa/status/123...	Not supported — skipped with a log line, never silently turned into a profile scrape

⚠️ A bare word with no spaces and no search: prefix (e.g. bitcoin) is a valid X handle length (1-15 characters), so it is routed as a username, not a keyword search. Use search: bitcoin when you mean the keyword.

How to scale to bulk post extraction

startUrls is an array — add as many profile URLs, usernames, user ids or (with a session) keywords as you want, one per line, and they are all scraped in the same run. Posts are de-duplicated by id across every target in the run, so overlapping targets are never billed twice. There is no separate "bulk mode" input; a single run with a long startUrls list already covers it.

💡 What can you do with X (Twitter) post data?

• 📊 A social-media analyst tracking brand mentions uses likes, retweets and views on repost rows to see which reposted content actually drives engagement.
• 🔬 A researcher studying public discourse uses text, timestamp and isReply to reconstruct a thread's chronology without needing X API access.
• 📈 A growth marketer benchmarking accounts uses user.totalFollowers, user.totalTweets and verified to compare size and posting cadence across a watchlist.
• 🏢 A brand-monitoring team uses isPinned and text to catch messaging changes on a pinned post before they spread.
• 🤖 An AI engineer building a RAG pipeline over public X commentary uses text and links as source passages, indexing id and timestamp as stable document keys.

Because the output is typed JSON, any of these fields can be piped straight into an agent tool or automation without extra parsing.

🛡️ How does Twitter (X.com) Tweets & Profiles Scraper handle rate limits and blocking?

The Actor talks to X's own GraphQL API directly over curl_cffi with Chrome TLS-fingerprint impersonation — there is no headless browser to detect. Each run activates a guest token (or uses your session cookies, if supplied) and discovers X's live GraphQL query ids from X's own web bundle at the start of the run, falling back to a built-in id map if discovery fails, so the Actor keeps working when X changes its internal endpoint ids.

Every request goes through the configured proxy (Apify Proxy by default). The Actor reads the x-rate-limit-remaining header X returns and logs a warning once the budget is nearly spent. On an HTTP 429 it waits for the window X reports in x-rate-limit-reset (5-90 seconds) and retries once; if a target still returns zero rows and a proxy is configured, it retries once more on a fresh connection before moving on. A run that ends with zero posts across every target fails loudly instead of finishing with a silently empty dataset.

⬇️ Input

Parameter	Required	Type	Description	Example value
startUrls	Yes	array	One target per line: profile URL, username (nasa or @nasa) or numeric user id — all work without a session. Keyword search needs a session: search: mars rover, #space, or an x.com/search?q=... URL. Single-post (/status/) URLs are skipped with a log line.	["https://x.com/nasa", "spacex"]
maxTweets	No	integer	Maximum posts to return per target (1-100, default 10). The Actor stops paging as soon as it has this many.	25
withReplies	No	boolean	Keep the account's replies in the output (default true). false returns original posts and reposts only. With a session, true also switches the run to X's dedicated replies timeline.	true
includeUserInfo	No	boolean	Attach the author profile object to every row (default true).	true
proxyConfiguration	No	object	Proxy used for every request the Actor makes (default {"useApifyProxy": true}). Residential exits are the most reliable against X.	{"useApifyProxy": true}
authToken	No	string (secret)	Value of the auth_token cookie from a logged-in x.com browser session. Only needed for keyword/hashtag search and the dedicated replies timeline.	""
ct0	No	string (secret)	Value of the ct0 cookie from the same session. Must be supplied together with authToken.	""

⚠️ maxTweets is hard-capped at 100 per target — in both the input schema and the code — so requesting more simply returns 100. Keyword and hashtag targets are skipped entirely (with a log line) unless authToken and ct0 are both supplied.

Example input

{
  "startUrls": [
    "https://x.com/nasa",
    "spacex",
    "@apify"
  ],
  "maxTweets": 25,
  "withReplies": true,
  "includeUserInfo": true,
  "proxyConfiguration": { "useApifyProxy": true },
  "authToken": "",
  "ct0": ""
}

⬆️ Output

Results are typed, normalized JSON — one row per post, with a consistent schema across runs. Export as JSON, CSV, Excel, XML or HTML directly from the Apify dataset, or read it via the API/apify-client.

Example output

{
  "id": "1954832017744209152",
  "url": "https://x.com/NASA/status/1954832017744209152",
  "verified": true,
  "username": "@NASA",
  "fullname": "NASA",
  "timestamp": "Tue Aug 12 15:32:10 +0000 2026",
  "text": "New images from the Perseverance rover show the clearest view yet of the Jezero Crater delta.",
  "links": ["https://www.nasa.gov/perseverance"],
  "isPinned": false,
  "isQuote": false,
  "isRetweet": false,
  "isReply": false,
  "likes": 8420,
  "replies": 312,
  "retweets": 1560,
  "quotes": 88,
  "views": 512300,
  "user": {
    "avatar": "https://pbs.twimg.com/profile_images/.../NASA_normal.jpg",
    "username": "NASA",
    "userFullName": "NASA",
    "description": "Explore the universe and discover our home planet.",
    "location": "Washington, DC",
    "website": "https://www.nasa.gov",
    "joinDate": "Wed Dec 19 20:20:32 +0000 2007",
    "verified": true,
    "totalLikes": 8934,
    "totalTweets": 76210,
    "totalFollowing": 380,
    "totalFollowers": 90310442,
    "url": "https://t.co/9RGofFTOO7",
    "userId": "11348282",
    "totalMediaCount": 21540
  },
  "media": [{ "type": "photo", "url": "https://pbs.twimg.com/media/example.jpg" }],
  "tweetUserId": "11348282",
  "retweetedFrom": null,
  "originalTweetId": null
}

⚙️ How does it work?

Twitter (X.com) Tweets & Profiles Scraper connects to the same GraphQL endpoints X's own web app uses, over an HTTP client that mimics a real Chrome browser at the TLS level, instead of driving a browser. It resolves your profile URL, username or numeric id to an X user id, then pages through that account's timeline — or the search timeline, with a session — until it has collected maxTweets posts. Anti-bot handling is limited to guest-token activation, TLS-fingerprint impersonation and proxy rotation; there is no CAPTCHA-solving step. Only what X serves to a logged-out (or, with a session, logged-in) viewer is ever returned — nothing is inferred or pulled from a private surface. The output schema stays the same run to run, even as X's own web bundle and internal query ids change underneath it.

🔌 Integrations

Twitter (X.com) Tweets & Profiles Scraper works with whatever you already use to run Apify Actors.

Calling it programmatically

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_API_TOKEN>")

run = client.actor("YOUR_USERNAME/twitter-x-tweets-profiles-scraper").call(run_input={
    "startUrls": ["https://x.com/nasa", "spacex"],
    "maxTweets": 20,
    "withReplies": True,
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["username"], item["text"][:80])

Works in Go, Ruby, Node.js, cURL — any language that can make an HTTP request.

No-code tools (n8n, Make, LangChain)

In n8n, use the HTTP Request node against the Apify API (or the community Apify node) to start a run and pull the resulting dataset. In Make, the Apify app's "Run Actor and Get Dataset Items" module does the same in two steps. In LangChain, ApifyWrapper can run the Actor and load its dataset directly into a document loader for RAG.

⚖️ Is it legal to scrape X (Twitter) posts?

Scraping publicly available web data is generally lawful — courts in the hiQ Labs v. LinkedIn line of cases have distinguished scraping public pages from unauthorized computer access. Twitter (X.com) Tweets & Profiles Scraper only returns what X already serves to a logged-out visitor: post text, engagement counts and public profile fields. Because that includes personal data — handles, display names, bios, locations and follower counts tied to identifiable people — GDPR (for EU residents' data) and CCPA (for California residents) still govern how you store, use and retain it, regardless of its public availability; you need a lawful basis and should honor access/deletion requests. Consult legal counsel if your use case involves bulk storage of personal data.

❓ Frequently asked questions

What post fields does Twitter (X.com) Tweets & Profiles Scraper return?

text, likes, retweets, views and user (the author's profile object) are the fields most people reach for first. See What data can I extract for the full list.

Does it require an X account or login?

No, not for profile scraping — a guest session is enough. Keyword search, hashtag search and the dedicated replies timeline are gated by X itself and need authToken + ct0 session cookies from a logged-in x.com browser.

How many posts can I extract in one run?

Up to 100 per target (the maxTweets maximum), and you can list as many targets as you want in startUrls — posts across all of them count toward the same run.

What happens if a profile is private, suspended or doesn't exist?

The Actor logs that the handle isn't a public account and moves to the next target, pushing zero rows for it — it does not throw for the whole run. Only a run where every target returns zero posts fails, with the reason logged (rate-limited, keyword search skipped for lack of a session, or "see the per-target log lines").

Can I scrape multiple X profiles, users or keywords in one run?

Yes — startUrls accepts one target per line: any mix of profile URLs, usernames, numeric user ids and (with a session) keywords or hashtags, all in the same run.

Does it work with Claude, ChatGPT and other AI agent tools?

It is callable as an HTTP endpoint (via the Apify API or apify-client) by any agent framework that can make an HTTP request — there is no dedicated MCP server documented for this Actor.

How does it compare to other X (Twitter) scrapers?

The same publishing account also runs Twitter X Tweets Scraper Scraper, which additionally accepts single-post (/status/) URLs and returns poll and bookmark data; this Actor focuses on profile timelines with a full author-profile object and repost-to-original resolution on every row.

Does it return data in a format LLMs can use directly?

Yes. Typed, normalized JSON with consistent field names across runs — no HTML parsing, no CSS selectors. Pass it directly to an LLM, index it into a vector store, or feed it to an agent tool.

What happens when X changes its layout or anti-bot system?

The Actor discovers X's live GraphQL query ids from X's own web bundle on every run, with a built-in fallback map, so it keeps working across most internal endpoint-id changes. The output schema is maintained; no specific update turnaround time is published.

Can I use it without managing proxies or browser infrastructure?

Yes. There is no browser to run — requests go directly to X's GraphQL API — and proxy configuration is a single input field backed by Apify Proxy.

Which post fields work best for AI training data and RAG indexing?

For RAG, index text (and links for source context) as the passage, with id and timestamp as stable keys. For training data, likes, replies, retweets, quotes and views return as consistent typed integers (or null) across every row.

🔗 Related scrapers

Scraper	What it extracts
Twitter X Tweets Scraper Scraper	Public X posts from profiles or single-post URLs, plus polls, bookmarks and quoted posts
LinkedIn Profile Post Scraper	LinkedIn profile details and post history
Threads Search Post Scraper: Posts, Quotes & Replies	Threads posts, quotes and replies by search
TikTok Data Scraper	TikTok videos and profile data
Pinterest Profile Scraper	Pinterest profile pins and boards
Reddit Subreddit Members Scraper (Active Participants)	Active members of a subreddit

💬 Your feedback

Found a bug or missing a field? Open an issue on this Actor's Issues tab in Apify Console, or use the "Contact developer" button on the Store listing — both reach the maintainer directly.