X / Twitter Scraper - 42 Fields Per Post

X.com Posts Scraper

Scrape X.com posts from user profiles and post URLs, up to a million per run. Each post comes with its full text, engagement counts, view count, timestamp and author profile. No API key and no login needed. Export to CSV, JSON, Excel or XML.

X's official API needs a developer account, project approval and a paid tier for any real volume. This Actor reads the same public data a logged-out visitor sees on x.com and returns it in one fixed schema. Keyword search across all of X is possible only if you add your own X session (see below), because X allows no search at all to logged-out visitors.

Who uses it	What they scrape X.com for
Market researchers	Track what a set of influential accounts are saying about a product category this month.
PR and comms teams	Monitor executive or brand account posts for sentiment and narrative shifts.
Data journalists	Collect a corpus of posts around a breaking event from known sources.
Growth marketers	Analyze competitor posting cadence and top-performing content by engagement.

What it does

The Actor collects posts from user profiles and direct post URLs and returns each one as a row with full text, engagement counts and the author's profile.

• Profiles: give usernames or profile URLs and it returns their posts, newest first.
• Single posts: give post URLs (https://x.com/user/status/123) and it returns those posts.
• Keyword filter: add a search term to keep only the posts from those accounts that contain it.
• Keyword search across X (optional, needs your session): with your own X session cookies, the search term also runs as a search of all of X, latest posts first.
• Enrichment (optional): attach Google Trends momentum, YouTube interest or Apollo.io company profiles to each post.

What you get for very large accounts

For some very large accounts (for example elonmusk, BarackObama, nytimes, naval, BillGates), X does not give logged-out visitors their timeline. It returns the account's latest few posts on the profile page and, through its API, a fixed set of about 100 all-time top posts from past years, with no way to page further back.

The Actor handles this case as follows:

• The latest posts from the profile page come first, in date order, with isTopPostsFallback: false.
• The all-time top posts follow, with isTopPostsFallback: true. They are real posts with real counts, but they are not the account's recent timeline and they skip long stretches of time.
• The run log says when an account was served this way.

If you need only recent posts, filter out rows where isTopPostsFallback is true. Accounts of normal size are not affected: their timeline pages back through time as usual.

What you can do with X.com data

Monitor brand or executive accounts.

A comms team lists five executive usernames, sets maxItems to 5000, and exports a CSV of their posts to audit messaging consistency.

Research a topic across known voices.

A market researcher enters ten fintech founder usernames and the search term 'regulation', then keeps only the matching posts.

Collect posts around a breaking event.

A journalist feeds the Actor a list of post URLs from a news thread and exports the full text and timestamps.

Benchmark competitor content performance.

A growth marketer scrapes a competitor's profile, sorts by views or likes in a spreadsheet, and looks at which post formats got the most replies.

Why choose this scraper

	What you get
No API key or approval	Reads the public, logged-out view of X. No developer account and no OAuth app.
Up to 1,000,000 posts per run	Set the volume once. Pagination, proxy rotation and retries are handled for you.
Honest rows	Top posts served instead of a timeline are flagged, and failed targets come back as unbilled error rows with a code.
Fixed flat schema	Every post arrives with the same columns, ready for pivot tables, BI tools or a database.

Configure the run

Give usernames, profile URLs or post URLs. The optional search term filters the collected posts. The Input tab lists every parameter.

A first run with the defaults:

{
 "maxItems": 10,
 "usernames": [
 "elonmusk"
 ]
}

A larger pull from two accounts, keeping only posts that mention Artemis:

{
 "maxItems": 200,
 "usernames": ["NASA", "NASAArtemis"],
 "searchQuery": "artemis"
}

Proxy

Runs use Apify Proxy datacenter IPs by default. That is enough for public profiles and posts. When X rate-limits a request, the Actor switches to a new IP with a new guest token and retries, a few times at most. You can pick Residential in the Proxy field if an account keeps coming back empty.

Optional: your own X session for keyword search

Logged out, X has no search at all: the search page redirects to login and the search API refuses guest requests. If you want the search term to search all of X, you can add the cookies of your own logged-in X session.

1. Log in to x.com in Chrome with a secondary account. Do not use your main account: X can lock accounts that are used for automated searching.
2. Open DevTools (F12), then Application, then Cookies, then https://x.com.
3. Copy the value of auth_token into "X session: auth_token cookie" and the value of ct0 into "X session: ct0 cookie". You can instead paste the whole cookie string into the third field.

How the session is used:

• The values are stored encrypted by Apify, are never written to the log or the dataset, and are sent only to x.com.
• Search requests keep one IP for the whole run and wait a few seconds between pages.
• If X answers 401, 403 or 429, the Actor stops using the session at once. The search then comes back as an unbilled error row with errorCode: "INVALID_SESSION", and the profiles and posts you listed are still collected.
• Search results have collectedFrom: "search" and are not re-filtered by the local keyword rules.

The logged-in search path has been checked against X's answers to invalid cookies only. It has not been run with a real account, so treat it as experimental.

Output fields

Every post row has the same columns as before, plus three newer ones:

Field	Meaning
viewCount	Views X shows under the post, or null when X does not report it.
isTopPostsFallback	True for all-time top posts X served in place of the timeline (see above). False otherwise.
collectedFrom	timeline, profilePage, post or search.

A target that produced nothing (a handle that does not exist, a suspended or protected account, a deleted post, a rejected session) is written as one error row with error, errorCode and target. Error rows are never charged.

Pricing

Pay per result: $0.25 per 1,000 posts on the Free plan, a little less on paid plans. Each run also has a start fee of about $0.08 per GB of memory (the default 256 MB counts as one). Error rows are not charged.

Posts collected	Approximate cost (Free plan)
100 posts	$0.03 plus the start fee
1,000 posts	$0.25 plus the start fee
10,000 posts	$2.50 plus the start fee

Optional connector enrichment: with enrichOutput on, each post whose connector lookup (Apollo or Trends) comes back adds a connector-enrichment event at $0.005 ($5 per 1,000). Leave it off and you pay only the prices above.

New Apify accounts start with $5 in free credit.

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the X.com Posts Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to X.com through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

$claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/x-com-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check that the usernames exist and are not suspended, and that your search term, if any, appears in their posts. Each target that produced nothing has an error row in the dataset that says why. A run that collects no posts at all ends as FAILED with the reason in its status message, and nothing is charged for it. Some accounts return an empty timeline to logged-out visitors even though they have posts; the Actor then falls back to the posts on the profile page, and a residential proxy may also help.

The newest posts of a big account are there, but then the dates jump back years.

That is X's top-posts behavior for very large accounts, described above. Those rows have isTopPostsFallback: true.

The Actor stops before reaching my maxItems limit.

It stops when it runs out of matching posts. For the very large accounts above, only the latest few posts and about 100 top posts exist in the logged-out view. If you filter by keyword, try a broader term or remove it.

I get an error about enrichment failing.

Make sure you have selected a valid enrichment connector and that its MCP server is reachable. If you use the override fields for proxy URL or token, clear them and try again with the defaults.

The dataset has fewer columns than I expected.

Enrichment columns appear only when enrichOutput is on and a connector is selected.

FAQ

Question	Answer
Do I need an X.com API key or login?	No, not for profiles and posts. A login (your own session cookies) is needed only for keyword search across all of X.
Can I search all of X for a keyword without listing usernames?	Only with your own X session, as described above. Without it, the search term filters the posts of the accounts you list, and a search term with no accounts ends the run with an explanation.
What is the difference between startUrls and usernames?	startUrls accepts post or profile URLs. usernames accepts handles. You can use both: if they name the same account it is scraped once, and if they disagree the URLs win and the ignored usernames are listed in the run log.
Does the run fail if my input is wrong?	Input that can be understood is repaired (a leading @, a missing https://, a profile URL typed into the usernames field). The run ends as FAILED only when it collects no posts at all, and then the status message says why.
Why are some rows marked isTopPostsFallback?	X served that account's all-time top posts instead of its timeline to a logged-out visitor. The rows are real, but not in date order.
How many posts can I collect in one run?	Set maxItems from 1 to 1,000,000. The Actor stops at your limit or when no more posts are available.
What does the enrichment feature do?	It attaches cross-platform context to each post: Google Trends momentum and YouTube interest for the topic, or Apollo.io company data for the post author, depending on the connector.
What export formats are supported?	CSV, JSON, Excel or XML, straight from the Apify platform.
Can I scrape replies to a specific post?	No. The Actor does not fetch reply threads. Give it the individual post URLs you need.
Is this legal?	The Actor reads publicly available pages. You are responsible for complying with X's terms of service and the laws that apply to you. Using your own session for search is your decision and your account.
How do I scrape a private account?	You can't. Protected accounts come back as an error row with errorCode PROTECTED.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by X Corp. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.