# Twitter Posts Scraper (`socialdataapi/twitter-posts-scraper`) Actor Twitter Posts Scraper extracts public posts from X (Twitter) accounts by username or URL: text, likes, replies, reposts, quotes, bookmarks, media, links, hashtags and author profile, up to 100 posts per account. No login needed. Export to JSON, CSV or Excel. - **URL**: https://apify.com/socialdataapi/twitter-posts-scraper.md - **Developed by:** [Social Data API](https://apify.com/socialdataapi) (community) - **Stats:** 2 total users, 1 monthly users, 50.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.67 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ### Twitter Posts Scraper — Post Text, Engagement, Media and Author Data Twitter Posts Scraper collects public posts from the X (Twitter) accounts you list: the full post text, likes, replies, reposts, quotes and bookmarks, photos and video links, hashtags, mentions and links, plus the author's profile with follower counts. Every response is structured JSON — ready to pass directly to an LLM, index into a vector store, or feed a social listening or content analysis pipeline. **No X account, login or cookies needed.** Start with one account, then follow every voice in your market. *** ### What data can I extract with Twitter Posts Scraper? Using this X posts scraper, you will be able to extract the following data for every post: | 📝 Full post text | ❤ī¸ Likes | đŸ’Ŧ Replies | 🔁 Reposts | |---|---|---|---| | 🗨ī¸ Quotes | 🔖 Bookmarks | 📌 Pinned post flag | ↩ī¸ Reply and conversation IDs | | đŸ–ŧī¸ Photos and video links | #ī¸âƒŖ Hashtags and mentions | 🔗 Links and link cards | 🌍 Language | | 👤 Author name and handle | đŸ‘Ĩ Followers and following | đŸĒĒ Bio, location and pictures | 🕒 Post time and time collected | *** ### How to use Twitter Posts Scraper Built for marketers, researchers and data teams: one run turns a list of X accounts into a table of their posts with engagement numbers, each returned as structured JSON with 32 fields covering text, engagement counts, media, entities and the author's profile. Order per account: pinned post, then the newest posts, then the timeline, then (when X ends the timeline early) the account's highlighted posts. #### Twitter Posts Scraper MCP Server: use these Actors from ChatGPT, Claude or Cursor Point your AI assistant at Apify and it can read X accounts on its own: the latest posts of a brand, a founder or a newsroom, with their engagement numbers. No glue code, no scraping logic in your prompts. ```bash claude mcp add --transport http apify "https://mcp.apify.com?tools=socialdataapi/twitter-posts-scraper" ``` #### Step by step 1. [Create a free Apify account](https://console.apify.com/sign-up). 2. Open [Twitter Posts Scraper](https://apify.com/socialdataapi/twitter-posts-scraper) in the Apify Store. 3. Add the X accounts you want, one per line: a username, @username, profile URL or post URL. 4. Set how many posts you want per account and click **Start** (or send your API request). 5. Download or stream your results in JSON, CSV or Excel. #### How to scale Twitter Posts Scraper to bulk Put any number of accounts in one run and set `maxTweets` up to 5,000 per account. All accounts are read at the same time, and rows are saved page by page, so the dataset fills while the run is going. Recorded runs: 1,000 posts for @elonmusk in 31 seconds, 500 for @nasa in 37 seconds, and 987 for @nasa when 1,000 were asked (everything X shows logged-out visitors for that account) in 69 seconds. To export tweets for a whole list of competitors or journalists, put them all in one run, then download the dataset as CSV or Excel — tweets to CSV in one click. To follow accounts over time, schedule the same run daily in Apify Console. *** ### ✨ Features Twitter Posts Scraper fetches clean, structured post data from public X profiles ⚡. It's ideal for social listening, brand monitoring, influencer research and AI datasets — a tweet scraper, Twitter timeline scraper and Twitter data extractor in one, and a simple Twitter API alternative when you only need public posts. It extracts, per post: - **Full text** – the complete text, also for long posts, with its line breaks - **Engagement** – likes, replies, reposts, quotes and bookmarks - **Media** – photos, videos and GIFs with their links, sizes and alt text - **Context** – reply and conversation IDs, repost and quote flags, the pinned post flag, link cards and places - **Entities** – hashtags, cashtags, links and mentions, always present as lists - **Author** – name, handle, bio, location, followers, following, post and media counts, verification, profile and cover pictures ⚡ Each account's pinned post comes first, then the newest posts shown on the profile, then the rest of the timeline. When X ends the logged-out timeline early, the account's highlighted posts (its most-engaged posts, not in date order) follow, with no post saved twice. *** ### Use cases 💡 - **Social media manager** — scrape tweets of competitors and compare what gets the most likes and reposts. - **Brand team** — follow the posts of partners, founders and newsrooms in one table. - **Researcher** — build a dataset of public posts from a list of accounts for analysis. - **Influencer marketer** — check engagement and follower counts before a collaboration. - **AI engineer** — feed post text into an LLM to summarize what an account talks about. Building an AI agent? [Connect Twitter Posts Scraper over MCP](#mcp-integration-for-ai-agents) and let it call the actor on its own. *** ### Why choose this actor? 1. **Complete posts.** Every row carries all 32 post fields plus 29 author fields on every run, long posts come back with their full text in `fullText`, self-threads are included, and promoted posts in the timeline are left out. 2. **Fast.** A recorded run returned 300 posts for three accounts in 6.1 seconds; another returned 500 posts for five accounts in 7.4 seconds. 3. **Works when X refuses a request.** Requests are retried on a fresh connection, and through Apify residential proxy when X refuses a direct one; a test run with every direct request refused still returned all 60 posts asked for. *** ### âŦ‡ī¸ Input | Parameter | Required | Type | Description | Example value | |---|---|---|---|---| | `startUrls` | Yes | array | X accounts, one per line: username, @username, profile URL (x.com or twitter.com) or post URL (its author is used). Repeated accounts are read once. | `["NASA", "https://x.com/github"]` | | `maxTweets` | No | integer | Most posts saved per account, 1–5000. Default `10`. | `500` | | `proxyConfiguration` | No | object | Optional — leave it off. The first request goes direct; when X refuses it, retries go through Apify residential proxy. A proxy you select is used for every request. | `{"useApifyProxy": false}` | X limits what logged-out visitors see. For some very large accounts (for example @elonmusk, @nytimes) the timeline is one page of about 100 popular posts; for others it ends after several hundred recent posts (@nasa: 790, back about seven months, in October 2026). The actor then adds the account's highlighted posts, so @elonmusk returned 1,000 posts and @nasa 987; @nytimes has only a couple of highlighted posts, so it returns about 100. The timeline part can include older posts; the newest posts shown on the profile are added first whenever X includes them on the public profile page. #### Free plan limits Free plan: up to 20 posts per run (all accounts together) and 200 per month; paid plans have no limit. A free-plan run with several accounts shares that same 20-post ceiling across all of them, not 20 per account — the Actor stops the whole run once the run total is reached and tells you so in the log and status message. Paid plans, local `apify run`s and Apify's own test runs are never limited. The monthly count is kept in a small storage in your own Apify account, separate for this Actor; if that storage can't be reached during a run, the run still delivers up to 20 posts and the log says "monthly tracking unavailable this run". #### Example input ```json { "startUrls": [ "NASA", "@github", "https://x.com/sama" ], "maxTweets": 500 } ``` *** ### âŦ†ī¸ Output Every post is one typed, normalized JSON item with the same 32 fields on every run; values X does not give are `null`, empty strings or empty objects and lists. Export the dataset as JSON (default), CSV or Excel. #### Example output One recorded row, a text post by @sama: ```json [ { "type": "tweet", "id": "2107691261776052633", "imageUrl": "https://pbs.twimg.com/profile_images/2046764873200394240/r7BxVezs_normal.jpg", "url": "https://x.com/sama/status/2107691261776052633", "twitterUrl": "https://twitter.com/sama/status/2107691261776052633", "text": "looking up at the stars with extra awe tonight.\n\n\"thy sea is so great and my boat is so small.\"", "fullText": "looking up at the stars with extra awe tonight.\n\n\"thy sea is so great and my boat is so small.\"", "source": "", "retweetCount": 676, "replyCount": 1209, "likeCount": 12598, "quoteCount": 302, "createdAt": "Wed Oct 07 04:35:30 +0000 2026", "lang": "en", "bookmarkCount": 958, "isReply": false, "inReplyToId": null, "conversationId": "2107691261776052633", "inReplyToUserId": null, "inReplyToUsername": null, "isPinned": false, "author": { "type": "user", "userName": "sama", "url": "https://x.com/sama", "twitterUrl": "https://twitter.com/sama", "id": "1605", "name": "Sam Altman", "isBlueVerified": true, "profilePicture": "https://pbs.twimg.com/profile_images/2046764873200394240/r7BxVezs_normal.jpg", "coverPicture": "", "description": "The mission of OpenAI is to ensure that AGI benefits all of humanity", "location": "SF", "followers": 6377977, "following": 1034, "status": "", "canDm": false, "canMediaTag": false, "createdAt": "Sun Jul 16 22:01:55 +0000 2006", "entities": { "description": {}, "url": { "urls": [ { "display_url": "blog.samaltman.com", "expanded_url": "http://blog.samaltman.com", "indices": [0, 23], "url": "https://t.co/teHmy8EFSH" } ] } }, "fastFollowersCount": 0, "favouritesCount": 1016, "hasCustomTimelines": false, "isTranslator": false, "mediaCount": 394, "statusesCount": 7999, "withheldInCountries": [], "affiliatesHighlightedLabel": { "label": { "badge": { "url": "https://pbs.twimg.com/profile_images/1885410181409820672/ztsaR0JW_bigger.jpg" }, "description": "OpenAI", "url": { "url": "https://twitter.com/OpenAI", "urlType": "DeepLink" }, "userLabelDisplayType": "Badge", "userLabelType": "BusinessLabel" } }, "possiblySensitive": false, "pinnedTweetIds": [], "professional": {} }, "authorDisplay": "@sama", "extendedEntities": {}, "card": {}, "place": {}, "entities": { "hashtags": [], "smarttags": [], "symbols": [], "timestamps": [], "urls": [], "user_mentions": [] }, "isRetweet": false, "isQuote": false, "media": [], "isConversationControlled": false, "scrapedTimestamp": "2026-10-14T07:52:11.330Z" } ] ``` On posts with photos or videos, `media` and `extendedEntities.media` hold one object per item, with the image link, sizes and, for videos, the download links in `video_info.variants`. `source` and `author.status` are empty because X does not show them to logged-out visitors. *** ### How does it work? The actor opens each account's public X profile and timeline — the same posts any visitor sees without logging in — and saves the pinned post, the newest posts on the profile, then the timeline page by page, then the account's highlighted posts if the timeline ends early, until `maxTweets` is reached or X shows no more posts. When X refuses a request, the actor waits, retries on a fresh connection and, if needed, through Apify residential proxy. It returns only publicly visible data, and the output fields stay the same from run to run, so your code and spreadsheets keep working when X changes its pages. *** ### How do you integrate Twitter Posts Scraper into your stack? It works with what you already use: call it from any language with the Apify API client, use MCP for AI agents, or connect a no-code automation tool. The Apify API client for Python, JavaScript, Go or any other supported language calls this actor by its technical name (shown on its Store page) and reads results back from its default dataset — works in any language that can make an HTTP request, including plain cURL. #### MCP integration for AI agents Add the Apify MCP server to your AI client with one command: ```bash claude mcp add --transport http apify "https://mcp.apify.com?tools=socialdataapi/twitter-posts-scraper" ``` This registers Twitter Posts Scraper as a tool your agent can call directly: the agent finds it through the server's search, adds it and runs it, then reads the posts back as JSON. Ask for "the last 50 posts of @NASA with their like counts" and it does the rest. It works in Claude Code, Cursor and Windsurf. #### No-code tools (n8n, Make, LangChain) **n8n:** add the Apify node and choose "Run an Actor", or use the HTTP Request node pointed at the run-sync endpoint with your token. Send new posts to a Google Sheet or Slack in the next node. **Make:** use the Apify app's "Run an Actor" module, then "Get Dataset Items" to pull the posts into the rest of your scenario. **LangChain:** install `langchain-apify` and wrap the actor with `ApifyActorsTool`, pointed at this actor's technical name from its Store page, so your agent can look up an account's posts. *** ### Is it legal to scrape Twitter posts? This actor returns only publicly available data — the posts and profile details any visitor sees on X without logging in — and never reads protected accounts, direct messages or account data. GDPR: scraping public data is generally permissible, but storing and using it must comply with local data-protection law. Consult legal counsel if your use case involves bulk storage of personal data. *** ### Frequently asked questions #### What fields does Twitter Posts Scraper return? It returns each post's ID, links, full text, likes, replies, reposts, quotes, bookmarks, language, reply and conversation details, media, entities, link card and place, plus the author's profile. See [the data fields section](#what-data-can-i-extract-with-twitter-posts-scraper) for the full list. #### Does Twitter Posts Scraper require an X account or login? No. Only public profiles and posts are read, and no credentials, cookies or API key are needed. #### How many posts can I extract per account? Up to `maxTweets` — at most 5,000 per account — and as many accounts as you list, as far as X shows posts to logged-out visitors: a few hundred to about 1,200 per account in our tests (timeline plus highlighted posts). Some accounts return fewer, for example about 100 for @nytimes. The log says when X has no more posts. Protected or suspended accounts return nothing. #### How fast does Twitter Posts Scraper run? A recorded run returned 500 posts for five accounts in 7.4 seconds, and another 20 posts for two accounts in 2.1 seconds. Every run reads live data, not a cache. #### What happens if an account does not exist or X does not answer? You get no rows for that account and are not charged for it; the run log and status name the account and the reason (not found, suspended or protected, or no answer from X). In code, group rows by `authorDisplay` to see how many each account returned. #### Can I scrape tweets from multiple accounts at once? Yes. List any number of accounts; they are read at the same time and each one gets up to `maxTweets` posts. #### Does Twitter Posts Scraper work with Claude, ChatGPT, and other AI agent tools? Yes — Twitter Posts Scraper (Apify Actor ID: socialdataapi/twitter-posts-scraper, https://apify.com/socialdataapi/twitter-posts-scraper) is available as an MCP tool. Register it in Claude Code, Cursor or Windsurf with `claude mcp add --transport http apify "https://mcp.apify.com?tools=socialdataapi/twitter-posts-scraper"`. Any agent that can call an HTTP endpoint can use the REST API directly. #### Does Twitter Posts Scraper return data in a format LLMs can use directly? Yes. Each post is typed, normalized JSON with consistent field names, so there is no HTML parsing and no selectors. Pass it straight to an LLM, index it into a vector store, or feed it to an agent tool — no transformation step. #### What happens when X changes its layout or updates its anti-bot system? The actor is maintained, and when X changes, it is updated so your calls keep returning the same JSON fields with no changes on your end. #### Can I use Twitter Posts Scraper without managing proxies or browser infrastructure? Yes. A default run connects directly and switches to Apify residential proxy only when X refuses a request, with retries handled on the actor's side, and no browser is needed. You send one request and get structured JSON back. #### Which Twitter post fields work best for AI training data and RAG indexing? For RAG, use `fullText`, `authorDisplay`, `createdAt` and the hashtags and mentions in `entities`. For training data and analytics, use the engagement counts, `lang`, `isReply` / `isQuote` and the author's follower counts. All fields are plain values, lists and objects, so no normalization is needed — a Twitter scraper output ready for any pipeline. *** ### Your feedback Found a bug or missing a field? Let us know in the actor's Issues tab. Every report is read, and fixes ship as new versions of the actor. # Actor input Schema ## `startUrls` (type: `array`): One X account per line. Accepted forms: username (elonmusk), @username, a profile URL (https://x.com/elonmusk or https://twitter.com/elonmusk) or a post URL (the post's author is used). Repeated accounts are read once. ## `maxTweets` (type: `integer`): Most posts saved per account (1–5000). The pinned post comes first, then the newest posts, then the rest of the timeline; when X ends the logged-out timeline early, the account's highlighted posts follow. X limits what logged-out visitors see, so some accounts return fewer posts than asked (for example, very large accounts show one page of about 100 popular posts on the timeline). Free plan: up to 20 posts per run (all accounts together) and 200 per month; paid plans have no limit. ## `proxyConfiguration` (type: `object`): Leave it off for most runs: the first request goes direct and, when X refuses it, retries go through Apify residential proxy automatically. If you select a proxy, it is used for every request. ## Actor input object example ```json { "startUrls": [ "elonmusk" ], "maxTweets": 10, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `results` (type: `string`): All scraped posts in the Actor's default dataset. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ "elonmusk" ], "proxyConfiguration": { "useApifyProxy": false } }; // Run the Actor and wait for it to finish const run = await client.actor("socialdataapi/twitter-posts-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": ["elonmusk"], "proxyConfiguration": { "useApifyProxy": False }, } # Run the Actor and wait for it to finish run = client.actor("socialdataapi/twitter-posts-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ "elonmusk" ], "proxyConfiguration": { "useApifyProxy": false } }' | apify call socialdataapi/twitter-posts-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,socialdataapi/twitter-posts-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/5aLrbThqGeh4cp3Fh/builds/lLt4cVm4G6AQC8O8x/openapi.json