Twitter X Posts Scraper

Twitter X Posts Scraper

The Twitter X Posts Scraper is a fast, reliable X posts scraper tool that collects recent posts from X (Twitter) profiles and usernames — including text, timestamps, engagement metrics, hashtags, and media links — in structured records you can export to CSV or JSON. It solves the manual copy-paste problem by automating data collection from profiles so marketers, developers, data analysts, and researchers can run scalable social listening, competitor tracking, and sentiment or marketing research. With per-user limits, sorting, and optional proxy fallback, this Twitter posts scraper and X tweet scraper scales from quick checks to bulk monitoring across multiple profiles.

What data / output can you get?

Below are the exact output fields the actor saves to the Apify dataset, with examples. You can export results to CSV or JSON.

Data type	Description	Example value
id	Post (tweet) ID	1988877569597260072
url	Public post URL	https://x.com/elonmusk/status/1988877569597260072
user_posted	Username (handle) of the author	elonmusk
name	Display name of the author	Elon Musk
description	Full text of the post	Long press on any image to turn it into a video...
date_posted	ISO-8601 timestamp (UTC)	2025-11-13T07:52:18.000Z
likes	Like count	1729
replies	Reply count	554
reposts	Repost/retweet count	368
quotes	Quote count	38
views	View/impression count	1399060
bookmarks	Bookmark count	213
is_verified	Blue verification status	true
followers	Author follower count	229031060
following	Author following count	1226
posts_count	Total posts by the author	89153
profile_image_link	Profile image URL	https://pbs.twimg.com/profile_images/..._normal.jpg
biography	Bio/description from profile	Building rockets and cars...
hashtags	List of hashtags found in the post	["AI","launch"]
tagged_users	Mentioned usernames	["tetsuoai"]
photos	First photo URL if present	https://pbs.twimg.com/media/XXXXX.jpg
videos	Video URLs (highest bitrate first)	["https://video.twimg.com/.../file.mp4"]
quoted_post	Object with data about quoted post (if any)	{"post_id":"...", "description":"..."}
external_url	Website from profile (if present)	https://tesla.com
input	Object with the actor-constructed input URL	{"url":"https://x.com/elonmusk/status/1988877569597260072/"}

Note: Media fields include a single “photos” URL (first photo) and an array of “videos” URLs. “quoted_post” provides context if the post quotes another post.

Key features

• ⚡ Fast, reliable profile post extraction
  Scrapes posts from X profiles/usernames with text, timestamps, engagement metrics, hashtags, and media links for analysis-ready datasets.

• 🧩 Bulk X posts scraper (multiple inputs)
  Add multiple usernames or profile URLs to startUrls to process profiles in batches — ideal for automated Twitter posts scraper workflows.

• 🔎 Sortable results (recent, oldest, popular)
  Control result order via sortOrder to prioritize latest posts, earliest posts, or most-liked posts for downstream analysis.

• 🎯 Per-user limits with maxTweets
  Define how many posts to collect per profile to manage cost, speed, and coverage.

• 🏷️ Hashtag and mention capture
  Works as a Twitter hashtag scraper for individual posts by extracting hashtags and tagged users from each post.

• 🔐 Optional Apify proxy with smart fallback
  If requests fail, the actor can step through datacenter and residential proxy options for more stable runs.

• 📦 Structured exports to CSV/JSON
  Clean, consistent output that’s ready for BI tools, pipelines, or simple downloads.

• 👥 X user posts scraper for analysts and growth teams
  Designed for social listening, competitor tracking, and marketing research — from individual profiles to bulk monitoring.

How to use Twitter X Posts Scraper - step by step

1. Create or log in to your Apify account
   Access the actor from your Apify dashboard.

2. Open the actor and add inputs
   In the startUrls field, enter one value per line using any of these formats:

   • username (e.g., elonmusk)
   • @username (e.g., @jack)
   • full profile URL (e.g., https://x.com/BarackObama or https://twitter.com/BarackObama)

3. Important: Do not add tweet/status URLs
   The actor skips tweet URLs and expects profile identifiers only. Provide usernames or user IDs.

4. Choose sortOrder
   Select one of:

   • recent (newest first)
   • oldest (oldest first)
   • popular (most liked first)

5. Set maxTweets
   Choose how many posts to collect per input (1–100). Higher values take longer.

6. Configure proxy (optional)
   Leave proxyConfiguration.useApifyProxy disabled by default. Enable Apify Proxy if you encounter blocks, timeouts, or empty results. The actor can fall back across proxy types for stability.

7. Run the actor and monitor progress
   The actor resolves each username, fetches posts, sorts them per your settings, and saves them to the dataset as it goes.

8. Export results
   Open the Dataset tab to download data as JSON or CSV for your pipelines, dashboards, or reports.

Pro Tip: Add multiple profiles to startUrls to build a lightweight X data extractor for posts that powers recurring social listening or competitor tracking workflows.

Use cases

Use case name	Description
Social listening for marketing	Track brand and competitor posts to analyze engagement patterns, hashtags, and audience reactions across profiles.
Competitor tracking for growth teams	Monitor rivals’ posting cadences and top-performing content using sortOrder=popular.
Sentiment and trend analysis	Export CSV/JSON for NLP pipelines to assess sentiment, themes, and virality across recent posts.
Campaign performance benchmarking	Compare likes, replies, quotes, and reposts over time per profile to fine-tune content strategy.
Research & journalism datasets	Build structured corpora of profile posts with timestamps and media links for transparent reporting.
Data enrichment for dashboards	Feed clean, structured post data into BI tools for automated reporting and alerts.
Bulk monitoring for agencies	Use the Bulk X posts scraper capability to track multiple clients’ profiles at once with consistent output fields.

Why choose Twitter X Posts Scraper?

This Twitter post extractor focuses on accurate, structured profile post data with scalable runs and clean exports.

• ✅ Accuracy-first output: Captures post text, timestamps, engagement, hashtags, media links, and author metadata in consistent records.
• 📈 Built for scale: Process multiple profiles in a single run with per-user limits and sorting for fast insights.
• 🧠 Smarter stability: Optional Apify proxy with fallback across datacenter and residential options when needed.
• 🧰 Developer- and analyst-friendly: Clear JSON/CSV exports for pipelines, notebooks, and BI tools.
• 💸 Efficient & focused: Only the fields you need for social listening, competitor tracking, and marketing research — no noisy extras.
• 🔄 Strong sorting controls: Recent/oldest/popular sortOrder to match your analysis workflow.
• 🐦 Better than browser extensions: Production-ready infrastructure and dataset exports — no flaky manual scraping.

Is it legal / ethical to use Twitter X Posts Scraper?

Yes, when used responsibly. This actor is designed to collect publicly available profile post data. As with any web data collection:

• Only collect and use public information you’re allowed to access.
• Review and respect X (Twitter) terms of service and your local regulations.
• Avoid abusive request patterns; enable proxies if you encounter rate limits or blocks.
• Do not attempt to access private profiles or protected data.
• Consult your legal team for specific compliance needs.

Input parameters & output format

Example JSON input

{
  "startUrls": [
    "elonmusk",
    "https://x.com/BarackObama",
    "@username"
  ],
  "sortOrder": "recent",
  "maxTweets": 10,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

Input parameters

• startUrls (array, required)
  Description: Twitter URLs, usernames, or keywords to scrape (add one value per line). Examples include profile URLs (https://x.com/username or https://twitter.com/username) and usernames with or without @.
  Default: none (prefill example: "elonmusk")

• sortOrder (string, optional)
  Description: How to order scraped tweets before saving. Options: recent (newest first), oldest (oldest first), popular (most liked first).
  Default: "recent"

• maxTweets (integer, optional)
  Description: Upper limit of tweets to collect per profile/input. Higher values take longer. Allowed range: 1–100.
  Default: 10

• proxyConfiguration (object, optional)
  Description: Apify proxy settings for Twitter/X. Default is no Apify proxy. The actor can step through fallback proxies if requests fail (datacenter, then residential retries). Enable when you see blocks, timeouts, or empty results.
  Default: {"useApifyProxy": false}

Example JSON output

[
  {
    "biography": "Building rockets and cars...",
    "bookmarks": 213,
    "date_posted": "2025-11-13T07:52:18.000Z",
    "description": "@tetsuoai Long press on any image to turn it into a video in less than 30 seconds https://t.co/Nsp7Ba0flp",
    "external_url": null,
    "followers": 229031060,
    "following": 1226,
    "hashtags": null,
    "id": "1988877569597260072",
    "input": {
      "url": "https://x.com/elonmusk/status/1988877569597260072/"
    },
    "is_verified": true,
    "likes": 1729,
    "name": "Elon Musk",
    "photos": null,
    "posts_count": 89153,
    "profile_image_link": "https://pbs.twimg.com/profile_images/1983681414370619392/oTT3nm5Z_normal.jpg",
    "quoted_post": {
      "data_posted": null,
      "description": null,
      "post_id": null,
      "profile_id": null,
      "profile_name": null,
      "url": null,
      "videos": null
    },
    "quotes": 38,
    "replies": 554,
    "reposts": 368,
    "tagged_users": ["tetsuoai"],
    "url": "https://x.com/elonmusk/status/1988877569597260072",
    "user_posted": "elonmusk",
    "videos": [
      "https://video.twimg.com/amplify_video/1988877511368019968/vid/avc1/576x856/34pcJSQSXqqM4JRQ.mp4?tag=23"
    ],
    "views": 1399060
  }
]

Notes:

• Tweet/status URLs as inputs are skipped. Provide usernames or user IDs/profile URLs in startUrls.
• Some fields may be null if not present in the source (e.g., media fields, external_url, hashtags).

FAQ

Can I use the Twitter X Posts Scraper without coding?

Yes. You can run this X posts scraper tool entirely in the Apify UI by pasting usernames or profile URLs, choosing options, and exporting CSV/JSON.

What input formats are supported?

Provide usernames (elonmusk), @usernames (@jack), or full profile URLs (https://x.com/BarackObama or https://twitter.com/BarackObama) in startUrls. Tweet/status URLs are skipped.

Can I sort results by popularity or recency?

Yes. Set sortOrder to recent, oldest, or popular to control whether newest, oldest, or most-liked posts appear first.

How many posts can I get per profile?

Use maxTweets to set the upper limit per profile. Allowed range is 1–100, with 10 as the default.

Does it extract media and hashtags?

Yes. Each record can include a first photo URL (photos), an array of video URLs (videos), and hashtags and tagged_users extracted from the post content.

When should I enable proxies?

Enable proxyConfiguration when you see blocks, timeouts, or empty results. The actor can step through datacenter and residential fallback options to improve stability.

Is this a Twitter hashtag scraper or X thread scraper?

This actor fetches posts from user profiles and extracts hashtags per post. It is not a hashtag or search results scraper and does not take tweet/status URLs as inputs.

What export formats are available?

You can download results from the Apify dataset in JSON or CSV to feed analytics, enrichment, or reporting workflows.

Closing CTA / Final thoughts

The Twitter X Posts Scraper is built to extract structured posts from X profiles at scale for social listening, competitor tracking, and research. With clean JSON/CSV output, per-user limits, sort controls, and optional proxy fallback, it helps marketers, developers, analysts, and researchers automate profile monitoring with confidence. Run it from the Apify UI for a Twitter posts scraper without coding, or export datasets to power your analytics and automation pipelines. Start extracting smarter profile insights today.