Twitter X Tweets Profiles Scraper

Twitter X Tweets Profiles Scraper

Twitter X Tweets Profiles Scraper is a fast, cloud-based data extractor that collects public tweets and profile context from X (Twitter) at scale — including text, timestamps, engagement, media, links, and profile metadata. It solves the pain of API limits and manual copy‑paste by letting you scrape Twitter without API and export Twitter tweets to CSV or JSON for analytics and automation. Built for marketers, developers, data analysts, and researchers, this Twitter profile scraper and X tweets scraper powers social listening, competitor analysis, and lead research — reliably and at scale. 🚀

What data / output can you get?

Below are the exact fields this actor pushes to the Apify dataset (one item per tweet). These are structured, analytics‑ready outputs that work great for “Scrape Twitter profiles and tweets,” “Twitter search results scraper,” and “Bulk tweet scraper” workflows.

Data type	Description	Example value
id	Tweet ID (string)	"1519480761749016577"
url	Canonical tweet URL	"https://x.com/elonmusk/status/1519480761749016577"
verified	Whether the author is verified	true
username	Screen name with @ prefix	"@elonmusk"
fullname	Display name of the author	"Elon Musk"
timestamp	Tweet creation time (UTC)	"Thu Apr 28 00:56:58 +0000 2022"
text	Full tweet text	"Next I'm buying Coca-Cola to put the cocaine back in"
links	Array of expanded URLs found in the tweet/card/media	[]
isPinned	Whether the tweet is pinned on profile	false
isQuote	Whether the tweet is a quote	false
isRetweet	Whether the tweet is a retweet	false
isReply	Whether the tweet is a reply	false
likes	Like count	4289224
replies	Reply count	170050
retweets	Retweet count	594428
quotes	Quote count	167104
media	Array of media objects { type, url }	[{"type":"photo","url":"https://pbs.twimg.com/media/..."}]
tweetUserId	Author user ID string from tweet legacy	"44196397"
user.avatar	Author avatar URL	"https://pbs.twimg.com/profile_images/..."
user.username	Author screen name (no @)	"elonmusk"
user.userFullName	Author display name	"Elon Musk"
user.description	Bio text	""
user.location	Profile location	null
user.website	Expanded website URL	null
user.joinDate	Account creation time (UTC)	"Tue Jun 02 20:12:29 +0000 2009"
user.verified	Whether the author is verified	true
user.totalLikes	Likes (profile total)	182734
user.totalTweets	Tweets (profile total)	89153
user.totalFollowing	Following count	1226
user.totalFollowers	Followers count	229037864
user.url	Profile website display URL	null
user.userId	Author user ID	"44196397"
user.totalMediaCount	Media count	4225

Notes:

• The user object is omitted when includeUserInfo is set to false.
• Media includes photos and best‑available MP4 URLs for videos/GIFs where applicable.
• Export datasets from Apify as JSON, CSV, Excel, XML, or HTML Table — perfect for “Export Twitter tweets to CSV” workflows.

Key features

• 🔀 Multi‑mode targeting (profiles & keywords) — Provide profile URLs, usernames (with or without @), numeric user IDs, or keyword queries. Ideal for an X profile scraper and Twitter search results scraper in one.
• 🧵 Reply filtering — Use withReplies to include or exclude replies for cleaner timelines or full conversations with this Twitter tweets scraper.
• 👤 Optional user metadata — Toggle includeUserInfo to attach rich profile details (bio, followers/following, join date, website, avatar) with each tweet — or disable for lighter payloads.
• 🖼️ Media & link extraction — Captures photos, best MP4 video links, and expanded URLs across entities, cards, and note tweets for complete context.
• 📈 Engagement metrics — Collects likes, replies, retweets, and quotes per tweet for performance benchmarking and competitor analysis.
• 🛡️ Smart proxy fallback — Automatically attempts Apify Residential proxy if requests are blocked, improving resilience for global data collection.
• ☁️ Cloud‑native & export‑ready — One‑click runs on Apify with dataset exports for BI tools and pipelines.
• 👩‍💻 Developer‑friendly Python actor — Built with apify and Playwright; integrate with your Twitter scraper Python pipelines and ETL jobs easily.

How to use Twitter X Tweets Profiles Scraper - step by step

1. Sign in to Apify
   Create a free account or log in to access your dashboard.

2. Open the actor
   Search for “Twitter X Tweets Profiles Scraper” on Apify and click Try.

3. Add your inputs
   In startUrls, paste your targets — for example:

   • Profile URL: https://x.com/username
   • Username/handle: username or @username
   • Keyword query: AI trends, earnings call, product launch

4. Configure options

   • maxTweets: 1–100 per input (default 10)
   • withReplies: include replies or filter them out
   • includeUserInfo: attach/detach profile details per tweet
   • sortOrder: choose "newest", "oldest", or "popular" (input available)
   • proxyConfiguration: use Apify Proxy and let the actor auto‑fallback when needed

5. Run the actor
   Click Start. The actor launches a headless browser, captures public timelines or keyword search results, and streams items to the dataset.

6. Monitor progress
   Watch logs in real time. If requests are blocked, the actor will attempt Residential proxy fallback automatically.

7. Download results
   Open the Run dataset and export as JSON, CSV, or Excel for analytics, dashboards, or enrichment flows.

Pro tip: For large-scale “Twitter search results scraper” jobs, set includeUserInfo to false to speed up runs; re‑enrich profiles later only for accounts you care about.

Use cases

Use case name	Description
Social listening for marketing	Track tweets from target profiles and keyword searches to measure engagement and sentiment across campaigns.
Competitor timeline analysis	Compare posting cadence, engagement metrics, and media usage using a reliable X profile scraper + X tweets scraper combo.
Hashtag/keyword monitoring	Monitor topics and events with a Twitter data extractor workflow; export Twitter tweets to CSV for BI.
Lead research on public profiles	Enrich outreach lists by pulling bios, websites, and follower counts with a Twitter user bios extractor.
Academic & NLP datasets	Collect structured tweet text, timestamps, and metadata for language modeling and topic analysis.
Product & PR monitoring	Catch mentions and quotes around announcements using a real-time Twitter scraper pipeline.
Data engineering pipelines	Run scheduled extractions on Apify and feed JSON/CSV to warehouses and dashboards.

Why choose Twitter X Tweets Profiles Scraper?

A production‑ready Twitter data extractor built for precision, resilience, and scale — not a fragile browser extension.

• 🎯 Accurate, structured output consistent across profiles and searches
• ⚡ Scales from quick pulls to high‑volume runs (clamped to 100 tweets per input)
• 👩‍💻 Developer‑friendly JSON with stable field names for pipelines and APIs
• 🖼️ Full media and link extraction for richer analytics
• 🛡️ Automatic proxy fallback to Residential when requests are blocked
• 🔓 Public data only — no login, no API keys required
• 💾 Export‑ready datasets (JSON, CSV, Excel, XML, HTML Table)

In short, a dependable X profile scraper and tweet extractor that delivers clean, analytics‑ready records at scale.

Is it legal / ethical to use Twitter X Tweets Profiles Scraper?

Yes — when used responsibly. This actor accesses only publicly available information on X (Twitter) and does not log into private accounts or scrape protected content.

Guidelines for compliant use:

• Scrape public pages and content only
• Review and respect Twitter/X Terms of Service
• Comply with data privacy regulations (e.g., GDPR, CCPA)
• Use outputs for legitimate analytics, research, and monitoring — not spam or abuse

Always consult your legal team for edge cases or jurisdiction‑specific requirements.

Input parameters & output format

Example JSON input

{
  "startUrls": [
    "https://x.com/elonmusk",
    "mrbeast",
    "AI trends"
  ],
  "sortOrder": "newest",
  "maxTweets": 10,
  "withReplies": true,
  "includeUserInfo": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

Input parameter reference

Field	Type	Required	Default	Description
startUrls	array (stringList)	Yes	—	List of Twitter/X profile URLs (e.g., https://x.com/username), usernames (e.g., username or @username), or search keywords.
sortOrder	string ("newest"	"oldest"	"popular")	No
maxTweets	integer (1–100)	No	10	Maximum number of tweets to scrape per profile/keyword (1–100).
withReplies	boolean	No	true	Include reply tweets in the results.
includeUserInfo	boolean	No	true	Include detailed user profile information with each tweet.
proxyConfiguration	object	No	{"useApifyProxy": false}	Proxy settings. Actor will automatically fallback to datacenter then residential proxy if requests are blocked.

Notes:

• Results are clamped to a maximum of 100 tweets per input per run.
• Keyword searches are supported by adding plain‑text queries to startUrls.

Example JSON output

{
  "id": "1519480761749016577",
  "url": "https://x.com/elonmusk/status/1519480761749016577",
  "verified": true,
  "username": "@elonmusk",
  "fullname": "Elon Musk",
  "timestamp": "Thu Apr 28 00:56:58 +0000 2022",
  "text": "Next I'm buying Coca-Cola to put the cocaine back in",
  "links": [],
  "isPinned": false,
  "isQuote": false,
  "isRetweet": false,
  "isReply": false,
  "likes": 4289224,
  "replies": 170050,
  "retweets": 594428,
  "quotes": 167104,
  "user": {
    "avatar": "https://pbs.twimg.com/profile_images/...",
    "username": "elonmusk",
    "userFullName": "Elon Musk",
    "description": "",
    "location": null,
    "website": null,
    "joinDate": "Tue Jun 02 20:12:29 +0000 2009",
    "verified": true,
    "totalLikes": 182734,
    "totalTweets": 89153,
    "totalFollowing": 1226,
    "totalFollowers": 229037864,
    "url": null,
    "userId": "44196397",
    "totalMediaCount": 4225
  },
  "media": [
    {
      "type": "photo",
      "url": "https://pbs.twimg.com/media/..."
    }
  ],
  "tweetUserId": "44196397"
}

Field notes:

• user is omitted when includeUserInfo is false.
• Some fields can be null when not present on the public profile (e.g., location, website).

FAQ

Is the Twitter X Tweets Profiles Scraper free?

You can start on Apify with free credits for small projects, then upgrade as needed. This actor also offers a monthly plan with trial minutes on Apify.

Does it need API keys or login?

No. It works without API keys and does not log in. It collects data from publicly accessible endpoints.

Can it scrape followers?

It does not extract follower lists. It returns follower metrics via the user object (e.g., totalFollowers, totalFollowing) when includeUserInfo is true.

Can I scrape keywords or hashtags?

Yes. Add keyword queries (e.g., “AI trends”) to startUrls to run a Twitter search results scraper. Hashtags can be included in your query string.

How many tweets can I export per input?

Up to 100 tweets per input per run. Control this with maxTweets (1–100). You can run repeated jobs to build larger histories over time.

What happens if my requests are blocked?

The actor includes a smart proxy fallback. If requests are blocked or rejected, it attempts to switch to Apify Residential proxy automatically.

What data formats can I export?

You can export datasets as JSON, CSV, Excel, XML, or HTML Table directly from Apify — ideal when you need to export Twitter tweets to CSV for BI.

Does sortOrder change the results?

The sortOrder input is available with options "newest", "oldest", and "popular". Timeline results generally follow public ordering; use maxTweets and withReplies to tailor scope.

Closing CTA / Final thoughts

Twitter X Tweets Profiles Scraper is built for fast, reliable collection of public tweets and profile context at scale. It delivers clean JSON records with engagement, links, media, and optional user metadata — ready for analytics and automation. Whether you’re a marketer, developer, researcher, or data analyst, this X tweets scraper and Twitter profile scraper helps you build social listening, brand monitoring, and competitor intelligence pipelines without API limits. Developers can integrate the Apify dataset output directly into Python ETL jobs and dashboards. Start extracting smarter, structured Twitter/X data today.