Twitter X Tweets Profiles Scraper

Twitter X Tweets Profiles Scraper

Twitter X Tweets Profiles Scraper is a cloud-executed Apify actor that collects public tweets, profile metadata, media, and engagement metrics from X (Twitter) at scale — without API keys. It replaces manual copy-paste and brittle scripts with a reliable Twitter tweets scraper and X profiles scraper for social listening, analytics, and research. Built for marketers, developers, data analysts, and researchers, this Twitter profile scraper and X tweets scraper lets you target profile URLs, @usernames, and keyword searches to power continuous audience insights, trend tracking, and competitive intelligence at scale. 🚀

What data / output can you get?

Below are real output fields this actor pushes to the dataset, with examples taken from public tweets.

Data type	Description	Example value
id	Tweet identifier (string)	1519480761749016577
url	Canonical tweet URL	https://x.com/elonmusk/status/1519480761749016577
username	Author handle (with @)	@elonmusk
fullname	Author’s display name	Elon Musk
verified	Author’s verification flag	true
timestamp	Tweet creation time (RFC-like)	Thu Apr 28 00:56:58 +0000 2022
text	Full tweet text	Next I'm buying Coca-Cola to put the cocaine back in
links	Unique expanded URLs referenced in the tweet	[]
isPinned	Whether the tweet is pinned	false
isQuote	Whether it’s a quote tweet	false
isRetweet	Whether it’s a retweet	false
isReply	Whether it’s a reply	false
likes	Favorite count	4289224
replies	Reply count	170050
retweets	Retweet count	594428
quotes	Quote count	167104
media	Extracted media with best video URLs when available	[{ "type": "photo", "url": "https://pbs.twimg.com/media/..." }]
user	Detailed user object (included when includeUserInfo=true)	See JSON example below
tweetUserId	Author user ID associated with the tweet	44196397

Notes:

• Bonus outputs include media (photos/GIFs/best-available MP4 video variants) and a rich user object with profile metadata (bio/description, avatar, location, website, joinDate, follower/following/tweet counts, etc.).
• Export your dataset to JSON, CSV, or Excel directly from Apify — perfect for a Twitter to CSV exporter workflow or downstream BI integrations.

Key features

• 🛡️ Smart proxy fallback
  Automatically detects blocks and falls back to Apify Residential proxy with retries to keep scraping uninterrupted.

• 🧭 Multi-mode targeting
  Accepts profile URLs (x.com/username), plain @usernames, numeric user IDs, and keyword searches — ideal for tweet monitoring and topic tracking with a Twitter search results scraper approach.

• 🔁 Reply inclusion control
  Use withReplies to include or exclude replies in your results and manage dataset noise for Twitter replies scraper use cases.

• 🎥 Rich media extraction
  Collects photos and selects the best available MP4 variant for videos where present.

• 👤 Toggle user metadata
  Include or exclude the user object per tweet via includeUserInfo to reduce payload or enrich context for a Twitter user profile data extractor workflow.

• ⚡ Real-time scraping via headless browser
  Playwright-powered session captures required tokens and fetches fresh data from public endpoints — a robust Twitter data scraper without API keys.

• ☁️ Cloud-native & scheduling
  Runs on Apify infrastructure. Schedule jobs and plug the dataset into your analytics workflows.

• 🧑‍💻 Developer-friendly
  Use the Apify API and dataset exports to integrate with pipelines, notebooks, or BI tools — a practical X data scraper for engineering teams.

• 📈 Scalable by design
  Configure maxTweets per input (1–100). The actor deduplicates and caps up to 100 results per input for reliable throughput.

• ✅ Production-ready reliability
  Automatic token collection, pagination via GraphQL/search timelines, and incremental dataset writes ensure robust runs.

How to use Twitter X Tweets Profiles Scraper - step by step

1. Create or log in to your Apify account
   Access the actor from the Apify platform.

2. Open the actor and prepare inputs
   In startUrls, enter targets as:

   • Profile URLs: https://x.com/username
   • Usernames: username or @username
   • Keywords: any phrase like AI research

3. Configure scraping options

   • maxTweets: 1–100 per input
   • withReplies: include or exclude replies
   • includeUserInfo: include the detailed user object per tweet
   • sortOrder: newest | oldest | popular (accepted as input)
   • proxyConfiguration: choose direct or Apify Proxy

4. Start the run
   Click Start. The actor launches a headless browser, captures session tokens, and begins scraping your targets.

5. Monitor progress
   View logs in real time. The actor pushes tweets incrementally to the dataset as they’re found.

6. Review results in the Dataset
   Open the run’s Dataset to preview structured tweets, media, and (optionally) user info.

7. Export your data
   Export the dataset as JSON, CSV, or Excel for analysis, dashboards, or ingestion into your pipelines.

Pro tip: To minimize payload size, set includeUserInfo to false. For ongoing monitoring, schedule the actor and fetch results via the Apify API into your data warehouse or BI tool.

Use cases

Use case name	Description
Social media analytics for brands	Track competitor and influencer timelines and quantify engagement with likes, replies, retweets, and quotes using a reliable Twitter data extractor.
Topic/keyword monitoring for research	Collect recent public tweets for keyword phrases to analyze trends, sentiment, and narratives with an X tweets scraper workflow.
Content curation for marketing teams	Aggregate tweets with media and links to power newsletters, reports, or editorial workflows.
Audience and profile insights	Enrich tweet data with user metadata (followers, following, tweets, joinDate) when includeUserInfo=true via a Twitter user profile data extractor approach.
Academic studies & NLP datasets	Build reproducible corpora from public tweets for text mining, sentiment analysis, or topic modeling.
Developer API pipelines	Automate runs and export datasets to JSON/CSV/Excel for ETL into analytics stacks and dashboards — a practical Twitter to CSV exporter.
Competitive posting analysis	Compare posting frequency and engagement across multiple public profiles with an X profiles scraper routine.

Why choose Twitter X Tweets Profiles Scraper?

This actor delivers structured, reliable X (Twitter) data with automation-first design and cloud reliability.

• 🎯 Accurate, structured fields out of the box — no manual parsing required
• 🚀 Scales across multiple inputs with configurable maxTweets per item (up to 100)
• 🧑‍💻 Easy developer access via Apify datasets and API for smooth integrations
• 🔐 No API keys required — works with public endpoints and session tokens
• 🛡️ Built-in resilience with automatic proxy fallback to keep runs flowing
• 💸 Cost-effective vs. maintaining custom scripts and brittle extensions
• ⚙️ Cloud-native automation — schedule runs and export results for repeatable pipelines

In short: a production-ready tweet and profile data scraper — not a fragile browser add-on.

Is it legal / ethical to use Twitter X Tweets Profiles Scraper?

Yes — when used responsibly. This actor accesses publicly available X (Twitter) content and does not log in or access private/protected profiles.

Guidelines for compliant use:

• Scrape only public data and respect Twitter/X’s Terms of Service
• Ensure alignment with data protection laws (e.g., GDPR, CCPA)
• Avoid sensitive use cases; do not spam or misuse collected data
• Consult your legal team for edge cases and jurisdictional specifics

Input parameters & output format

Example input JSON

{
  "startUrls": [
    "https://x.com/elonmusk",
    "mrbeast",
    "AI research"
  ],
  "maxTweets": 10,
  "withReplies": true,
  "includeUserInfo": true,
  "sortOrder": "newest",
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

Input parameters

• startUrls (array, required): List of Twitter/X profile URLs (e.g., https://x.com/username), usernames (e.g., username or @username), or search keywords. Default: none (required). UI prefill: ["https://x.com/elonmusk", "mrbeast"].
• sortOrder (string): How to sort the tweets. Allowed values: newest, oldest, popular. Default: "newest".
• maxTweets (integer): Maximum number of tweets to scrape per profile/keyword (1–100). Default: 10.
• withReplies (boolean): Include reply tweets in the results. Default: true.
• includeUserInfo (boolean): Include detailed user profile information with each tweet. Default: true.
• proxyConfiguration (object): Proxy settings. Actor will automatically fallback to datacenter then residential proxy if requests are blocked. Default: not set (UI prefill shown: { "useApifyProxy": false }).

Example output JSON (single item)

{
  "id": "1519480761749016577",
  "url": "https://x.com/elonmusk/status/1519480761749016577",
  "verified": true,
  "username": "@elonmusk",
  "fullname": "Elon Musk",
  "timestamp": "Thu Apr 28 00:56:58 +0000 2022",
  "text": "Next I'm buying Coca-Cola to put the cocaine back in",
  "links": [],
  "isPinned": false,
  "isQuote": false,
  "isRetweet": false,
  "isReply": false,
  "likes": 4289224,
  "replies": 170050,
  "retweets": 594428,
  "quotes": 167104,
  "user": {
    "avatar": "https://pbs.twimg.com/profile_images/...",
    "username": "elonmusk",
    "userFullName": "Elon Musk",
    "description": "",
    "location": null,
    "website": null,
    "joinDate": "Tue Jun 02 20:12:29 +0000 2009",
    "verified": true,
    "totalLikes": 182734,
    "totalTweets": 89153,
    "totalFollowing": 1226,
    "totalFollowers": 229037864,
    "url": null,
    "userId": "44196397",
    "totalMediaCount": 4225
  },
  "media": [
    {
      "type": "photo",
      "url": "https://pbs.twimg.com/media/..."
    }
  ],
  "tweetUserId": "44196397"
}

Notes:

• When includeUserInfo is set to false, the user object is omitted from each pushed item to reduce payload size.
• The actor pushes items incrementally to the dataset during the run.

FAQ

Is the Twitter X Tweets Profiles Scraper free?

You can start with Apify trial minutes, then upgrade as needed. The actor runs on Apify’s cloud with pay-as-you-go resources and available subscription options.

Does it need API keys or authentication?

No. It works without API keys by collecting session tokens from public browsing and accessing publicly available endpoints — a true Twitter data scraper without API.

Can it scrape keywords from search results?

Yes. Add any keyword phrase with spaces (e.g., "AI research") to startUrls and the actor will fetch public search results and push matching tweets to the dataset as a Twitter search results scraper.

Can it scrape followers?

It does not extract follower lists. However, when includeUserInfo=true, each tweet includes user.totalFollowers and other profile metrics provided by the public response.

How many tweets can it collect per input?

Configure maxTweets from 1 to 100 per input. The actor deduplicates and caps results at up to 100 tweets per profile/keyword in a single run.

Does it scrape private or protected accounts?

No. It only accesses publicly available pages and content. Private/protected profiles and login-required endpoints are not scraped.

What export formats are supported?

You can export the Apify dataset as JSON, CSV, or Excel for downstream analysis, dashboards, and integrations — ideal for a Twitter to CSV exporter workflow.

What happens if requests are blocked?

The actor automatically attempts a proxy fallback, switching to Apify Residential proxy with retries to maintain continuity.

Can I automate and integrate this into my pipeline?

Yes. Schedule runs on Apify and pull results via the Apify API. This is ideal for recurring analytics and data pipelines across marketing, research, and engineering teams.

Can I collect media URLs (photos/videos)?

Yes. The media array includes photos and, when available, selects the best video/mp4 variant — useful for content curation and media analysis.

Closing CTA / Final thoughts

Twitter X Tweets Profiles Scraper is built for reliable, structured tweet and profile data extraction at scale. With reply inclusion control, rich media capture, optional user metadata, and resilient proxy fallback, it delivers analysis-ready datasets for marketers, developers, data analysts, and researchers. Use the Apify API to automate exports to JSON/CSV/Excel and wire results into your BI or NLP pipelines. Start extracting smarter, fresher X (Twitter) data — on your schedule and at your scale.