X (Twitter) Real-Time Data Scraper — Tweets, Profiles, Followers, Engagement & More

Turn X (Twitter) into a clean, structured data feed. Search tweets, pull profiles and timelines, map followers, find who retweeted or quoted a post, export lists, communities, and Spaces — and optionally your own notifications, analytics, and DMs. One Actor, 15 data sources, one predictable JSON schema.

No X API keys. No developer account. No browser automation to babysit. Just check the boxes for the data you want and press Start.

⚡ Real-time results • 🧩 15 features in one Actor • 🤖 LLM & MCP-ready JSON • 💸 Pay only for what you enable

---

Why this Actor

• Skip the official X API. No $100–$5,000/mo API tiers, no OAuth apps, no rate-limit math. Enable a feature and get data.
• 15 data sources, one schema. Every row carries a featureType tag and flat camelCase fields — so search results, profiles, and followers all slot into the same table, spreadsheet, or database.
• Built for scale. Streams results straight to the dataset as they arrive. Runs comfortably on 256 MB and scrapes up to 100,000 items per run out of the box (raise it if you need more).
• Never loses a run to one bad input. A private account or deleted tweet becomes a tidy unavailable row — the rest of your run keeps going.
• Automation-native. Pipe results to Google Sheets, Airtable, a webhook, your database, or an AI agent via the Apify API and MCP.

---

What you can collect

#	Feature	Turn on with	You get
1	Keyword & advanced search (on by default)	enableSearch	Tweets or people matching keywords, hashtags, cashtags, or full X operators
2	User profiles	enableUserProfiles	Follower/following counts, bio, verification, location, join date
3	User tweets / timelines	enableUserTweets	Everything an account posts (with or without replies)
4	Tweet details + replies	enableTweetDetails	Full metrics for one or many tweets, plus entire reply threads
5	Engagers	enableEngagers	Who retweeted and who quoted a tweet
6	Followers & following	enableNetwork	Followers, following, verified followers, subscriptions
7	Friendship check	enableFriendship	Does user A follow user B? Mutual?
8	Lists	enableLists	List metadata, tweets, members, followers
9	Communities	enableCommunities	Community details, tweets, members, and discovery search
10	Spaces	enableSpaces	Space metadata, host, speakers, and optional audio stream URL
11	Translation	enableTranslate	Any tweet translated into your target language
12	Scrape by URL	enableScrapeByUrl	Paste any X link — auto-detects tweet, profile, list, community, Space, or search
13	Your notifications 🔐	enableNotifications	Your account's notification timeline
14	Your analytics 🔐	enableAnalytics	Impressions, engagements, follower growth over time
15	Your DM inbox 🔐	enableDmInbox	Your conversations and messages

🔐 = uses your own X session — see Account data.

---

Use cases & outcomes

📈 Social listening & brand monitoring Track every mention of your brand, product, or campaign in real time. Search by keyword + language, filter by minimum likes, exclude retweets, and route hits to Slack or a dashboard.

🔍 Competitor & market intelligence Pull competitors' full timelines, watch their follower growth, and see who's amplifying them via retweets and quotes.

🎯 Lead generation & audience building Export the followers of an account (or a curated list), enrich each profile with follower counts and bios, and filter for accounts above a follower threshold — a ready-made prospect list.

🧠 Influencer & creator discovery Find people (not tweets) with People search, rank by followers and verification, and check verified-follower overlap between accounts.

📰 Research, journalism & OSINT Reconstruct full conversation threads, map quote/retweet networks, and archive lists, communities, and Spaces with canonical x.com URLs and ISO timestamps on every record.

🤖 AI agents & RAG pipelines Feed clean, flat JSON straight into an LLM. Stable featureType tags and predictable fields mean no brittle parsing — perfect for Apify MCP and agent tooling.

📊 Your own performance analytics Export your notifications, engagement time series, and DM history for personal analytics, backups, or CRM sync.

---

Quick start

1. Open the Actor in Apify Console and press Start. Search is on by default — you'll get tweets for the sample keyword immediately.
2. Type your own keywords, or check any of the other 14 features and fill in usernames, tweet IDs, or URLs.
3. Press Start and watch rows stream into the dataset.
4. Export as JSON / CSV / Excel, or pull via the Apify API, integrations, or MCP.

Try it — search only

{
  "searchTerms": ["AI agents", "LLM"],
  "searchType": "Latest",
  "searchMaxResultsPerQuery": 100
}

Followers → lead list

{
  "enableSearch": false,
  "enableNetwork": true,
  "networkUsernames": ["openai"],
  "networkType": "followers",
  "networkMinFollowers": 500,
  "maxItems": 5000
}

Profiles + full timelines

{
  "enableSearch": false,
  "enableUserProfiles": true,
  "profileUsernames": ["nasa", "spacex"],
  "enableUserTweets": true,
  "timelineUsernames": ["nasa", "spacex"],
  "includeReplies": false
}

Who engaged with a tweet

{
  "enableSearch": false,
  "enableEngagers": true,
  "engagerTweetUrls": ["https://x.com/nasa/status/1000000000000000001"],
  "engagerType": "both"
}

Paste any link

{
  "enableSearch": false,
  "enableScrapeByUrl": true,
  "scrapeUrls": [
    "https://x.com/nasa",
    "https://x.com/i/status/1000000000000000001",
    "https://x.com/i/lists/1234567890123456"
  ]
}

💡 API tip: Search is on by default. For a single-purpose API run, set "enableSearch": false and turn on only the feature you need.

---

Output you can trust

Every row is a flat JSON object with a consistent envelope, so mixed results stay easy to filter and join:

Field	Meaning
featureType	Which feature produced this row — filter on it
scrapedAt	ISO 8601 UTC timestamp of collection
sourceUrl	Canonical https://x.com/... link when applicable
inputRef	The query, ID, or URL that produced the row

Tweet rows include: id, url, text, lang, createdAt, likeCount, replyCount, retweetCount, quoteCount, bookmarkCount, viewCount, hashtags, mentions, urls, media, isReply, isRetweet, isQuote, author, and more.

User rows include: id, username, displayName, bio, followersCount, followingCount, tweetCount, verified, isBlueVerified, location, website, profileImageUrl, and more.

Example — a search result:

{
  "featureType": "search",
  "scrapedAt": "2026-07-05T12:00:00.000Z",
  "sourceUrl": "https://x.com/nasa/status/1000000000000000001",
  "inputRef": "AI agents",
  "id": "1000000000000000001",
  "text": "Exploring the universe with AI agents…",
  "lang": "en",
  "likeCount": 1240,
  "retweetCount": 320,
  "viewCount": 98000,
  "authorUsername": "nasa",
  "createdAt": "2026-07-05T09:14:00.000Z"
}

Couldn't be fetched? You get an explicit marker instead of a silent gap:

{
  "featureType": "unavailable",
  "reason": "private",
  "message": "User @example is protected.",
  "inputRef": "example",
  "scrapedAt": "2026-07-05T12:00:00.000Z"
}

After every run, the OUTPUT tab shows a summary: enabledFeatures, recordCounts per feature, upstreamRequestCount, and any non-fatal errors.

---

Built for LLMs, agents & MCP

This Actor is designed to drop straight into AI workflows:

• Flat, predictable records — camelCase keys, null for missing values, no deep nesting to unravel.
• Stable featureType enum — agents filter by row type without guessing shapes.
• Self-describing rows — inputRef, sourceUrl, and scrapedAt on every record.
• Canonical URLs — full https://x.com/... links for every tweet and profile.

Use it through Apify MCP and ask your agent to filter on featureType == "search", or export JSON/CSV from the dataset API and pipe into your RAG store.

---

Dataset views

The dataset ships with ready-made tabbed views in the Console — Overview, Search, Profiles, User tweets, Tweet details, Engagers, Network, Notifications, Analytics, Direct messages, and Unavailable — so you can read results without building a single filter.

---

Account data (your own session)

Three optional features — notifications, analytics, and DM inbox — read your own X account. They require credentials you provide at run time (all secret, never logged):

Field	Needed for	Format
X auth token (authToken)	Notifications, Analytics, DM Inbox	Your auth_token cookie value
Proxy host / port / username / password	All account-data features	Filled in four simple fields (assembled for you)
X ct0 token (ct0)	DM Inbox	Your ct0 CSRF cookie

How to get your auth token (and ct0):

1. Log in to x.com in Chrome.
2. Press F12 → open the Application tab → Cookies → https://x.com.
3. Copy the value of the cookie named auth_token (and ct0 for DM Inbox).

Easier option: install the free Cookie-Editor or EditThisCookie Chrome extension, click it on x.com, and copy the auth_token / ct0 values.

Proxy: enter your host, port, username, and password in the four dedicated fields — the Actor assembles them for you. (Advanced users can paste a full host:port@user:pass string instead.) Use a stable, sticky residential/ISP proxy you control — rotating IPs (including Apify's rotating proxy) can trigger X security challenges and are never used for logged-in features.

These features are read-only (no posting, liking, following, or sending messages). You're responsible for complying with X's Terms of Service and applicable privacy laws when exporting your own data.

---

Controlling scope & cost

Field	Default	What it does
maxItems	100000	Global ceiling for the whole run — the main cost lever
Per-feature max (e.g. searchMaxResultsPerQuery, networkMaxUsers)	varies	Cap a single feature; 0 = unlimited within maxItems
maxRetries	5	Automatic retries on rate limits / server errors
includeRaw	false	Attach the untouched upstream payload to each row

Heavy jobs (full follower lists, complete reply threads) use more compute and time — lower maxItems or per-feature caps to keep runs lean. The full field list with defaults lives in the Console input form.

---

FAQ

Do I need an X/Twitter API key? No. Public features work out of the box.

Is this real-time? Yes — each run fetches live data at request time.

What formats can I export? JSON, CSV, Excel, HTML, or RSS — plus the Apify API, webhooks, and integrations.

Will one bad URL break my run? No. Failed items become unavailable rows and the run continues.

Can I schedule it? Yes — use Apify Schedules to run hourly, daily, or on any cron you like.

---

Support

Open an issue on the Actor page or contact the publisher with your run ID, the features you enabled, and your (redacted) input JSON.

---

Contact me

Need something built beyond this Actor? I take on custom projects — from Apify scrapers and data pipelines to full-stack web apps.

Email	dubem115@gmail.com
GitHub	github.com/DrunkCodes

What I can help with:

• Custom scraping actors (X, Amazon, LinkedIn, e-commerce, lead gen, and more)
• Data extraction, enrichment, and automation workflows
• Web applications of any kind — dashboards, SaaS tools, internal tools, and APIs

Reach out with a short description of your project and timeline — happy to discuss scope and pricing.