🐦 Twitter / X Tweets Scraper

The fastest, friendliest way to collect public tweets from any X (Twitter) profile — at scale, in real time, and with built-in anti-block defences.

Paste one (or hundreds) of profile URLs, pick how many tweets per profile, and watch structured JSON / CSV / Excel rolls into your dataset live — already labelled, sorted, and ready to drop into your BI tool, spreadsheet, or LLM pipeline.

---

✨ Why this Actor?

🟢 This Actor	🔴 Other tools
⚡ Rust-powered HTTP via impit — looks identical to a real Chrome browser	Plain requests / httpx that anti-bot engines block on sight
🛡️ 4-tier proxy escalator (direct → datacenter → residential, sticky)	Single proxy, no fallback
📡 Live streaming output — first tweet appears in seconds	Wait until the run ends to see any data
📅 since_date filter built in	Manual post-filtering required
🪵 Per-profile section logs with engagement preview	Wall-of-text logs
🔐 Cookie-based authentication — bypasses guest rate limits	Limited guest quotas

---

🚀 Key Features

• 🔗 Bulk URL input — paste an unlimited number of profile URLs or @handles at once
• 🎯 Per-profile cap — stop after N tweets per profile
• 📅 Since-date filter — only tweets posted on or after a given date
• 🌐 Smart proxy fallback — starts direct, automatically escalates if X pushes back
• ⏱️ Polite, jittered pacing — configurable inter-request delay with random jitter
• 📊 Live dataset writes — every tweet is pushed the moment it is parsed; nothing is lost on crash
• 🪵 Beautiful run log — engagement counts, previews, and section headers per profile
• 🧾 Rich tweet schema — text, language, full entities, all engagement metrics, full user object, view counts, and more

---

📥 Input

Field	Type	Required	Description
start_urls	array	✅	List of X profile URLs (e.g. https://x.com/peckshield) or bare usernames
auth_token	string (secret)	✅	Your X.com auth_token cookie
ct0	string (secret)	✅	Your X.com ct0 (CSRF) cookie
max_items	integer	–	Max tweets per profile (default 100, range 1–10,000)
since_date	string	–	Apify datepicker: absolute YYYY-MM-DD or relative (7 days, 2 weeks, …). Only tweets from the resolved date forward
request_delay	number	–	Seconds between page requests (default 1.0, jitter added in code)
proxy_configuration	object	–	Defaults to direct connection. Fallback escalator kicks in automatically

🔐 How to get the cookies

1. Open https://x.com in your browser, signed in.
2. Press F12 → Application tab → Cookies → https://x.com.
3. Copy the values of auth_token and ct0 into the input fields above.
4. The Actor stores both as secrets — they never appear in the run log.

📋 Example input

{
  "start_urls": [
    { "url": "https://x.com/peckshield" },
    { "url": "https://x.com/elonmusk" },
    { "url": "naval" }
  ],
  "max_items": 200,
  "since_date": "2026-01-01",
  "auth_token": "REDACTED",
  "ct0": "REDACTED",
  "request_delay": 1.0,
  "proxy_configuration": { "useApifyProxy": false }
}

---

📤 Output

Every tweet is pushed to the default dataset as soon as it is parsed. Open the Dataset → Overview tab for a clean, sortable table.

📊 Example output row

{
  "id_str": "2056183051357495735",
  "url": "https://x.com/peckshield/status/2056183051357495735",
  "created_at": "Mon May 18 01:20:16 +0000 2026",
  "full_text": "@VerusCoin token has been hacked. ~$1.78M stolen…",
  "lang": "en",
  "favorite_count": 142,
  "retweet_count": 31,
  "reply_count": 12,
  "quote_count": 4,
  "bookmark_count": 24,
  "views_count": "98231",
  "is_quote_status": false,
  "possibly_sensitive": false,
  "user_screen_name": "peckshield",
  "user_name": "PeckShield Inc.",
  "user_followers_count": 312874,
  "user_is_verified": true,
  "_section": "@peckshield",
  "entities": { "media": [ … ], "urls": [ … ], "user_mentions": [ … ] },
  "extended_entities": { "media": [ … ] },
  "user": { "rest_id": "…", "legacy": { … }, "professional": { … } }
}

The _section column tags every row with the source profile so you can slice the dataset cleanly per profile in the Console table view.

📂 Dataset view

The dataset comes preconfigured with an Overview table that shows:

Tweet ID · Tweet URL · Posted At · Tweet · Lang · ❤️ Likes · 🔁 Retweets · 💬 Replies · 📝 Quotes · 🔖 Bookmarks · 👁 Views · @handle · Followers · Verified

Export options: JSON · JSONL · CSV · Excel · HTML · RSS · XML.

---

🚀 How to Use (Apify Console)

1. Log in at https://console.apify.com → Actors.
2. Open Twitter / X Tweets Scraper.
3. Paste profile URLs / handles into 🔗 Twitter / X Profile URLs.
4. Paste your auth_token and ct0 cookies into the secret fields.
5. Pick Max tweets per profile and an optional Since date.
6. Click Start.
7. Watch tweets stream into the Output tab in real time.
8. When the run finishes, export to JSON / CSV / Excel.

---

🤖 Use via API / MCP

REST API (sync — returns dataset items in one call)

curl -X POST "https://api.apify.com/v2/acts/<ACTOR_ID>/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "start_urls": [{ "url": "https://x.com/peckshield" }],
       "max_items": 100,
       "since_date": "2026-01-01",
       "auth_token": "REDACTED",
       "ct0": "REDACTED"
     }'

Python SDK

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("<ACTOR_ID>").call(run_input={
    "start_urls": [{"url": "https://x.com/peckshield"}],
    "max_items": 50,
    "auth_token": "REDACTED",
    "ct0": "REDACTED",
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["full_text"])

---

🎯 Best Use Cases

• 📈 Market & competitor monitoring — track what top accounts in your space are saying
• 🛡️ Security / threat intel — watch researchers like @peckshield, @SlowMist_Team for live exploit reports
• 🪙 Crypto signal mining — feed tweets into your sentiment model
• 📰 News & journalism — bulk-collect statements from public figures
• 🧠 LLM training / fine-tuning — curate domain-specific tweet corpora
• 🔍 Brand mentions & PR — pull every tweet from a brand's official handle for analysis

---

💲 Pricing

This Actor is monetized with pay-per-event (PPE) — you pay only for the scraped tweets you actually receive.

Event	What you pay for
🪙 apify-actor-start	Per-run startup (covered up to 5s of compute)
📥 result-item	Per tweet pushed to the dataset (the primary value unit)

Configure exact prices in the Pricing tab in Console. Platform usage (compute, proxy) is included in the per-event price. The Actor exits gracefully when your ACTOR_MAX_TOTAL_CHARGE_USD is reached — you never get charged beyond your cap.

---

❓ Frequently Asked Questions

Q: Do I really need the auth_token and ct0 cookies? Yes. X's guest endpoint is heavily throttled; authenticated requests are required for stable bulk scraping. The cookies are stored as Actor secrets and never logged.

Q: My account is at risk of being banned? X may flag accounts that scrape aggressively. We recommend using a dedicated burner account, conservative max_items, and a positive request_delay. The Actor uses real-browser HTTP/TLS fingerprints to look like normal traffic.

Q: What if I get rate limited? The Actor automatically escalates from direct → datacenter proxy → residential proxy (sticky, with 3 IP rotations) the moment X starts pushing back. No action needed from you.

Q: Can I scrape protected / private accounts? No — the Actor only scrapes public tweets. Following a private account does not grant scrape access.

Q: How fresh is the data? Live. Each tweet hits your dataset within milliseconds of being parsed.

Q: Does this work for replies / threads / quote tweets? Yes — replies, quotes, retweets, media-only tweets, and longform note tweets are all extracted with full text and entities.

---

⚖️ Cautions & Legal

• Data is collected only from publicly available sources on x.com.
• You are responsible for legal compliance: GDPR, CCPA, anti-spam laws, X's Terms of Service, and any local regulation that applies to you.
• Do not use this Actor to scrape private accounts, automate harassment, or build profiles of private individuals.
• The Actor honours reasonable rate limits and robots.txt etiquette by default.

---

📬 Support & Feedback

• Open an issue on the Actor page if something is off.
• Feature requests welcome — we ship fast.
• For commercial / high-volume usage, reach out via the Actor profile.