🐦 Twitter / X Tweets Scraper

🐦 Twitter / X Tweets Scraper — Authenticated Timeline Extraction with X's Native Data Shape

Scrape tweets from X (Twitter) profiles using your own logged-in session, and get X's native, unflattened data structure back: full_text, favorite_count, bookmark_count, entities, extended_entities, and a complete user object with verification, professional and privacy metadata.

Bulk profile input, a per-profile cap, and an optional since-date that accepts absolute dates or rolling windows.

---

What is Twitter / X Tweets Scraper?

This Actor is for people who want X's own data shape, not a simplified version of it.

Where flattened scrapers pick a handful of fields and rename them, this one preserves X's internal structure — legacy fields, entities, extended_entities, and a rich user object including verification, professional, privacy, dm_permissions, tipjar_settings and affiliates_highlighted_label. If you have parsers written against X's API responses, they will generally work unchanged.

It authenticates with your session cookies, which is what allows reliable timeline access, and includes a built-in escalation path when direct requests fail.

---

What data can you extract?

Tweet (X native fields)

id_str, full_text, created_at, conversation_id_str, lang, favorite_count, retweet_count, reply_count, quote_count, bookmark_count, favorited, is_quote_status, possibly_sensitive, possibly_sensitive_editable, entities, extended_entities (media).

user (X native structure)

rest_id, id, legacy (the full classic user object), is_blue_verified, verification, avatar, location, professional, privacy, dm_permissions, media_permissions, relationship_perspectives, profile_image_shape, has_graduated_access, super_follow_eligible, super_followed_by, super_following, tipjar_settings, affiliates_highlighted_label, parody_commentary_fan_label.

---

Why teams scrape X timelines

For developers with existing X tooling

Native field names mean no remapping. Code written against X's API response shape — or against older Twitter libraries — usually consumes this output directly.

For research requiring full fidelity

entities and extended_entities carry the complete structure of mentions, URLs, hashtags and media variants, including video bitrate options. Flattened outputs discard most of it.

For engagement analysis

bookmark_count alongside likes, reposts, replies and quotes gives you the full engagement picture, including the metric that best signals genuine usefulness.

For account profiling

The user object exposes signals most scrapers drop — professional category, verification type, DM permissions, super-follow status, tip-jar settings — which together describe how an account is actually configured.

For time-bounded collection

since_date accepts absolute dates or rolling windows (7 days, 3 months), so scheduled runs collect only the period you care about.

For bulk monitoring

Paste many profiles at once and cap each independently.

---

How to scrape X tweets step by step

1. Open x.com in your browser and log in.
2. Open DevTools → Application → Cookies → copy auth_token and ct0.
3. Paste both into the Actor's cookie fields.
4. Add profile URLs or usernames to Twitter / X Profile URLs or Usernames.
5. Set Maximum Tweets per Profile and, optionally, a since-date.
6. Click Start, then export the Output tab as JSON.

---

⬇️ Input

Example input

{
  "start_urls": [
    "https://x.com/peckshield",
    "elonmusk"
  ],
  "max_items": 200,
  "since_date": "30 days",
  "auth_token": "<YOUR_AUTH_TOKEN_COOKIE>",
  "ct0": "<YOUR_CT0_COOKIE>"
}

Input reference

Field	Type	Default	Description
start_urls	array	— (required)	X profile URLs (https://x.com/peckshield) or plain usernames (peckshield). Bulk input supported — paste many at once.
max_items	integer	10	Stop after this many tweets per profile. High for full sweeps, low for quick samples.
since_date	string	—	Lower bound on tweet time. Absolute (YYYY-MM-DD) or relative (7 days, 2 weeks, 3 months, 1 year). Only tweets on or after the resolved date are returned. Empty = no lower bound.
auth_token	string	— (required)	Your X auth_token cookie. DevTools → Application → Cookies. Stored securely.
ct0	string	— (required)	Your X ct0 (CSRF) cookie, from the same Cookies panel. Stored securely.
proxy_configuration	object	no proxy	Apify Proxy options (groups, country). Built-in escalation still applies when direct requests fail.

⚠️ Both cookies are required and must come from the same logged-in session. They grant access to that X account — use a dedicated secondary account, keep the values private, and refresh them when runs start failing. Automated activity can lead X to restrict accounts.

---

⬆️ Output

Example output (trimmed)

{
  "id_str": "1789012345678901234",
  "full_text": "Shipped a big update today. Full changelog in the replies 👇",
  "created_at": "Tue Aug 04 14:22:31 +0000 2026",
  "conversation_id_str": "1789012345678901234",
  "lang": "en",
  "favorite_count": 18420,
  "retweet_count": 2140,
  "reply_count": 612,
  "quote_count": 184,
  "bookmark_count": 3902,
  "favorited": false,
  "is_quote_status": false,
  "possibly_sensitive": false,
  "entities": {
    "hashtags": [{ "text": "buildinpublic" }],
    "user_mentions": [],
    "urls": []
  },
  "extended_entities": {
    "media": [
      { "type": "photo", "media_url_https": "https://pbs.twimg.com/media/…" }
    ]
  },
  "user": {
    "rest_id": "44196397",
    "is_blue_verified": true,
    "profile_image_shape": "Circle",
    "location": { "location": "Berlin, Germany" },
    "professional": { "category": [{ "name": "Software Company" }] },
    "verification": { "verified": false },
    "legacy": {
      "screen_name": "exampleuser",
      "name": "Example User",
      "description": "Building things on the internet.",
      "followers_count": 482100,
      "friends_count": 913,
      "statuses_count": 26410
    }
  }
}

Illustrative values — a live run returns current X data.

---

Usage recipes

Rolling 30-day window

{
  "start_urls": ["yourcompetitor"],
  "max_items": 500,
  "since_date": "30 days",
  "auth_token": "<YOUR_AUTH_TOKEN_COOKIE>",
  "ct0": "<YOUR_CT0_COOKIE>"
}

Bulk profile monitoring

{
  "start_urls": ["account1", "account2", "account3", "account4"],
  "max_items": 100,
  "since_date": "7 days",
  "auth_token": "<YOUR_AUTH_TOKEN_COOKIE>",
  "ct0": "<YOUR_CT0_COOKIE>"
}

Remember max_items is per profile, so this returns up to 400 rows.

From a fixed historical date

{
  "start_urls": ["exampleaccount"],
  "max_items": 1000,
  "since_date": "2026-01-01",
  "auth_token": "<YOUR_AUTH_TOKEN_COOKIE>",
  "ct0": "<YOUR_CT0_COOKIE>"
}

Full media analysis

Read extended_entities.media for complete media objects, including video variants with bitrates — detail that flattened scrapers discard.

Daily incremental collection

Set since_date: "1 day", attach a daily Schedule, and deduplicate on id_str.

---

How does this compare to X's official API?

X's API is a paid, tiered product with monthly post caps; timeline reading at research volumes starts at Basic and scales up in cost. The free tier is not usable for this.

This Actor reads timelines through your own logged-in session, exactly as your browser does, and returns X's native data structure — so it is often a drop-in substitute for code written against the old API shape. If you need contractual guarantees, full-archive access and SLAs, use the paid API.

---

Integrate and automate

Python

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_API_TOKEN>")

run = client.actor("scraperforge/twitter-tweets-scraper").call(run_input={
    "start_urls": ["peckshield"],
    "max_items": 200,
    "since_date": "30 days",
    "auth_token": "<YOUR_AUTH_TOKEN_COOKIE>",
    "ct0": "<YOUR_CT0_COOKIE>",
})

for t in client.dataset(run["defaultDatasetId"]).iterate_items():
    legacy = (t.get("user") or {}).get("legacy") or {}
    print(t["created_at"], t["favorite_count"], "|", legacy.get("screen_name"), "|", t["full_text"][:60])

JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_APIFY_API_TOKEN>' });

const run = await client.actor('scraperforge/twitter-tweets-scraper').call({
    start_urls: ['peckshield'],
    max_items: 200,
    auth_token: '<YOUR_AUTH_TOKEN_COOKIE>',
    ct0: '<YOUR_CT0_COOKIE>',
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

REST API

curl -X POST "https://api.apify.com/v2/acts/scraperforge~twitter-tweets-scraper/runs?token=<YOUR_APIFY_API_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"start_urls":["peckshield"],"max_items":200,"auth_token":"<YOUR_AUTH_TOKEN_COOKIE>","ct0":"<YOUR_CT0_COOKIE>"}'

n8n, Make, Zapier and AI agents

Call the Actor from n8n, Make, Zapier or an MCP-capable agent — since_date: "1 day" plus a daily schedule makes a clean incremental feed.

Schedules and webhooks

Match your Schedule interval to since_date, deduplicate on id_str, and route results with webhooks or the Google Sheets / Airtable / Slack integrations.

---

Pricing and what you are charged for

Pay-per-event: a small Actor-start charge plus a charge per tweet row delivered. max_items is per profile, so your ceiling is max_items × number of profiles.

since_date is the cost control on recurring runs — a tight window means you only pay for new tweets.

Current rates are on the Pricing tab of this Actor's page, and Apify shows an estimate before and during every run.

---

Limits, reliability and blocking

• Both cookies are required and must come from the same session. Expired cookies are the most common cause of a failing run — refresh them from your browser.
• max_items is per profile, so bulk input multiplies your row count.
• Public profiles only. Protected accounts are not accessible even with a session, unless that account already follows them.
• X limits timeline depth, so a wide since_date may not reach as far back as you expect.
• Output is X's native shape. It is deliberately nested — user.legacy holds the classic user fields — which is a feature for developers and a nuisance in CSV.
• Media URLs expire — they point at X's CDN.
• Use a dedicated secondary account. Automated sessions can be restricted by X.
• Proxy escalation is built in and still applies when direct requests fail; configure groups or country only if you need to.
• Default run options are 4 GB memory and a 1-hour timeout.

---

Is it legal to scrape X (Twitter)?

This Actor reads public timelines through your own authenticated session, the same way your browser does. It does not post, follow, access direct messages, or read private accounts.

Posts and media remain the property of their authors, and handles, bios, locations and avatars are personal data. Ensure your use complies with X's Terms of Service, copyright, and GDPR or comparable regulations. You are responsible for the account whose cookies you supply, including X's restrictions on automated access.

---

❓ Frequently asked questions

Why do I need cookies?

They authenticate the session used to read timelines, which makes access far more reliable than anonymous requests.

Where do I find auth_token and ct0?

Log in to x.com → DevTools → Application → Cookies → copy both values from the same panel.

Why is the output nested rather than flat?

Deliberately — it preserves X's native structure so existing parsers work unchanged. If you want a flat shape, use one of the flattened X Actors linked below.

What does since_date accept?

Absolute dates (2026-01-01) or rolling windows (7 days, 2 weeks, 3 months, 1 year). Only tweets on or after the resolved date are returned.

Is max_items per profile or per run?

Per profile.

Can I scrape protected accounts?

Only if the account whose session you supply already has access to them. Otherwise, no.

Do I get bookmark counts?

Yes — bookmark_count is included, alongside likes, reposts, replies and quotes.

What is in extended_entities?

Full media objects, including video variants with bitrates — the detail flattened scrapers usually drop.

Which export format should I use?

JSON. The output is intentionally nested; CSV would flatten user.legacy and extended_entities awkwardly.

---

🔗 Related scrapers

• X (Twitter) Posts Scraper — no cookies needed, up to 1,000 posts per target.
• X Posts Scraper (Twitter) — flat output with views and bookmarks.
• X Posts Search Scraper — keyword and advanced-operator search with time windows.
• Twitter Profile Scraper — date-bounded extraction with reply threads.

Browse the full collection on the ScraperForge profile.

---

💬 Feedback

Need reply threads, search support, or a custom X data pipeline? Open an issue on the Issues tab of this Actor.