Twitter Profile Scraper

Twitter Profile Scraper — Profile Tweets with Date Filters, Replies & About Data

Scrape tweets from X (Twitter) profiles with the controls that matter for research: a date range, image-only filtering, native-retweet exclusion, an optional profile "about" record per user, and — with login cookies — the replies to each qualifying tweet.

Supply profile URLs or bare handles, cap the run, and get X's native tweet structure back.

---

What is Twitter Profile Scraper?

This Actor is built for bounded, filtered timeline extraction rather than "grab the latest N tweets".

The difference shows in three places:

• Date range, accepting absolute dates (2026-01-15) or relative values (7 days) on both ends — so you can pull exactly one month, or exactly last week, instead of over-fetching and filtering afterwards.
• Reply threads, fetched per tweet through X's tweet-detail endpoint, with a minReplyCount gate so you only spend requests on conversations that actually happened.
• About data, an optional one-per-user profile record emitted alongside the tweets, so a run gives you both the content and the account context in one dataset.

Output uses X's own native field names (full_text, favorite_count, id_str), which drops straight into tooling written against the classic API shape.

---

What data can you extract?

Tweet records

id_str, full_text, created_at, conversation_id_str, lang, favorite_count, retweet_count, reply_count, quote_count, bookmark_count, favorited, retweeted, bookmarked, is_quote_status, display_text_range, entities, extended_entities (media), user_id_str, in_reply_to_status_id_str, in_reply_to_user_id_str, in_reply_to_screen_name, and a nested user object.

About-data records (getAboutData)

aboutData: true, screen_name, rest_id, is_blue_verified, verification, profile_image_shape, avatar, core, location, professional, profile_bio, and a merged user object.

Reply records (getReplies)

Replies to each qualifying tweet, in the same tweet structure, linked by conversation.

---

Why teams scrape X profiles

For time-bounded research

Studying a campaign, an incident or a quarter means fetching a defined window. Date bounds on both ends make the dataset reproducible, which matters if anyone will check your work.

For conversation analysis

Tweets alone show what an account said. Replies show how people responded. minReplyCount keeps that affordable by skipping tweets nobody engaged with.

For visual content research

onlyImages isolates image tweets, which behave very differently from text-only posts in both reach and engagement.

For original-content analysis

includeNativeRetweets: false drops RT @ retweets so you measure what an account actually produces rather than what it amplifies.

For account profiling

About data gives you the account's bio, location, professional category and verification state alongside its output — useful when profiling many accounts at once.

For pipelines expecting X's classic shape

Native field names mean existing parsers and scripts usually work without remapping.

---

How to scrape X profiles step by step

1. Open the Actor and add Profile URLs, or bare handles in Twitter Handles — either works.
2. Set Maximum Items — a hard cap across the whole run.
3. (Optional) Set a Start Date and End Date, absolute or relative.
4. (Optional) Turn on Only Tweets With Images, Include Native Retweets, or Emit Profile About Data.
5. For replies, enable Scrape Replies and supply your auth_token and ct0 cookies.
6. Click Start, then export the Output tab as JSON.

---

⬇️ Input

Example input

{
  "twitterHandles": ["elonmusk", "nasa"],
  "maxItems": 200,
  "start": "30 days",
  "includeNativeRetweets": false,
  "getAboutData": true
}

Input reference — targets and volume

Field	Type	Default	Description
startUrls	array	["elonmusk"]	Profile URLs (https://x.com/elonmusk). Required if twitterHandles is empty.
twitterHandles	array	[]	Usernames without the @. Required if startUrls is empty.
maxItems	integer	10	Hard cap on total items written across the whole run — not per profile.

Input reference — date range

Field	Type	Default	Description
start	string	—	Lower bound. Absolute (2026-01-15) or relative (7 days). Empty = no lower bound.
end	string	—	Upper bound. Absolute or relative (1 day). Empty = no upper bound.

Input reference — filters

Field	Type	Default	Description
includeNativeRetweets	boolean	false	When false, native retweets (text starting RT @) are dropped. Only effective for roughly the last 7–10 days, because X exposes the retweet form only on recent posts.
onlyImages	boolean	false	Keep only tweets containing at least one photo.
getAboutData	boolean	false	Emit one profile-about record per user, built from the embedded user object.

Input reference — replies and cookies

Field	Type	Default	Description
getReplies	boolean	false	Fetch replies to each qualifying tweet. Requires login cookies.
minReplyCount	integer	0	Only fetch replies for tweets whose reply count is at least this — the cost control for reply scraping.
authToken	string	—	auth_token cookie from x.com (devtools → Application → Cookies). Required for getReplies.
csrfToken	string	—	ct0 cookie from the same session. Required for getReplies.
proxyConfiguration	object	—	Apify proxy configuration. Recommended for reliability against rate limits.

⚠️ Cookies are credentials and are needed only when getReplies is on. They grant access to that X account — use a dedicated secondary account, and note that automated activity can lead X to restrict accounts.

---

⬆️ Output

Example output — tweet record (trimmed)

{
  "id_str": "1789012345678901234",
  "created_at": "Tue Aug 04 14:22:31 +0000 2026",
  "full_text": "Shipped a big update today. Full changelog in the replies 👇",
  "conversation_id_str": "1789012345678901234",
  "lang": "en",
  "favorite_count": 18420,
  "retweet_count": 2140,
  "reply_count": 612,
  "quote_count": 184,
  "bookmark_count": 3902,
  "is_quote_status": false,
  "user_id_str": "44196397",
  "entities": { "hashtags": [], "user_mentions": [], "urls": [] },
  "user": {
    "screen_name": "exampleuser",
    "name": "Example User",
    "created_at": "Tue Mar 21 20:50:14 +0000 2017"
  }
}

Example output — about-data record (trimmed)

{
  "aboutData": true,
  "screen_name": "exampleuser",
  "rest_id": "44196397",
  "is_blue_verified": true,
  "location": { "location": "Berlin, Germany" },
  "professional": { "category": [{ "name": "Software Company" }] },
  "profile_bio": { "description": "Building things on the internet." },
  "user": { "screen_name": "exampleuser", "name": "Example User" }
}

Illustrative values — a live run returns current X data.

Two record types share the dataset. Filter on aboutData to separate profile records from tweets.

---

Usage recipes

Exactly the last 30 days, original content only

{
  "twitterHandles": ["yourcompetitor"],
  "start": "30 days",
  "includeNativeRetweets": false,
  "maxItems": 500
}

A specific historical window

{
  "twitterHandles": ["exampleaccount"],
  "start": "2026-01-01",
  "end": "2026-03-31",
  "maxItems": 1000
}

Absolute bounds on both ends make the dataset reproducible.

Conversations on high-engagement tweets only

{
  "twitterHandles": ["exampleaccount"],
  "getReplies": true,
  "minReplyCount": 50,
  "authToken": "<YOUR_AUTH_TOKEN_COOKIE>",
  "csrfToken": "<YOUR_CT0_COOKIE>",
  "maxItems": 300
}

minReplyCount is what stops reply scraping from becoming expensive — it skips tweets nobody replied to.

Visual content study

{
  "twitterHandles": ["brandaccount"],
  "onlyImages": true,
  "start": "90 days",
  "maxItems": 300
}

Profile plus content in one run

Enable getAboutData alongside a normal tweet run, then split the export on the aboutData flag.

---

How does this compare to X's official API?

X's API is a paid, tiered product. Meaningful read access — timelines, search, conversation threads — sits behind Basic, Pro or Enterprise plans with monthly post caps, and free access is effectively write-only for most use cases.

This Actor reads public profile timelines without an API key or subscription. Reply scraping is the one place it needs credentials, because X does not serve conversation threads to logged-out visitors. If you need official guarantees, full-archive search and SLAs, the paid API is the right tool.

---

Integrate and automate

Python

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_API_TOKEN>")

run = client.actor("scraperforge/twitter-profile-scraper").call(run_input={
    "twitterHandles": ["elonmusk"],
    "maxItems": 200,
    "start": "30 days",
    "includeNativeRetweets": False,
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    if item.get("aboutData"):
        continue
    print(item["created_at"], item["favorite_count"], "|", item["full_text"][:70])

JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_APIFY_API_TOKEN>' });

const run = await client.actor('scraperforge/twitter-profile-scraper').call({
    twitterHandles: ['elonmusk'],
    maxItems: 200,
    start: '30 days',
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

REST API

curl -X POST "https://api.apify.com/v2/acts/scraperforge~twitter-profile-scraper/runs?token=<YOUR_APIFY_API_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"twitterHandles":["elonmusk"],"maxItems":200,"start":"30 days"}'

n8n, Make, Zapier and AI agents

Call the Actor from n8n, Make, Zapier or an MCP-capable agent — relative date bounds like 1 day make it a natural fit for scheduled monitoring.

Schedules and webhooks

Attach a Schedule with start: "1 day" for daily collection, and use webhooks or the Google Sheets / Airtable / Slack integrations to route results.

---

Pricing and what you are charged for

Pay-per-event: a small Actor-start charge plus a charge per row delivered — and tweets, about-data records and replies are all rows.

Two cost levers:

• maxItems caps the whole run, not each profile, so it is a genuine ceiling.
• minReplyCount gates reply fetching, which is the part that multiplies row counts fastest.

Current rates are on the Pricing tab of this Actor's page, and Apify shows an estimate before and during every run.

---

Limits, reliability and blocking

• maxItems is a run-wide total, shared across profiles and record types — unlike per-target caps in other Actors.
• includeNativeRetweets: false only works reliably for the last ~7–10 days, because X stops exposing the retweet form on older posts. Beyond that window some retweets may slip through.
• Replies require both cookies. authToken and csrfToken must come from the same logged-in session.
• Public profiles only. Protected accounts are not accessible.
• X limits timeline depth, so a wide date range may not reach as far back as you expect.
• Date bounds accept absolute or relative values — mixing them (start: "2026-01-01", end: "1 day") is allowed and often useful.
• Two record types share the dataset — filter on aboutData when analysing.
• A proxy is recommended for reliability against rate limiting on larger runs.
• Default run options are 4 GB memory and a 1-hour timeout.

---

Is it legal to scrape X profiles?

This Actor collects publicly available X content — timelines any visitor can read without logging in. Reply scraping uses your own session to reach conversation threads, exactly as your browser would; it does not access private accounts or direct messages.

Posts and media remain the property of their authors, and handles, bios, locations and avatars are personal data. Ensure your use complies with X's terms, copyright, and GDPR or comparable regulations. If you supply cookies, you are responsible for that account, including X's restrictions on automated access.

---

❓ Frequently asked questions

Do I need X API access?

No. Public timelines are read without a developer account. Cookies are needed only for replies.

URLs or handles — which should I use?

Either. Fill startUrls with profile URLs or twitterHandles with bare usernames; one of the two must be non-empty.

How do I get exactly one month of tweets?

Set start and end to absolute dates. Relative values like 30 days work too when you want a rolling window.

Why are some retweets still appearing?

Because includeNativeRetweets: false relies on a form X only exposes for roughly the last 7–10 days. Older retweets cannot always be identified.

Why do replies need cookies?

X does not serve conversation threads to logged-out visitors. The Actor uses your session for that endpoint only.

How do I keep reply scraping affordable?

Raise minReplyCount. Fetching replies for tweets with two replies is rarely worth a request.

What is about data?

An optional one-per-user profile record — bio, location, professional category, verification — emitted alongside the tweets. Filter on the aboutData flag to separate them.

Is maxItems per profile?

No. It caps the entire run, including tweets, replies and about records.

Which export format should I use?

JSON — the output uses X's native nested structures (entities, extended_entities, user).

---

🔗 Related scrapers

• Twitter (X) Tweets Profiles Scraper — mixes profiles and keyword search in one input.
• X Posts Search Scraper — search-first, with full operator support.
• X (Twitter) Posts Scraper — higher per-target limits for timeline extraction.
• 🐦 Twitter / X Tweets Scraper — another timeline extraction option.

Browse the full collection on the ScraperForge profile.

---

💬 Feedback

Need deeper history, better retweet detection, or a custom X research pipeline? Open an issue on the Issues tab of this Actor.