X Twitter Posts Search With Author Profile Details

X (Twitter) Scraper — Posts, Author Profiles and Verification Data

X (Twitter) Posts Search With Author Profile Details searches X by profile, @handle, keyword, hashtag or a pasted search URL, and returns every matching post as typed JSON — with the full author card already attached to each row: follower count, account age, bio, location, website, verification type, avatar and banner. The author data comes from the same response that carries the post, so it costs zero extra requests. Unlike scraping frameworks that hand you raw HTML to parse, this Actor returns a stable, typed schema — ready for a database, a spreadsheet, or an LLM context window without a cleanup pass. This guide covers every input and output field byte-for-byte, and how teams actually run it: real-time enrichment, scheduled monitoring, and bulk dataset builds.

What Does This X (Twitter) Scraper Do?

It runs against X's own internal GraphQL endpoints — UserByScreenName, UserTweets and SearchTimeline — the same calls the x.com web app makes, and normalizes the responses into one consistent row shape whether the target is a profile or a search. No X account is required for profile (@handle) targets; keyword, hashtag and search-URL targets need your own X session cookies, because X answers logged-out keyword search with an HTTP 404.

• Accepts profile URLs, bare @handles, keywords, hashtags, X search URLs and advanced search: query syntax in the same input list
• Attaches a 25-field author card to every post at no extra request cost, pulled from the post payload X already sent
• Adds an optional 6-field verification tier (identity verification, verified-since date, affiliate counts) via one extra guest request per distinct author, not per post
• Filters by verification badge (onlyVerifiedAuthors) or minimum follower count (minAuthorFollowers) before rows are written
• Applies a time window in days or hours on both search modes and profile timelines
• Writes a typed, uncharged accounting row instead of a silent empty run whenever a target can't be served — login-required, rate-limited, filtered out, or genuinely empty
• Serves full-resolution avatar (400×400) and banner (1500×500) images on request, alongside the default thumbnail

Features & Capabilities

The core capability set spans target parsing, live credential discovery, and dual-tier author enrichment — each addressing a specific way X's front end resists straightforward scraping.

Core features

• Mixed target types in one run — startUrls accepts profile URLs, @handles, bare usernames, keywords, #hashtags, full x.com/search?... URLs and search: python from:NASA min_retweets:50 advanced syntax, resolved per row rather than requiring separate runs per type
• Live GraphQL credential discovery — on startup the Actor fetches x.com's current main.*.js bundle and extracts the live bearer token and query IDs for UserByScreenName, UserTweets and SearchTimeline, falling back to pinned known-good values only if that discovery fails, so a routine X front-end deploy doesn't silently break the Actor
• POST, not GET, for keyword search — X answers a GET SearchTimeline request with an HTTP 404 and a zero-byte body; every keyword and hashtag target here is issued as POST
• Real block detection — a response is only accepted as genuine data when the resolved final URL matches the expected GraphQL operation and the expected result container is present in the payload; a wrong-verb 404 and an auth-gated 404 both return HTTP 404 with an empty body, and only the structural check tells them apart
• Dual-schema author merge — X's search surface returns author data in a modern, nested schema (core.created_at, avatar.image_url, verification.verified_type); UserByScreenName returns the legacy flat schema. Every author field in this Actor is read through a merge function that folds both into one shape, so a reader tuned to only one schema does not silently return nulls
• Author fields returned per post: authorId, authorHandle, authorName, authorBio, authorLocation, authorWebsite, authorFollowers, authorFollowing, authorAccountAgeDays, authorIsBlueVerified, authorVerifiedType, authorProfileImageUrl, authorBannerUrl and more — 25 fields at zero extra request cost, 31 with the optional verification tier
• Uncharged accounting rows — a target that hits login_required, rate_limited, filtered_out, empty or an unsupported single-post link still produces a dataset row explaining why, and none of those rows are billed

How this Actor compares to other X (Twitter) scrapers

Checked on the Apify Store, 25 July 2026. scraper_one/x-profile-posts-scraper is the closest same-platform competitor; futurizerush/meta-threads-scraper is included because it documents a comparable posts-plus-profile-plus-search shape on a different platform.

Feature	This Actor	scraper_one/x-profile-posts-scraper	futurizerush/meta-threads-scraper
Output format	Typed JSON	Typed JSON	Typed JSON
Keyword / hashtag / search-URL targets	✅ (needs your X cookies)	Not documented — input is profileUrls only	✅ search mode, no login
Profile targets	✅ (no login needed)	✅ (no login needed)	✅ user mode
Author fields per post	25 base + 6 verification-tier	6 fields shown in example output (name, screenName, followersCount, favouritesCount, friendsCount, description)	Bio, follower count, verification status, contact fields extracted from bio
Uncharged accounting rows for failed/empty targets	✅ rowType: "target_status", never charged	Not documented	Documented: a zero-result run ends SUCCEEDED with a status message, not a per-row accounting record
Time-window filtering (days/hours)	✅ on both search modes and profile targets	Not documented	✅ via start_date / end_date, search mode only
Max results per target	Up to 10,000	Up to 200 (default 5)	max_posts, per-user or per-keyword

If your use case is feeding structured data to an LLM, the author-fields row is the decision-maker here — a scraper that gives you six flat author fields still leaves you re-fetching the profile separately for anything else; one that gives you thirty-one closes that gap in the same run.

When another tool might suit you better

If you need Bluesky or Threads data instead of X, this Actor won't help — angaba92/bluesky-scraper and futurizerush/meta-threads-scraper cover those platforms directly. If your only need is a handful of posts from one public profile with no author enrichment and no keyword search, a narrower single-purpose profile scraper carries less input surface to configure. This Actor is built for the case where you need posts and the person behind them, across multiple target types, in one schema.

This Actor within the Scrapio X (Twitter) suite

This Actor covers posts and author profiles from search, hashtag and profile targets. For trending-topic monitoring instead of a specific search, use Twitter Trends Scraper - Brand & Keyword Monitoring. For a narrower keyword/hashtag-only posts feed without the author-profile enrichment, use Twitter X Posts Scraper By Keyword & Hashtag Search. For replies to a specific post with sentiment and tone scoring, use Twitter X Reply Scraper: Reply Sentiment & Tone Breakdown.

Why do developers and data teams scrape X (Twitter)?

X remains the fastest public signal for breaking news, brand sentiment and account-level credibility checks — and its own API access is priced and rate-limited in ways that make ad-hoc, one-off pulls impractical for a lot of teams. The audiences below are the ones that recur.

Social listening and brand monitoring teams

A brand team searches #YourBrand or search: "your brand" min_retweets:5 with searchType: "latest" and a timeWindowHours window, then filters with minAuthorFollowers to strip low-reach noise before anyone reviews a single row. Every row already carries authorFollowers and authorVerifiedType, so an account worth escalating — a verified news outlet quoting the brand, or a high-follower critic — is visible without a second lookup. The output lands directly in the dataset as JSON, ready to pipe into a Slack alert or a monitoring dashboard.

AI training data and RAG indexing

postText is the highest-information field for both RAG indexing and training data — it is the actual post content, already unwrapped from X's note_tweet long-form container when a post exceeds the short-form limit. Two concrete uses: (1) RAG enrichment — index postText alongside authorBio and authorName so an agent answering "what has this account said publicly about X" retrieves grounded, attributable text instead of a summary. (2) Training data — favouriteCount, repostCount, replyCount, authorFollowers and authorAccountAgeDays are numeric fields with a consistent shape across every row, useful as engagement-prediction or account-credibility features without a normalization pass.

Competitive and market intelligence

Track a competitor's own account with a profile target and enrichAuthorProfiles on, or track mentions of their brand with a keyword search. authorVerifiedType and authorIsBlueVerified tell you who is actually engaging — an unverified bot network reads very differently from verified press accounts picking up a story. Re-running the same target on a schedule and diffing on postId surfaces new posts without re-processing ones you've already seen.

Research and academic use

Public-post datasets built from keyword or hashtag targets support social-science and communications research — discourse analysis, misinformation studies, network mapping by retweetedBy. This Actor only returns what X shows to a logged-in or logged-out viewer on the timeline or search surface; nothing behind a private account or a login-gated feature is collected.

Product and SaaS development

Influencer-discovery tools, lead-scoring products and monitoring dashboards can be built directly on this Actor's output: authorFollowers, authorAccountAgeDays and authorVerifiedType are exactly the fields a scoring model needs, and the schema doesn't change shape between a profile run and a search run — one integration handles both.

⬇️ Input Parameters

All twelve parameters, in schema order. startUrls and searchType are required; everything else is optional.

Parameter	Required	Type	Description	Example Value
startUrls	✅ Yes	array (stringList)	Targets to scrape. Accepts profile URLs, @handles, bare usernames, keywords, hashtags, X search URLs and search: ... advanced queries, mixed freely in one list. Each entry can also be an object with a url, query or value key instead of a plain string. Single-post links (.../status/123...) are not supported and are skipped with an uncharged row.	["@NASA", "climate change", "#worldcup"]
maxTweets	No	integer	Cap on posts collected per target. Minimum 1, maximum 10000, default 10. Clamped server-side to that range regardless of what is passed.	100
searchType	✅ Yes	string (enum)	"top" (default) returns popular/relevant posts; "latest" returns newest first. On profile targets X has no sort parameter, so this re-orders the fetched window itself (newest-first for Latest, most-liked-first for Top) instead of changing which posts X returns.	"latest"
timeWindow	No	integer	Keep only posts newer than N days (UTC). Minimum 1. Applies to both search modes and profile targets. Leave empty to collect up to maxTweets regardless of date.	7
timeWindowHours	No	integer	Keep only posts newer than N hours (UTC). Minimum 1. Takes precedence over timeWindow if both are set. Use for breaking-news / real-time monitoring.	24
authToken	No	string (secret)	The auth_token cookie value from a browser signed in to x.com (~40-character hex string). Declared isSecret — stored encrypted, masked in the Console, never written to the run log. Required together with csrfToken for keyword, hashtag and search-URL targets; profile targets work without it. Leave empty to run logged-out — keyword targets are then reported as login_required and not charged.	"a1b2c3..."
csrfToken	No	string (secret)	The ct0 cookie value from the same browser session as authToken (~160-character hex string). Declared isSecret — stored encrypted, never logged.	"d4e5f6..."
enrichAuthorProfiles	No	boolean	Default false. Makes one extra guest-mode GraphQL request per distinct author (not per post) to fetch authorIsIdentityVerified, authorVerifiedSince, authorVerificationReason, authorAffiliatesCount, authorCreatorSubscriptions and authorBirthdate. Runs logged-out — your session cookies are never used for it. A run with 200 posts from one account costs exactly one lookup.	true
onlyVerifiedAuthors	No	boolean	Default false. Keep only posts whose author has a blue check or an official Government/Business label. Evaluated from the post payload — no deep lookup needed, no extra cost.	false
minAuthorFollowers	No	integer	Minimum author follower count to keep a post. Unit: followers. Minimum 0. Evaluated from the post payload at no extra cost. Leave empty to keep every author.	5000
fullResolutionImages	No	boolean	Default true. Returns the 400×400 avatar and the 1500×500 banner instead of X's small thumbnail. The small thumbnail is always additionally returned in authorProfileImageUrlSmall.	true
proxyConfiguration	No	object (proxy editor)	Apify Proxy configuration. Prefilled with {"useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"]}. X accepts both datacenter and residential exits; residential is the safer default for long runs. Leave off for a direct connection.	{"useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"]}

Example input

{
  "startUrls": [
    "https://x.com/NASA",
    "climate change",
    "#worldcup",
    "search: python from:NASA min_retweets:50"
  ],
  "maxTweets": 100,
  "searchType": "latest",
  "timeWindowHours": 48,
  "authToken": "your_auth_token_cookie_value",
  "csrfToken": "your_ct0_cookie_value",
  "enrichAuthorProfiles": true,
  "onlyVerifiedAuthors": false,
  "minAuthorFollowers": 1000,
  "fullResolutionImages": true,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

Supported URL types and input formats

Every row in startUrls is parsed into either a profile target or a search target:

• Profile URL — https://x.com/elonmusk or https://twitter.com/elonmusk (and the www./mobile. host variants)
• Bare handle — @NASA or NASA (no @, single word matching X's username pattern)
• Keyword or phrase — climate change, world cup final — any input containing a space is treated as a search
• Hashtag — #NASA or https://x.com/hashtag/NASA
• X search URL — https://x.com/search?q=climate%20change&f=live — the query string is decoded and run as a search
• Advanced search syntax — search: python from:NASA min_retweets:50, passed through to X's own search operators
• Not supported — single-post links (https://x.com/user/status/12345...) are recognized and skipped with an uncharged unsupported_target row rather than silently dropped
• Not a profile — x.com/search, /hashtag, /explore, /i, /home, /settings and similar reserved path segments are routed to a search target instead of being misread as a @handle named after the path

⬆️ Output Format

Every run pushes one dataset row per collected post, plus one uncharged accounting row per target that could not be served. Twenty fields make up the default view; two more (author and authorProfile) are nested objects, and errorDetail appears only on accounting rows. Export as JSON, CSV or Excel from the Apify Console, or read the dataset through the Apify API.

Output for posts

Field	Type	Description
postId	string	The post's numeric ID
postUrl	string	Full https://x.com/{handle}/status/{id} URL
postText	string	Post body, taken from X's long-form note_tweet container when present, otherwise the standard full_text
timestamp	integer	Post creation time, epoch milliseconds
postedAt	string	Post creation time, ISO 8601 UTC
conversationId	string	The thread/conversation ID the post belongs to
media	array	Attached media; each item is {"mediaUrlHttps", "type", "id"}
author	object	Base author card (see below)
replyCount	integer	Reply count
quoteCount	integer	Quote count
repostCount	integer	Repost (retweet) count
favouriteCount	integer	Like count
isRetweet	boolean	true if the row is a repost of another post — X's own timeline silently substitutes the retweeting account's view, so this flag and retweetedBy recover what actually happened
retweetedBy	string | null	The reposting account's handle, when isRetweet is true
rowType	string	"post" for a real collected post, "target_status" for an uncharged accounting row
targetInput	string	The exact startUrls entry this row came from
targetType	string	"user", "search", or "unsupported"
errorReason	string | null	null on a normal post row; on an accounting row, one of unsupported_target, login_required, rate_limited, unavailable, server_error, network_error, redirected, filtered_out, empty
errorDetail	string	Present only on accounting rows (rowType: "target_status") — a human-readable explanation of errorReason
authorLookupStatus	string	"embedded" (default — data came from the post payload), "enriched" (verification tier applied), "not_found", "skipped", "enrich_failed", or "not_applicable" on accounting rows
authorProfile	object | null	Full author card, see below. null on accounting rows

The author object (base card, always present on post rows):

Field	Type	Description
name	string	Display name
screenName	string	@handle without the @
followersCount	integer	Follower count as of the post's payload
favouritesCount	integer	Likes given by the author
friendsCount	integer	Accounts the author follows
description	string	Bio text

Example output — post row

{
  "postText": "New imagery from the Perseverance rover just landed. Thread below.",
  "postUrl": "https://x.com/NASA/status/1834521098234567890",
  "timestamp": 1721822400000,
  "conversationId": "1834521098234567890",
  "postId": "1834521098234567890",
  "media": [
    { "mediaUrlHttps": "https://pbs.twimg.com/media/example.jpg", "type": "photo", "id": "1834521090000000000" }
  ],
  "author": {
    "name": "NASA",
    "screenName": "NASA",
    "followersCount": 96412873,
    "favouritesCount": 5231,
    "friendsCount": 312,
    "description": "Explore the universe and discover our home planet."
  },
  "replyCount": 812,
  "quoteCount": 240,
  "repostCount": 4103,
  "favouriteCount": 28540,
  "postedAt": "2026-07-24T14:00:00Z",
  "isRetweet": false,
  "retweetedBy": null,
  "rowType": "post",
  "targetInput": "@NASA",
  "targetType": "user",
  "errorReason": null,
  "authorLookupStatus": "enriched",
  "authorProfile": {
    "authorId": "11348282",
    "authorHandle": "NASA",
    "authorName": "NASA",
    "authorProfileUrl": "https://x.com/NASA",
    "authorBio": "Explore the universe and discover our home planet.",
    "authorLocation": "Washington, DC",
    "authorWebsite": "https://www.nasa.gov",
    "authorFollowers": 96412873,
    "authorFollowing": 312,
    "authorPosts": 76221,
    "authorMediaCount": 21033,
    "authorListedCount": 148021,
    "authorLikesGiven": 5231,
    "authorCreatedAt": "2007-06-01T20:31:00Z",
    "authorCreatedAtTimestamp": 1180729860000,
    "authorAccountAgeDays": 7002,
    "authorIsBlueVerified": true,
    "authorVerifiedType": "Government",
    "authorIsProtected": false,
    "authorProfileImageUrl": "https://pbs.twimg.com/profile_images/example_400x400.jpg",
    "authorProfileImageUrlSmall": "https://pbs.twimg.com/profile_images/example_normal.jpg",
    "authorBannerUrl": "https://pbs.twimg.com/profile_banners/11348282/example/1500x500",
    "authorPinnedPostIds": ["1830000000000000000"],
    "authorProfessionalType": null,
    "authorAffiliation": null,
    "authorIsIdentityVerified": true,
    "authorVerifiedSince": "2023-04-11T00:00:00Z",
    "authorVerificationReason": "Government organization",
    "authorAffiliatesCount": 0,
    "authorCreatorSubscriptions": null,
    "authorBirthdate": null
  }
}

Output for author profiles

The authorProfile object is the same shape whether it comes from a post row or is browsed through the dedicated authors dataset view (postUrl, authorProfile, authorLookupStatus). Twenty-five fields require no extra request (Tier A — read straight from the payload the Actor already fetched); six more (Tier B) are populated only when enrichAuthorProfiles is true, via one guest-mode lookup per distinct author.

Tier A — always present at zero extra cost:

authorId, authorHandle, authorName, authorProfileUrl, authorBio, authorLocation, authorWebsite (t.co links pre-resolved to their real destination), authorFollowers, authorFollowing, authorPosts, authorMediaCount, authorListedCount, authorLikesGiven, authorCreatedAt, authorCreatedAtTimestamp, authorAccountAgeDays, authorIsBlueVerified, authorVerifiedType (e.g. "Government", "Business", or null), authorIsProtected, authorProfileImageUrl, authorProfileImageUrlSmall, authorBannerUrl, authorPinnedPostIds (array), authorProfessionalType, authorAffiliation (object with name, url, badgeUrl, or null).

Tier B — requires enrichAuthorProfiles: true, one guest request per distinct author:

authorIsIdentityVerified, authorVerifiedSince, authorVerificationReason, authorAffiliatesCount, authorCreatorSubscriptions, authorBirthdate (returned as YYYY-MM-DD or --MM-DD when X only discloses month and day; null when not disclosed).

Every field the deep lookup can't resolve, or that the embedded payload left empty, stays null rather than being omitted — the key is always present.

Schema stability and status rows

Field names stay stable across runs: this Actor reads X's structured GraphQL responses rather than parsing rendered HTML, and the live credential-discovery step re-fetches X's current bearer token and query IDs on every run so a routine front-end deploy doesn't quietly break field extraction. If X retires a query ID the Actor was using, it logs the fallback and continues on the last known-good ID rather than failing the run.

A row with rowType: "target_status" is never a post — it's a typed record of why a target produced nothing: login_required (keyword/hashtag/search-URL target without cookies), rate_limited, unavailable (surface returned an unexpected shape), server_error, network_error, redirected, filtered_out (posts existed but no author passed your onlyVerifiedAuthors / minAuthorFollowers filters), empty (X answered normally with zero matching posts), or unsupported_target (a single-post link). These rows are never billed — only rows with rowType: "post" are charged, under the row_result pay-per-event.

X (Twitter) Posts Search Strategy Guide

Three ways teams actually put this Actor into production, beyond a one-off manual run.

Strategy 1: Real-time enrichment pipeline

Trigger a run per new lead or account you need qualified — a list of X handles from a signup form or CRM import, as startUrls, with enrichAuthorProfiles: true. Append authorFollowers, authorVerifiedType, authorAccountAgeDays and authorBio back onto the source record, keyed by authorHandle, and write the enriched record to your CRM or lead-scoring table. A verified Government/Business account or a 15-year-old account with 50,000 followers scores very differently from a week-old account with 12 followers — and both are visible in the same run without a second lookup.

Strategy 2: Scheduled monitoring and alerting

Set searchType: "latest", a timeWindowHours window matching your check interval, and minAuthorFollowers to cut noise, then run this Actor on an Apify Schedule at that same interval. Diff each new run's postId values against the previous run's to isolate genuinely new posts, then alert on the delta where authorVerifiedType is set or authorFollowers crosses your threshold — the two fields that separate a story worth escalating from routine mention volume.

Strategy 3: Bulk dataset build

Feed a long startUrls list — mixed handles and keyword segments — with maxTweets set high (up to 10,000 per target), and split the list across parallel Actor runs by segment (one run per keyword theme, or per account cohort). Aggregate each run's dataset export to CSV, or load it into a database, for a research or training corpus. X's own search endpoint is the real ceiling here, not this Actor: it allows a documented amount of search traffic per 15-minute window per authenticated X account, which the Actor logs as it consumes it — spread keyword-heavy segments across separate authToken/csrfToken pairs if you need more throughput than one X account's window allows.

Strategy comparison at a glance

Strategy	Best for	Run pattern	Output format
Real-time enrichment	CRM / lead-scoring pipelines	On-demand run per new batch	JSON via API, written back per row
Scheduled monitoring	Brand and keyword alerting	Recurring Apify Schedule, diffed on postId	Dataset diff computed downstream, alert on delta
Bulk dataset build	Research or training corpora	Parallel runs split by keyword/account segment	Aggregated CSV/Excel export, or database load

🔗 Related X (Twitter) Scrapers & Tools

Other Scrapio X/Twitter Actors, plus the closest cross-platform equivalents when your target isn't X.

Scraper	What it extracts
Twitter Trends Scraper - Brand & Keyword Monitoring	Live X trending topics, watched against a brand keyword list
Twitter X Posts Scraper By Keyword & Hashtag Search	Posts by keyword and hashtag, without the author-profile enrichment layer
Twitter X Reply Scraper: Reply Sentiment & Tone Breakdown	Replies to a given post, with sentiment and tone scoring
Bluesky Scraper (angaba92/bluesky-scraper)	The same posts/profile/search shape on Bluesky's AT Protocol
Threads Scraper (futurizerush/meta-threads-scraper)	The same posts/profile/search shape on Meta's Threads

How to integrate this Actor with your stack

This Actor works with any language or tool that can make an HTTP request through the Apify API — you are not limited to the Console UI.

Python

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run_input = {
    "startUrls": ["@NASA", "climate change"],
    "maxTweets": 100,
    "searchType": "latest",
    "timeWindowHours": 24,
    "authToken": "<YOUR_X_AUTH_TOKEN_COOKIE>",
    "csrfToken": "<YOUR_X_CT0_COOKIE>",
    "enrichAuthorProfiles": True,
    "minAuthorFollowers": 1000,
}

run = client.actor("<YOUR_USERNAME>/x-twitter-posts-search-author-profile-details").call(run_input=run_input)

for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    if row["rowType"] != "post":
        continue
    profile = row["authorProfile"] or {}
    print(row["postUrl"], profile.get("authorHandle"), profile.get("authorFollowers"), row["favouriteCount"])

Node.js

const { ApifyClient } = require('apify-client');

const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });

const run = await client.actor('<YOUR_USERNAME>/x-twitter-posts-search-author-profile-details').call({
    startUrls: ['@NASA', 'climate change'],
    maxTweets: 50,
    searchType: 'top',
    onlyVerifiedAuthors: true,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const row of items) {
    if (row.rowType !== 'post') continue;
    console.log(row.postUrl, row.authorProfile?.authorHandle, row.authorProfile?.authorFollowers);
}

Async and scheduled pipelines

For large or recurring jobs, don't wait on the run synchronously — start it, and poll client.run(runId).get() for status, or check the dataset's item count directly through the API. Apify Schedules trigger a run automatically on a cron expression from the Console, and an Apify webhook can fire on run completion independently of the Actor's own code, so a monitoring pipeline needs no polling loop at all.

Who Needs This X (Twitter) Scraper? (Use Cases & Industries)

Social listening and brand teams

A marketing team runs search: "#YourBrand" min_faves:5 with searchType: "latest", timeWindowHours: 12 and minAuthorFollowers: 500, then routes every row where authorVerifiedType is set to a priority Slack channel — a verified account amplifying the brand is worth a same-day response.

AI training data and RAG teams

An AI engineer indexes postText and authorBio from a topic-keyword search into a vector store, so an agent answering questions about public sentiment on a topic retrieves attributable, sourced text (postUrl) instead of a paraphrase with no citation trail.

Competitive and market intelligence teams

An analyst runs a profile target against a competitor's account with enrichAuthorProfiles on, tracking authorFollowers growth and favouriteCount/repostCount per post over weekly scheduled runs to benchmark engagement against their own account's numbers.

Researchers

A communications researcher builds a corpus from a set of hashtag and keyword targets, using retweetedBy to map amplification networks and authorAccountAgeDays to separate established accounts from ones created just before a trending event — public data only, no login-gated content collected.

Product and SaaS builders

A lead-scoring SaaS product runs this Actor against prospect handles nightly, feeding authorFollowers, authorVerifiedType and authorAccountAgeDays into its own scoring model as one signal among several, with no separate profile-lookup integration needed beyond this single Actor.

Is it legal to scrape X (Twitter) posts and profiles?

Scraping publicly accessible social media data is broadly permitted in the United States: in hiQ Labs, Inc. v. LinkedIn Corp., 9th Cir. (2019, reaffirmed on remand 2022), the Ninth Circuit held that accessing data a website makes publicly viewable, without needing to bypass a login, does not violate the Computer Fraud and Abuse Act. More specific to this platform, in X Corp. v. Bright Data Ltd., N.D. Cal. (2024), the court largely rejected X's attempt to use its terms of service and the CFAA to block a third party from scraping publicly available X data, though the ruling turned on the specific claims argued and is not a blanket clearance for every scraping use case.

Two separate questions matter here. Scraping public data has real, favorable precedent behind it in the US, as above — but precedent varies by jurisdiction, and this is not legal advice for your situation. Violating X's Terms of Service is a contractual/civil risk, not a criminal one, and is a separate question from whether the CFAA is implicated. Data protection obligations do apply to this Actor's output: author profile fields — name, bio, location, follower count and, when the verification tier is enabled, birthdate — are personal data under GDPR and similar regimes when tied to an identifiable person, even though X displays them publicly.

This Actor returns only publicly accessible data. What you do with that data is your responsibility — consult legal counsel for commercial applications involving personal data.

❓ Frequently Asked Questions

Does this Actor work without an X account?

Partially. Profile (@handle) targets work with no cookies at all — the Actor runs as a logged-out guest. Keyword, hashtag and search-URL targets need your own X session (authToken + csrfToken), because X answers logged-out keyword search with an HTTP 404 with no rows. Without cookies, a keyword target is reported as login_required and is not charged.

How does this Actor handle X's anti-scraping measures?

It discovers X's own current bearer token and GraphQL query IDs from the live main.*.js bundle on every run rather than relying only on hardcoded values, and classifies every response by resolved final URL plus the presence of the expected result container — not a loose text match — so a wrong-verb 404 is never mistaken for "no data." Failed requests retry up to three times with exponential backoff, five times longer when the failure is a rate limit. Apify Proxy (datacenter or residential) is optional and configurable per run.

Can I run this at scale without getting blocked?

X's own search endpoint enforces a documented rate limit shared across every run using the same X account — the Actor logs remaining quota as it consumes it. When onlyVerifiedAuthors or minAuthorFollowers is set, the Actor automatically scans a wider window of posts per target (up to ten times maxTweets) so the filters don't silently starve your result count. There is no published uptime or success-rate figure for this Actor beyond what's described here.

How fresh is the data this Actor returns?

Live, per run. Every post and author field is fetched from X at the moment the run executes — nothing is served from a stored cache between runs. The one exception is within a single run: the optional verification-tier lookup caches a resolved author for the duration of that run so a post-heavy target from one prolific author only triggers one guest request.

Does this Actor return personal data, and who is responsible for using it lawfully?

Yes — author name, bio, location, follower count and (when enrichAuthorProfiles is on) birthdate are personal data when identifiable. The Actor returns only what X already displays publicly on the post or profile; it does not access private accounts or login-gated fields. Lawful basis for storing and using that data — especially at commercial scale — sits with you, not with the Actor.

Which fields work best for AI training data and RAG indexing?

For RAG: postText is the primary indexable content, with authorBio and authorName for attribution context — pair every chunk with postUrl so a retrieval answer can cite its source. For training data: favouriteCount, repostCount, replyCount, authorFollowers and authorAccountAgeDays are numeric fields with a consistent shape across every row. Every field returns as a typed primitive, array or nested object — no string-to-number parsing required before indexing.

Does this Actor work with Claude, ChatGPT and other AI agent tools?

Yes, as a standard HTTP endpoint through the Apify API — any agent framework that can make a request (LangChain, CrewAI, a hand-written tool definition, n8n) can call it and get typed JSON straight back, with no HTML parsing step in the loop.

How does this Actor compare to other X (Twitter) scrapers?

Checked on the Apify Store, 25 July 2026: scraper_one/x-profile-posts-scraper is a simpler, profile-only scraper — its own README documents profileUrls, a 1–200 result cap (default 5), and a six-field author object with no verification data and no keyword search. This Actor covers the same profile-timeline case plus keyword, hashtag and search-URL targets in the same run, and attaches up to 31 author fields including verification status, at up to 10,000 results per target. Where scraper_one's Actor may suit you better: if all you need is a handful of recent posts from one known account with no author enrichment, its narrower input surface is simpler to configure for that single case.

ℹ️ Disclaimer

This Actor extracts only publicly available data from X (Twitter). It is intended for lawful use cases only. Users are responsible for complying with X's Terms of Service and applicable data protection laws in their jurisdiction, particularly when storing or processing author profile fields that constitute personal data.

💬 Your Feedback

Found a bug, or need a field that X's payload carries but this Actor doesn't yet surface? Open an issue on the Actor's Issues tab. A report that includes the exact input you ran and the target (handle, keyword or URL) is the fastest to reproduce and fix.