X (Twitter) Scraper

Collect public X (Twitter) data into a structured Apify dataset. Export JSON, CSV, Excel or JSONL from the dataset. Managed provider access is included; customers do not supply API keys.

Provider access is included; no provider API key is required in input.

Quick start

{
    "mode": "profiles",
    "profiles": ["NASA"],
    "maxItems": 1,
    "maxRequests": 2
}

Choose a mode, supply only the matching source field, set result/request caps, and run. Empty profiles input in the default profile mode uses the example account. Clear the prefilled profiles when switching to other source types.

Collection modes

Mode	Collects	Source field	Pagination
profiles	Profiles	profiles	Single response
profilesById	Profiles by ID	userIds	Single response
posts	Profile posts	profiles	Yes
media	Profile media posts	profiles	Yes
likes	Liked posts	profiles	Yes
followers	Followers	profiles	Yes
following	Following	profiles	Yes
postsByUrl	Posts by URL or ID	postUrls	Single response
postDetails	Post detail	postUrls	Single response
likers	Post likers	postUrls	Yes
reposters	Post reposters	postUrls	Yes
listPosts	List posts	listIds	Yes
followerIds	Follower IDs	userIds	Yes
followingIds	Following IDs	userIds	Yes
trends	Trends	locations	Single response

Output

Every row includes an exact string ID, data type, source, URL, username, display name, text, timestamp and available counts. Unavailable values are null, not invented zeroes. mediaUrls contains links; files are not downloaded. includeRaw adds the original record. IDs are never converted to floating point numbers. The OUTPUT key-value record contains counts, stop reason, requests, estimated weighted units, retries and warnings.

Rows are deduplicated by type + ID across sources. If the same item appears in multiple sources, the first source is retained. A run can stop with item-limit, request-budget, charge-limit, or usage-limit; it may contain partial results. Provider failures mark the run failed while preserving saved data. Aborted runs record their summary.

Pricing

Operation modes	USD / 1,000 saved results	Reference
posts, media, likes, listPosts	0.225	Kaito tweets, Free plan
profiles, profilesById	3.96	API Dojo user scraper: $0.004 profile query + $0.0004 result
postsByUrl, postDetails	0.81	Axery post detail, Free plan (excluding start fee)
followers, following, likers, reposters, followerIds, followingIds	0.135	Kaito followers/following; adjacent modes use the same group rate
trends	0.36	Karamelo trends, Free plan

Each saved record triggers exactly one operation event. $1.00 per 1,000 empty-check units applies only to validated successful empty responses, including configured endpoint weights. No start fee. Errors, malformed data, duplicates and filtered records do not create empty fees. Lookups used to resolve usernames are not separately charged.

Prices are fixed across Apify plans and are at most 10% below the named comparator's baseline result rate, as checked on September 28, 2026. This compares result rates, not total invoices: competitors may have extra start/query fees or paid-plan discounts. Adjacent operations use the stated group rate, not a claim of an exact feature match.

Set a maximum run charge as well as request and result caps. A request may return many rows; the request cap includes lookups and retries. Endpoint weights default to one pending metering reconciliation. Provider access is included; see the Apify pricing configuration for platform usage billing.

Limitations

X keyword search is not available in this provider API. Likes, likers and reposters can be restricted by X and returned empty; emptiness is not evidence of no activity. Post detail exports the requested post; it does not expand the replies array. Some provider endpoints ignore count, so limits are enforced locally.

Only public, provider-accessible data is supported. There is no login, access to private accounts, deleted-content recovery, or content download. No short-link resolution. No guarantees of complete history or exhaustive follower lists; provider pagination and availability control coverage.

Local development

Node 22 or newer and Redis (for integration tests):

npm ci
npm run check
npm run publication:check
node --env-file=.env --import tsx src/main.ts

Copy INPUT.example.json to storage/key_value_stores/default/INPUT.json for local input. .env and samples/raw/ are ignored. Required owner configuration: RAPIDMINE_API_KEY; cloud runs additionally require Redis URL or Upstash REST URL/token. Local runs use the request cap and per-process rate; shared cloud usage counters are enforced on Apify. Never commit credentials or put them in the actor input.

The shared provider cap is 90,000 weighted units per internal 30-day guard window, with at most 60 requests per minute across all runs; paid user cap 1,000/day, free user 3/window and shared free pool 10/window. These internal windows start on first use and do not mirror provider billing dates. Unknown/free status uses the conservative free allowance. Redis errors stop calls before provider access.

Monitoring and deployment

PROVIDER_MONITOR_URL and PROVIDER_MONITOR_TOKEN optionally publish usage observations to the local dashboard collector. Cloud actors require a reachable collector URL; localhost on your laptop is not reachable from Apify. Monitoring failure does not fail a scrape.

See publication/READINESS.md, publication/PUBLISH.md and docs/CONTRACTS.md for verification evidence and release details.

Related scrapers for brand and audience research

Combine public X posts and profiles with other sources when researching a brand, creator or topic.

• TikTok Scraper — Collect video metadata, creator profiles and comments to compare social content and engagement.
• Reddit Scraper — Search Reddit posts and retrieve comments to explore longer discussions around the same topic.
• Similarweb Quick Scraper — Add estimated website traffic, traffic sources and geographic reach for relevant domains.

Run each Actor separately and combine its exported data in your own workflow. Each Actor has its own input, output and pricing.