X / Twitter Profile Timeline Scraper

Designed for cheap, fast scraping of complete accessible profile timelines within your chosen date range. Collect posts from one or multiple X profiles, including captions, engagement counts, followers, photos, and video links. Results appear during the run and are ready to export as CSV or JSON.

No X login, cookies, or API keys are required from you. Account sessions are managed by the Actor owner. Just enter profiles, a cutoff date, and any post limits you want.

For a complete collection within your date range, leave both post limits blank and allow the run to finish. Coverage is limited to posts X makes available in the profile’s Posts timeline; deleted or inaccessible posts and replies absent from that timeline cannot be included.

Why use this scraper?

• Built to keep runs cheap and fast: unnecessary image, video, and font downloads are blocked; media is returned as links.
• Multiple profiles in one run: paste handles or X/Twitter profile links, one per line.
• Live results: posts arrive in your dataset as each page is processed.
• Flexible limits: cap posts per profile, across the whole run, or continue to your cutoff date.
• Useful media links: original-size photos and the highest-bitrate MP4 available in the response.
• Ready for analysis: download CSV or JSON for spreadsheets, reporting, and your own tools.

Actual speed and cost depend on the number of posts, date range, and rate-limit waits. This is a profile timeline scraper, not a keyword search or replies scraper.

Get started

1. Paste your profiles into Profiles, one per line.
2. Set Scrape till to the oldest date you want included.
3. Optionally set a per-profile limit, a total limit, or both.
4. Click Start and view your results in Output.
5. Use Export to download CSV or JSON.

Accepted profile formats include:

NASA
@SpaceX
http://x.com/@username
https://twitter.com/another_username
www.x.com/username
mobile.twitter.com/username

Duplicate handles are removed automatically, regardless of capitalization. Blank lines are ignored. Use profile links rather than links to individual posts.

Choose your inputs

Input	What to enter
Profiles (profiles)	Handles or profile links, one per line. Required.
Scrape till (scrapeTill)	The oldest date or datetime to include. Required. Dates use UTC.
Maximum posts per profile (maxPostsPerProfile)	Optional cap for each profile. Leave blank for no per-profile cap.
Maximum posts across all profiles (maxPostsTotal)	Optional cap for the entire run. Leave blank for no total cap.

The scraper starts with the latest posts and works backwards. For example, 2026-09-09 includes eligible posts from September 9 at 00:00 UTC up to the time the run starts. It does not mean September 9 only. You can also enter an exact datetime, such as 2026-09-09T00:00:00+05:30 for midnight in India.

Both limits apply when provided. Use positive whole numbers; leave a limit blank instead of entering zero. Profiles run in input order, so later profiles may be skipped once the total cap is reached.

Example: 100 posts per profile, 150 overall

{
  "profiles": "https://x.com/NASA\nhttps://twitter.com/SpaceX",
  "scrapeTill": "2026-09-09",
  "maxPostsPerProfile": 100,
  "maxPostsTotal": 150
}

If the first profile supplies 100 eligible posts, the second can contribute at most 50. If the first supplies only 20, the second can still contribute up to its own limit of 100. Results can be fewer when the cutoff or available timeline is reached.

Example: collect the accessible timeline back to a date

{
  "profiles": "NASA\nSpaceX",
  "scrapeTill": "2026-09-01"
}

With both limits omitted, collection continues towards the cutoff, subject to available posts and the run’s time budget.

What you get

Column	Description
Profile	Profile whose timeline was scraped.
Post_ID	Post or repost event ID. Keep it as text in spreadsheets.
Post_Caption	Post text, including long-form text when available.
Post_Url	Direct link to the post or repost event.
Post_Date	Post or repost event datetime in UTC.
Shares	Repost count.
Comments	Reply count.
Likes	Like count.
Views	View count, when available.
Engagement	Likes + reposts + replies.
Name	Content author’s display name.
Username	Content author’s handle.
Followers	Content author’s follower count, when available.
thumbnail	First media preview image, when available.
content_type	text, image, video, gif, or mixed.
media_url	JSON-formatted text containing photo and video URLs.

For reposts, Name, Username, and Followers describe the original author; Profile identifies the timeline you requested. Counts reflect collection time, not historical values. Unavailable values remain blank or null. The same post on two requested timelines counts as two output rows.

Photos and videos

The media_url column contains JSON-formatted text, using arrays to preserve multiple attachments. For example, with illustrative URLs:

{
  "photo": ["https://pbs.twimg.com/media/IMAGE_ID?format=jpg&name=orig"],
  "video": ["https://video.twimg.com/example.mp4"]
}

Photos request original size; videos use the highest-bitrate available MP4. Animated GIFs appear as MP4 links under video. Posts without usable media URLs contain {}. Files are linked, not downloaded.

See results while the scraper runs

Each processed page is uploaded to the dataset without waiting for the entire run to finish. Refresh the Output preview if it has not updated yet.

The diagnostic summary is refreshed periodically during processing and at profile completion, so its count can briefly lag the live dataset.

Progress logs identify the profile being processed:

username=@NASA responses=2 scanned=40 saved=21 min_date=2026-09-10

scanned counts unique posts inspected; saved counts posts kept for that profile. These numbers can differ because of the cutoff and post limits. min_date is the oldest non-pinned date in that response, before filtering.

Memory, rate limits, and incomplete runs

The Actor uses 128 MB RAM and direct HTTP requests, with no Chromium process. It handles one page at a time and stores duplicate IDs on disk. Give the run enough time for all profiles and possible rate-limit waits. Requests remain sequential.

When X rate-limits an account, the scraper rotates to another available account. If all accounts are cooling down, it waits. These waits can increase runtime and cost.

If a profile fails, other profiles are still attempted when possible. Already uploaded results remain available even if the run ends with a failure. Check the SUMMARY record in the run’s key-value store for per-profile counts and stop reasons. A storage failure stops the run to avoid unsafe retries.

Unavailable profiles (profile_unavailable) produce a warning listing the affected handles in the run log and final status message, instead of failing the run. They remain marked incomplete in SUMMARY so unsuccessful collections are visible. Other incomplete profiles and errors still cause the run to fail.

Run outputs contain scraped posts and a customer-facing SUMMARY. Internal account status is synchronized only to the owner's account store; no SESSION_STATUS record is written to the run store. Public progress messages omit account identities, rotation details, and cooldown schedules. Internal failure reasons are reported as collection_failed in the summary.

For complete date-range collection, inspect the summary: a post cap, timeout, stall, or error can stop a profile before the cutoff. Timeline ordering and X’s available history also affect coverage. After a failed or interrupted run, start a fresh run; automatic continuation across Actor restarts or migrations is not supported.

For the Actor owner

Deployment, account setup, and development instructions are in DEPLOYMENT.md. Customers do not need to follow those steps.