# Twitter X profile timeline scraper (`plummy_bookbag/x-profile-timeline-scraper`) Actor

Scrape multiple X profiles from latest posts to a UTC cutoff with optional per-profile and total post limits.

- **URL**: https://apify.com/plummy\_bookbag/x-profile-timeline-scraper.md
- **Developed by:** [Mark Pandit](https://apify.com/plummy_bookbag) (community)
- **Stats:** 1 total users, 1 monthly users, 95.5% runs succeeded, 0 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

from $0.65 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## X / Twitter Profile Timeline Scraper

**Designed for cheap, fast scraping of complete accessible profile timelines within your chosen date range.** Collect posts from one or multiple X profiles, including captions, engagement counts, followers, photos, and video links. Results appear during the run and are ready to export as CSV or JSON.

**No X login, cookies, or API keys are required from you.** Account sessions are managed by the Actor owner. Just enter profiles, a cutoff date, and any post limits you want.

For a complete collection within your date range, leave both post limits blank and allow the run to finish. Coverage is limited to posts X makes available in the profile’s Posts timeline; deleted or inaccessible posts and replies absent from that timeline cannot be included.

### Why use this scraper?

- **Built to keep runs cheap and fast:** unnecessary image, video, and font downloads are blocked; media is returned as links.
- **Multiple profiles in one run:** paste handles or X/Twitter profile links, one per line.
- **Live results:** posts arrive in your dataset as each page is processed.
- **Flexible limits:** cap posts per profile, across the whole run, or continue to your cutoff date.
- **Useful media links:** original-size photos and the highest-bitrate MP4 available in the response.
- **Ready for analysis:** download CSV or JSON for spreadsheets, reporting, and your own tools.

Actual speed and cost depend on the number of posts, date range, and rate-limit waits. This is a profile timeline scraper, not a keyword search or replies scraper.

### Get started

1. Paste your profiles into **Profiles**, one per line.
2. Set **Scrape till** to the oldest date you want included.
3. Optionally set a per-profile limit, a total limit, or both.
4. Click **Start** and view your results in **Output**.
5. Use **Export** to download CSV or JSON.

Accepted profile formats include:

```text
NASA
@SpaceX
http://x.com/@username
https://twitter.com/another_username
www.x.com/username
mobile.twitter.com/username
```

Duplicate handles are removed automatically, regardless of capitalization. Blank lines are ignored. Use profile links rather than links to individual posts.

### Choose your inputs

| Input | What to enter |
| --- | --- |
| **Profiles** (`profiles`) | Handles or profile links, one per line. Required. |
| **Scrape till** (`scrapeTill`) | The oldest date or datetime to include. Required. Dates use UTC. |
| **Maximum posts per profile** (`maxPostsPerProfile`) | Optional cap for each profile. Leave blank for no per-profile cap. |
| **Maximum posts across all profiles** (`maxPostsTotal`) | Optional cap for the entire run. Leave blank for no total cap. |

The scraper starts with the latest posts and works backwards. For example, `2026-09-09` includes eligible posts from **September 9 at 00:00 UTC up to the time the run starts**. It does not mean September 9 only. You can also enter an exact datetime, such as `2026-09-09T00:00:00+05:30` for midnight in India.

Both limits apply when provided. Use positive whole numbers; leave a limit blank instead of entering zero. Profiles run in input order, so later profiles may be skipped once the total cap is reached.

#### Example: 100 posts per profile, 150 overall

```json
{
  "profiles": "https://x.com/NASA\nhttps://twitter.com/SpaceX",
  "scrapeTill": "2026-09-09",
  "maxPostsPerProfile": 100,
  "maxPostsTotal": 150
}
```

If the first profile supplies 100 eligible posts, the second can contribute at most 50. If the first supplies only 20, the second can still contribute up to its own limit of 100. Results can be fewer when the cutoff or available timeline is reached.

#### Example: collect the accessible timeline back to a date

```json
{
  "profiles": "NASA\nSpaceX",
  "scrapeTill": "2026-09-01"
}
```

With both limits omitted, collection continues towards the cutoff, subject to available posts and the run’s time budget.

### What you get

| Column | Description |
| --- | --- |
| `Profile` | Profile whose timeline was scraped. |
| `Post_ID` | Post or repost event ID. Keep it as text in spreadsheets. |
| `Post_Caption` | Post text, including long-form text when available. |
| `Post_Url` | Direct link to the post or repost event. |
| `Post_Date` | Post or repost event datetime in UTC. |
| `Shares` | Repost count. |
| `Comments` | Reply count. |
| `Likes` | Like count. |
| `Views` | View count, when available. |
| `Engagement` | Likes + reposts + replies. |
| `Name` | Content author’s display name. |
| `Username` | Content author’s handle. |
| `Followers` | Content author’s follower count, when available. |
| `thumbnail` | First media preview image, when available. |
| `content_type` | `text`, `image`, `video`, `gif`, or `mixed`. |
| `media_url` | JSON-formatted text containing photo and video URLs. |

For reposts, `Name`, `Username`, and `Followers` describe the original author; `Profile` identifies the timeline you requested. Counts reflect collection time, not historical values. Unavailable values remain blank or null. The same post on two requested timelines counts as two output rows.

#### Photos and videos

The `media_url` column contains JSON-formatted **text**, using arrays to preserve multiple attachments. For example, with illustrative URLs:

```json
{
  "photo": ["https://pbs.twimg.com/media/IMAGE_ID?format=jpg&name=orig"],
  "video": ["https://video.twimg.com/example.mp4"]
}
```

Photos request original size; videos use the highest-bitrate available MP4. Animated GIFs appear as MP4 links under `video`. Posts without usable media URLs contain `{}`. Files are linked, not downloaded.

### See results while the scraper runs

Each processed page is uploaded to the dataset without waiting for the entire run to finish. Refresh the Output preview if it has not updated yet.

The diagnostic summary is refreshed periodically during processing and at profile completion, so its count can briefly lag the live dataset.

Progress logs identify the profile being processed:

```text
username=@NASA responses=2 scanned=40 saved=21 min_date=2026-09-10
```

`scanned` counts unique posts inspected; `saved` counts posts kept for that profile. These numbers can differ because of the cutoff and post limits. `min_date` is the oldest non-pinned date in that response, before filtering.

### Memory, rate limits, and incomplete runs

The Actor uses **128 MB RAM** and direct HTTP requests, with no Chromium process. It handles one page at a time and stores duplicate IDs on disk. Give the run enough time for all profiles and possible rate-limit waits. Requests remain sequential.

When X rate-limits an account, the scraper rotates to another available account. If all accounts are cooling down, it waits. These waits can increase runtime and cost.

If a profile fails, other profiles are still attempted when possible. Already uploaded results remain available even if the run ends with a failure. Check the **`SUMMARY` record in the run’s key-value store** for per-profile counts and stop reasons. A storage failure stops the run to avoid unsafe retries.

Unavailable profiles (`profile_unavailable`) produce a warning listing the affected handles in the run log and final status message, instead of failing the run. They remain marked `incomplete` in `SUMMARY` so unsuccessful collections are visible. Other incomplete profiles and errors still cause the run to fail.

Run outputs contain scraped posts and a customer-facing `SUMMARY`. Internal account status is synchronized only to the owner's account store; no `SESSION_STATUS` record is written to the run store. Public progress messages omit account identities, rotation details, and cooldown schedules. Internal failure reasons are reported as `collection_failed` in the summary.

For complete date-range collection, inspect the summary: a post cap, timeout, stall, or error can stop a profile before the cutoff. Timeline ordering and X’s available history also affect coverage. After a failed or interrupted run, start a fresh run; automatic continuation across Actor restarts or migrations is not supported.

### For the Actor owner

Deployment, account setup, and development instructions are in [DEPLOYMENT.md](DEPLOYMENT.md). Customers do not need to follow those steps.

# Actor input Schema

## `profiles` (type: `string`):

One handle or profile URL per line. Accepts username, @username, http://x.com/@username, https://twitter.com/username, www/mobile variants and URLs without a scheme. Duplicate handles are removed.

## `scrapeTill` (type: `string`):

Start at the latest posts and go backwards to this date/time. Example: 2026-09-01 includes posts from September 1 at 00:00 UTC up to run start. Accepts ISO datetimes too.

## `maxPostsPerProfile` (type: `integer`):

Maximum posts to save from each profile. Leave blank to disable this limit. Stops sooner if the date cutoff or total limit is reached.

## `maxPostsTotal` (type: `integer`):

Maximum combined dataset rows. Profiles run in input order; later profiles may be skipped when this limit is reached. Leave blank to disable.

## Actor input object example

```json
{
  "profiles": "https://x.com/NASA",
  "scrapeTill": "2026-09-01",
  "maxPostsPerProfile": 5,
  "maxPostsTotal": 100
}
```

# Actor output Schema

## `posts` (type: `string`):

Posts saved to the default dataset, including source profile, post text, engagement counts, and available media links.

## `summary` (type: `string`):

Date range, configured post limits, total saved posts, and per-profile counts, completion status, and stop reasons, including unavailable profiles.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "profiles": "https://x.com/NASA",
    "scrapeTill": "2026-09-01",
    "maxPostsPerProfile": 5,
    "maxPostsTotal": 100
};

// Run the Actor and wait for it to finish
const run = await client.actor("plummy_bookbag/x-profile-timeline-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "profiles": "https://x.com/NASA",
    "scrapeTill": "2026-09-01",
    "maxPostsPerProfile": 5,
    "maxPostsTotal": 100,
}

# Run the Actor and wait for it to finish
run = client.actor("plummy_bookbag/x-profile-timeline-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "profiles": "https://x.com/NASA",
  "scrapeTill": "2026-09-01",
  "maxPostsPerProfile": 5,
  "maxPostsTotal": 100
}' |
apify call plummy_bookbag/x-profile-timeline-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,plummy_bookbag/x-profile-timeline-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/aXWkOw7wNk2G1wB1X/builds/LKeRKGSXajZZRpmOu/openapi.json