# X / Twitter Scraper — Posts, Profiles & Monitoring (`yearly_register/x-twitter-scraper-dev`) Actor

Scrape public X/Twitter profiles, posts, paginated public timelines and visible replies without login or cookies. Monitor new posts and public metric changes.

- **URL**: https://apify.com/yearly\_register/x-twitter-scraper-dev.md
- **Developed by:** [Automation Tech](https://apify.com/yearly_register) (community)
- **Categories:** Social media
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.50 / 1,000 post results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## X / Twitter Scraper — Posts, Profiles & Monitoring

Scrape public X/Twitter profiles, posts and paginated profile timelines without an X login. Capture visible replies from public post pages and monitor the same targets for new posts and public metric changes. No X cookies or account setup required.

### Quick start

```json
{ "targets": ["@NASA"] }
```

The default collects up to 25 posts per profile, plus the profile record. You can mix public handles, profile URLs and post URLs:

```json
{
  "targets": [
    "@NASA",
    "https://x.com/OpenAI",
    "https://x.com/NASA/status/2102882545427714266"
  ],
  "maxPosts": 100,
  "maxPages": 6,
  "scrapeReplies": true
}
```

Supported targets are a valid handle (`@NASA` or `NASA`), an X/Twitter profile URL, a post URL, or explicit `profile:` and `post:` prefixes. Search terms, hashtags, reply/media/repost timeline prefixes, follower/following targets and Lists are rejected with a clear error. Media, quotes and repost relationships can still appear in posts discovered through supported targets.

### Ready-made Tasks

The release includes five reusable Apify Tasks built only around surfaces verified without an X account:

- **Profile & Recent Timeline** — profile + bounded public timeline pagination
- **Post & Visible Replies** — one public post plus replies exposed by the logged-out page
- **Profile Monitor** — new posts and public metric/profile changes across scheduled runs
- **Multi-Profile Research** — scrape several public profiles in one deduplicated run
- **High-Engagement Posts** — filter a public profile timeline by engagement thresholds

No Task asks for X cookies, credentials, OAuth, or account login. Search/followers/Lists Tasks are intentionally not offered because those surfaces are not currently reliable logged-out.

### What comes back

- **Profile** records: stable user ID when exposed, handle, display name, biography, public counts, verification flags, images and other available metadata.
- **Post** records: canonical URL, text, timestamp, author, public engagement counts, media URLs, entities, and reply/quote/repost relationships where exposed.
- **Change** records in monitoring mode: previous and current metrics for a post or tracked profile.

Missing upstream fields remain `null`; missing counts are not invented as zero. Records include `recordType`, `scrapedAt` and `schemaVersion`. The same post found through multiple supported targets is deduplicated by X ID and retains provenance. Common post fields (`authorUsername`, `likes`, `replies`, `reposts`, `views`, `mediaCount`) are also flattened for CSV/Excel; nested author, engagement and media objects remain in JSON.

For post URL targets, `scrapeReplies: true` adds connected replies present in the public page payload. It is **not a complete conversation**. The tested post showed a public count of 93 replies while only three connected replies were accessible in that first window. `maxReplies`, `includeNestedReplies` and `maxReplyDepth` bound traversal of relationships actually present.

Profile timelines begin with public HTML and continue through guest GraphQL cursors when available. `maxPosts`, `maxPages`, `dateFrom` and `dateTo` bound collection. Cursor loops, duplicate-only pages and continuation failures stop safely; already collected valid records are retained and the target is marked partial in `RUN_SUMMARY`.

### Monitoring

```json
{
  "targets": ["@NASA"],
  "monitor": true,
  "stateKey": "nasa-x",
  "emitMode": "new_and_changed",
  "maxResults": 100
}
```

`emitMode` is `all`, `new`, `changed` or `new_and_changed`. The Actor stores bounded ID and metric snapshots in an Apify Key-Value Store. `maxStateRecords` and `changeRetentionDays` prune history. Records excluded by the delivery cap or charge limit are not marked as delivered. Use the same `stateKey` in scheduled runs to continue monitoring.

Three separate private Apify runs with one state key verified `all` → `new` → `new_and_changed`: the first two capped runs emitted ten distinct records each and stored ten then twenty snapshots. The third emitted real metric/profile changes and one previously unseen post. Another stable-post `new` run emitted zero rows cleanly.

### Filters and limits

Local filters include dates, minimum engagement counts, author inclusion/exclusion, keywords, languages, media type, verified authors, replies/roots, and quote/repost inclusion. Sort by time or engagement metrics. Filters run after normalization and deduplication. `RUN_SUMMARY` reports `filtered` and `droppedByResultCap` separately.

The default is direct HTTP. Guest tokens are acquired automatically, reused and rotated when needed. An optional Apify proxy configuration is available for retries; no X account credentials or cookies are accepted. Residential escalation and live rate-limit recovery are not yet proven.

### Dataset and billing

One mixed dataset contains profile, post and change records. Dataset views named “Post columns”, “Profile columns”, “Reply columns” and “Change columns” are **column projections**, not record-type filters. Filter on `recordType` in an export or integration; replies are posts with `isReply: true`. Raw nested JSON remains in the dataset.

Launch pay-per-event pricing is $0.40 per 1,000 delivered posts/visible replies, $0.20 per 1,000 profiles, and $0.40 per 1,000 change records. The standard Apify `apify-actor-start` event is $0.00005 per run and covers the first five seconds of compute. There is no automatic dataset-item event. The delivery path checks the spending limit before each billable row, pushes it, then charges its post/profile/change event. Failed targets, retries, empty pages, filtered records and duplicates are not delivered as paid results.

A priced private run delivered 25 posts and one profile, with exactly 25 `post-result` and one `profile-result` events. Another run with a $0.001 charge limit discovered 26 unique records, requested ten after the result cap, delivered and charged two posts, then stopped; monitoring state stored only those two IDs. Event accounting and state were checked in `RUN_SUMMARY` and the named state store. Platform usage is included in this pricing configuration.

### Verified cloud evidence and limits

- NASA profile: 100 unique posts plus profile across the public HTML window and six guest GraphQL pages. A separate `maxPages: 2` run stopped after two cursor pages.
- Ten distinct profiles at concurrency six: 300 unique posts and ten profiles discovered across 28 pages. A global cap delivered 300 rows and reported ten excluded by cap. Zero failed or partial targets, one guest-token acquisition, 17.0 seconds wall time, about 133 MB peak RAM.
- A public post URL produced its root and three directly connected replies with stable relationship IDs.
- Staged 10/25/50/100 mixed-target tests reached 58/165/263/378 delivered rows. Those older workloads included unsupported targets and therefore had failures/partial targets; they do not establish 1,000+ unique records.

X can change its public web payloads and guest operations. Deep pagination is verified for profile timelines on several public accounts, not complete history or every account. Search, follower/following lists, X Lists, separate reply/media/repost timelines, quote listings and full conversation traversal are deliberately outside the supported product. Residential proxy cost and live 429 recovery are not proven; optional proxy use should be evaluated separately before Store publication. See [private release audit](docs/RELEASE_AUDIT.md) for run IDs and evidence.

### API example

```bash
curl -X POST "https://api.apify.com/v2/acts/yearly_register~x-twitter-scraper-dev/runs" \
  -H "Authorization: Bearer YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"targets":["@NASA"],"maxPosts":25}'
```

Keep the Apify API token in a secret or environment variable. The Actor itself requires no X login.

# Actor input Schema

## `targets` (type: `array`):

Add public X/Twitter handles, profile URLs, or post URLs. Mix targets in one run. No X login or cookies required.

## `maxResults` (type: `integer`):

Global cap on records delivered to the dataset; records beyond this cap are not marked as seen in monitoring.

## `maxPosts` (type: `integer`):

Maximum posts per profile. Profile timelines continue through public guest cursors when available.

## `maxPages` (type: `integer`):

Maximum guest cursor pages per profile after the public HTML window. Stops earlier at the post cap or date boundary.

## `scrapeReplies` (type: `boolean`):

For post URL targets, include connected replies available on the public page. This is not the full conversation.

## `includeNestedReplies` (type: `boolean`):

Include connected nested replies only when they appear in the public page payload.

## `maxReplies` (type: `integer`):

Cap on replies emitted per post URL target.

## `maxReplyDepth` (type: `integer`):

Bound traversal of reply relationships present in the public page payload.

## `dateFrom` (type: `string`):

Include records on or after this date.

## `dateTo` (type: `string`):

Include records on or before this date.

## `monitor` (type: `boolean`):

Turn this scrape into a persistent monitor. Reuse the same state key on scheduled runs.

## `stateKey` (type: `string`):

Key identifying this monitoring state.

## `emitMode` (type: `string`):

Select all, new, changed, or both new and changed records.

## `proxyConfiguration` (type: `object`):

Optional Apify proxy; direct requests by default.

## `maxConcurrency` (type: `integer`):

Maximum simultaneous target requests.

## `requestTimeoutSecs` (type: `integer`):

Per-request timeout in seconds.

## `maxRequestRetries` (type: `integer`):

Bounded retries per failed request.

## `maxStateRecords` (type: `integer`):

Maximum snapshot entries retained in state.

## `changeRetentionDays` (type: `integer`):

Days to keep previous snapshot entries.

## `minLikes` (type: `integer`):

Filter posts below this like count.

## `minReplies` (type: `integer`):

Filter posts below this reply count.

## `minReposts` (type: `integer`):

Filter posts below this repost count.

## `minViews` (type: `integer`):

Filter posts below this view count.

## `verifiedOnly` (type: `boolean`):

Keep records from verified authors only.

## `includeAuthors` (type: `array`):

Only include posts by these usernames.

## `excludeAuthors` (type: `array`):

Exclude posts by these usernames.

## `includeKeywords` (type: `array`):

Only include posts matching these keywords.

## `excludeKeywords` (type: `array`):

Exclude posts matching these keywords.

## `languages` (type: `array`):

Only include these language codes.

## `mediaTypes` (type: `array`):

Only include posts with these media types.

## `repliesOnly` (type: `boolean`):

Only include reply posts.

## `rootsOnly` (type: `boolean`):

Only include non-reply posts.

## `includeReposts` (type: `boolean`):

Include reposts in results.

## `includeQuotes` (type: `boolean`):

Include quote posts in results.

## `sort` (type: `string`):

Order the delivered results.

## Actor input object example

```json
{
  "targets": [
    "@openai"
  ],
  "maxResults": 100,
  "maxPosts": 25,
  "maxPages": 3,
  "scrapeReplies": false,
  "includeNestedReplies": true,
  "maxReplies": 100,
  "maxReplyDepth": 5,
  "monitor": false,
  "stateKey": "x-monitor",
  "emitMode": "all",
  "proxyConfiguration": {
    "useApifyProxy": false
  },
  "maxConcurrency": 4,
  "requestTimeoutSecs": 30,
  "maxRequestRetries": 4,
  "maxStateRecords": 50000,
  "changeRetentionDays": 30,
  "verifiedOnly": false,
  "repliesOnly": false,
  "rootsOnly": false,
  "includeReposts": true,
  "includeQuotes": true,
  "sort": "default"
}
```

# Actor output Schema

## `dataset` (type: `string`):

Public profiles, posts, visible replies, and monitoring change records delivered by this run.

## `runSummary` (type: `string`):

Target status, pages, deduplication, filters, caps, retries, guest sessions, proxy escalation, monitoring and PPE counters.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "targets": [
        "@openai"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("yearly_register/x-twitter-scraper-dev").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "targets": ["@openai"] }

# Run the Actor and wait for it to finish
run = client.actor("yearly_register/x-twitter-scraper-dev").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "targets": [
    "@openai"
  ]
}' |
apify call yearly_register/x-twitter-scraper-dev --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,yearly_register/x-twitter-scraper-dev"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/OmT3WaVxhUPBSuR3v/builds/FB22voTdd6rQJs24p/openapi.json