# Ultimate X Scraper (`apigeek/ultimate-x-scraper`) Actor

Production-grade X/Twitter data acquisition engine. Extract posts, user timelines, and search results with deterministic JSON schema and optional AI content intelligence.

- **URL**: https://apify.com/apigeek/ultimate-x-scraper.md
- **Developed by:** [API Geek](https://apify.com/apigeek) (community)
- **Categories:** Social media, AI
- **Stats:** 7 total users, 3 monthly users, 98.3% runs succeeded, 3 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

from $0.35 / 1,000 scraped results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Ultimate X Scraper

> Production-grade X (formerly Twitter) data extraction and optional AI content intelligence engine for Apify.

**Ultimate X Scraper** is a high-performance, production-ready Actor built for reliable, structured data acquisition from X. Extract post content, author metadata, public profile timelines, and X search results with deterministic schema, automatic target resolution, and optional structured AI intelligence.

***

### Capabilities & Key Features

- **Flexible Target Resolution**: Provide profile handles (`@handle`), direct post URLs (`https://x.com/user/status/123`), or search queries in a single unified input list.
- **Profile Timelines**: Extract recent posts, reposts, replies, and author engagement metrics for public X profiles.
- **Direct Post Extraction**: Extract metadata for individual post links with author profile data and engagement counts when available.
- **X Search Results**: Search public X posts for keywords, hashtags, cashtags, or search queries with chronological (`Latest`) or relevance (`Top`) sorting.
- **Optional AI Intelligence**: Automated post classification, sentiment analysis, promotion detection, entity extraction, and structured summarization.
- **Resilience & Non-Blocking AI**: AI analysis runs post-collection. If AI service is unconfigured or experiences temporary outages, post extraction completes cleanly with `ai.analyzed = false`.
- **Deduplication & Target Normalization**: Automatic deduplication across target inputs prevents redundant data collection.
- **Clean Normalized Output**: Consistent JSON dataset schema ready for export to JSON/CSV or downstream integration workflows.

***

### Supported Target Inputs

You can supply targets via the unified `targets` list or using specialized input arrays:

| Input Field | Type | Description |
|---|---|---|
| `targets` | `Array<string>` | Unified list of mixed targets: usernames (`@jack`), profile links, post URLs, or search queries (`"bitcoin news"`). |
| `usernames` | `Array<string>` | Explicit list of X profile usernames/handles to collect timelines for. |
| `tweetUrls` | `Array<string>` | Explicit list of direct X post URLs to fetch. |
| `searchQueries` | `Array<string>` | Explicit list of search terms or search queries. |

***

### Quick Start Examples

#### Example 1: Single Direct Post

```json
{
  "targets": [
    "https://x.com/jack/status/20"
  ]
}
```

#### Example 2: Public User Profile Timeline

```json
{
  "usernames": [
    "jack"
  ],
  "maxTweetsPerUser": 20,
  "includeReplies": true,
  "includeRetweets": false
}
```

#### Example 3: Public X Search

```json
{
  "searchQueries": [
    "artificial intelligence"
  ],
  "maxSearchResults": 50,
  "searchProduct": "Latest"
}
```

#### Example 4: Mixed Multi-Target Collection

```json
{
  "targets": [
    "@jack",
    "https://x.com/jack/status/20",
    "crypto market"
  ],
  "maxTweetsPerUser": 20,
  "maxSearchResults": 20,
  "concurrency": 4
}
```

#### Example 5: Generic AI Intelligence Analysis

```json
{
  "targets": [
    "https://x.com/jack/status/20"
  ],
  "aiAnalysis": true
}
```

#### Example 6: Targeted Brand AI Analysis (Tesla)

```json
{
  "targets": [
    "elonmusk"
  ],
  "aiAnalysis": true,
  "targetBrand": "Tesla"
}
```

#### Example 7: Targeted Campaign AI Analysis (Tesla Model Y & Robotaxi)

```json
{
  "targets": [
    "elonmusk"
  ],
  "aiAnalysis": true,
  "targetBrand": "Tesla",
  "campaignKeywords": [
    "Model Y",
    "Robotaxi"
  ]
}
```

***

### Dataset Output Format

*ACTUAL CAPTURED OUTPUT RECORD (Verified Run on Post `20`)*

Each item pushed to the default dataset adheres to a deterministic, production-ready schema:

```json
{
  "schemaVersion": "1",
  "sourceType": "tweet",
  "query": null,
  "requestedUsername": null,
  "id": "20",
  "url": "https://x.com/jack/status/20",
  "text": "just setting up my twttr",
  "createdAt": "2006-03-21T20:50:14.000Z",
  "language": "en",
  "isReply": false,
  "isRetweet": false,
  "author": {
    "username": "jack",
    "name": "jack"
  },
  "metrics": {
    "likes": 308542,
    "replies": 17980
  },
  "media": [],
  "scrapedAt": "2026-08-09T18:50:00.000Z",
  "ai": {
    "analyzed": true,
    "classification": "other",
    "confidence": 0.9,
    "promotionDetected": false,
    "brandMentioned": false,
    "entities": [],
    "sentiment": "unknown",
    "relevance": "unknown",
    "evidence": [],
    "summary": "The post is a simple statement about setting up Twitter."
  }
}
```

***

### Metric Availability & Disclosure

- **Likes and Replies**: Likes and replies are included when available.
- **Additional Engagement Metrics**: Additional engagement metrics (such as retweets, quotes, bookmarks, or view counts) may be available depending on the result context and data exposed by X.
- **Transparency**: Metrics unavailable for a specific item are omitted from the output object rather than fabricated as zero.

***

### AI Content Intelligence

When `aiAnalysis` is set to `true`, collected records are enriched with structured AI intelligence:

- **Classification**: Categorizes post content (e.g. `promotional`, `informational`, `announcement`, `opinion`, `news`, `engagement`, `other`).
- **Sentiment**: Analyzes content tone (`positive`, `neutral`, `negative`, `mixed`, `unknown`).
- **Promotion Detection**: Detects commercial calls-to-action or promotional content (`promotionDetected: true/false`).
- **Brand & Entity Extraction**: Identifies mentioned entities and flags brand mentions (`brandMentioned: true/false`).
- **Structured Summarization**: Generates a concise content summary (`summary`).
- **Analytical Metrics**: Returns confidence level (`confidence`, 0.0 to 1.0) and relevance assessment (`relevance`).

***

### Pricing & Metering

**Ultimate X Scraper** uses Pay-Per-Event (PPE) usage pricing:

- **Scraped Result Event**: `$0.35` per 1,000 successfully scraped records (`$0.00035` / record).
- **AI Analyzed Result Event**: `+$3.00` per 1,000 successfully AI-analyzed records (`$0.00300` / analyzed record).

#### Additive Billing Structure

AI intelligence is an optional enrichment. Event charges apply to requested and successfully produced results:

- **1,000 Scraped Records (AI OFF)**: `1,000 × $0.00035` = **$0.35**
- **1,000 Scraped Records (1,000 AI Analyzed)**: `1,000 × $0.00035 + 1,000 × $0.00300` = **$3.35**
- **1,000 Scraped Records (200 AI Analyzed)**: `1,000 × $0.00035 + 200 × $0.00300` = **$0.95**

#### Usage-Based Event Metering

- **Scraping Events**: Scraping events are metered for successfully delivered results. Invalid or rejected inputs that produce no output generate zero scraped-result events.
- **AI Analysis Events**: AI analysis events are metered only for records successfully enriched by AI (`ai.analyzed = true`). If AI analysis is degraded or unconfigured (`ai.analyzed = false`), zero AI-analyzed events are charged.

***

### Common Use Cases

1. **Brand & Media Monitoring**: Track brand mentions, public feedback, and content engagement across X.
2. **Competitive Intelligence**: Analyze public posting cadence, audience response, and content strategies of market competitors.
3. **Market & Trend Research**: Study conversation sentiment and topic volume surrounding emerging industry trends or public news.
4. **Creator & Public Profile Analysis**: Review public post metrics and timeline activity for featured accounts.
5. **AI / ML Pipeline Integration**: Extract structured social datasets formatted for language model ingestion, sentiment analytics, or automated reporting.

***

### Error Handling & Run Statuses

At run completion, an executive run summary is written to Key-Value Store key `OUTPUT`:

```json
{
  "schemaVersion": "1",
  "status": "SUCCEEDED",
  "targetsReceived": 3,
  "targetsSucceeded": 3,
  "targetsFailed": 0,
  "tweetsScraped": 42,
  "scrapedResultCount": 42,
  "aiAnalyzedResultCount": 42,
  "durationMs": 3450,
  "timestamp": "2026-08-09T18:50:00.000Z"
}
```

#### Standard Error Codes

If a target fails, it is recorded in `summary.errors` using standardized error codes:

- `ERR_X101`: Invalid input specification format.
- `ERR_X201`: Profile handle or account not found.
- `ERR_X202`: Post or Tweet URL not found.
- `ERR_X301`: Temporary rate limit encountered.
- `ERR_X302`: Target content restricted or protected.
- `ERR_X305`: Temporary collection runtime error.
- `ERR_X501`: Network connection timeout.

***

### Limitations

- **Public Data Only**: Extracts publicly available posts, timelines, and search results. Protected accounts or private posts cannot be retrieved.
- **Platform Variability**: Data fields and engagement metric availability depend on public information provided by X for a given post or profile.
- **Content Availability**: Posts deleted or restricted by X will not be returned in datasets.

***

### Downstream Integration & Export

Dataset results can be exported to JSON or CSV format, or integrated with downstream analytics pipelines and automation workflows via the Apify API, Python SDK, or Node.js client.

***

### Responsible Use & Compliance

Users are responsible for ensuring that their data extraction activities comply with applicable privacy laws, data protection regulations, platform terms of service, and ethical scraping practices.

***

### License

MIT License

# Actor input Schema

## `targets` (type: `array`):

List of mixed targets to scrape. Accepts usernames (@username or handle), profile URLs, individual Tweet URLs, or search queries.

## `usernames` (type: `array`):

Explicit list of Twitter/X usernames to scrape timelines for (e.g. \['jack']).

## `tweetUrls` (type: `array`):

Explicit list of direct Tweet URLs to fetch (e.g. \['https://x.com/jack/status/20']).

## `searchQueries` (type: `array`):

Explicit list of search terms or advanced search queries (e.g. \['crypto', '"Web3"']).

## `maxTweetsPerUser` (type: `integer`):

Maximum number of tweets to collect per user timeline (1 to 500).

## `maxSearchResults` (type: `integer`):

Maximum number of search results to collect per query (1 to 500).

## `searchProduct` (type: `string`):

Search sorting product: 'Latest' (chronological) or 'Top' (relevance/engagement).

## `includeReplies` (type: `boolean`):

Whether to include reply tweets in the output.

## `includeRetweets` (type: `boolean`):

Whether to include retweets/reposts in the output.

## `concurrency` (type: `integer`):

Number of target tasks to process concurrently (1 to 10).

## `aiAnalysis` (type: `boolean`):

Enable optional AI content intelligence analysis for collected posts.

## `aiFeatures` (type: `array`):

Selected AI intelligence analysis features to execute.

## `targetBrand` (type: `string`):

Optional target brand or organization name to evaluate in collected content (e.g. 'Tesla'). Leave empty for general AI analysis.

## `campaignKeywords` (type: `array`):

Optional active campaign keywords or hashtags to evaluate content relevance against.

## `promoUrlPatterns` (type: `array`):

Optional accepted campaign promo or referral URL patterns/domains.

## Actor input object example

```json
{
  "targets": [],
  "usernames": [],
  "tweetUrls": [
    "https://x.com/jack/status/20"
  ],
  "searchQueries": [],
  "maxTweetsPerUser": 40,
  "maxSearchResults": 40,
  "searchProduct": "Latest",
  "includeReplies": true,
  "includeRetweets": true,
  "concurrency": 4,
  "aiAnalysis": false,
  "aiFeatures": [
    "classification",
    "promotion",
    "brandDetection",
    "sentiment",
    "summary"
  ],
  "campaignKeywords": [],
  "promoUrlPatterns": []
}
```

# Actor output Schema

## `dataset` (type: `string`):

Normalized dataset items written to default dataset.

## `summary` (type: `string`):

Run summary written to Key-Value store OUTPUT.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "tweetUrls": [
        "https://x.com/jack/status/20"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("apigeek/ultimate-x-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "tweetUrls": ["https://x.com/jack/status/20"] }

# Run the Actor and wait for it to finish
run = client.actor("apigeek/ultimate-x-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "tweetUrls": [
    "https://x.com/jack/status/20"
  ]
}' |
apify call apigeek/ultimate-x-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,apigeek/ultimate-x-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/7Ld215JEgruSbhMyF/builds/oQPPVFDKGed4Xw5RX/openapi.json