# X Scraper (Advance) (`forge-api/x-scraper-advance`) Actor

Scrape X/Twitter data effortlessly. Extract tweets, profiles, users, and lists with advanced filtering. Supports search, media-only mode, engagement filters, date ranges, and location. Auto-detects content type and delivers structured JSON data instantly.

- **URL**: https://apify.com/forge-api/x-scraper-advance.md
- **Developed by:** [Forge Api](https://apify.com/forge-api) (community)
- **Categories:** Automation, Lead generation, Social media
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $10.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## 🛠️ X / Twitter Scraper (Ultimate)

**Professional-grade Twitter/X data extraction tool** that delivers structured JSON data instantly. Perfect for social media monitoring, lead generation, content research, and competitive analysis.

***

### What You Can Extract

#### 📱 Profile Data

- Complete user profiles with bio, location, website, avatar, banner
- Follower/following counts and tweet statistics
- Join dates and verification status
- Profile metadata for lead generation

#### 🐦 Tweet Content

- Full tweet text with hashtags and mentions
- Media URLs (images, videos with HLS preview links)
- Engagement metrics (likes, retweets, comments, quotes)
- Timestamps and tweet URLs
- Author information with verification details

#### 🔍 Search Capabilities

- **Global Tweet Search**: Find tweets across all of Twitter/X
- **User Search**: Discover accounts matching your criteria
- **Profile Search**: Search within specific user profiles

#### 📋 List Scraping

- Extract all tweets from Twitter Lists
- Track curated content collections
- Monitor list updates automatically

***

### Key Features

✨ **Intelligent Auto-Detection** - Automatically determines the best scraping mode based on your inputs\
🎯 **Advanced Filtering** - Filter by date range, location, media type, engagement, language, and more\
🔒 **Built-in Residential Proxy** - Reliable data extraction with automatic proxy management\
📊 **Multiple Data Views** - Pre-configured views for profiles, tweets, media, and engagement analysis\
⚡ **Streaming Results** - Real-time data delivery as scraping progresses\
🔄 **Smart Retry Logic** - Automatic retries with exponential backoff for failed requests\
📈 **Engagement Analytics** - Filter by minimum likes, retweets, or replies

***

### Input Parameters

#### Core Inputs

**Twitter Usernames** (`twitterHandles`)

- Enter one or more usernames (with or without @)
- Example: `elonmusk`, `apify`, `@naval`
- Automatically scrapes profile tweets

**List URLs** (`listUrls`)

- Twitter/X list URLs in format: `https://x.com/i/lists/123456`
- Scrapes all tweets from the specified lists

**Search Terms** (`searchTerms`)

- Search queries to find tweets or users
- Examples: `"web scraping"`, `"#AI"`, `"from:apify"`
- Works with global search or profile-specific search

#### Search Configuration

**Search Type** (`searchType`)

- `tweets` - Search tweets globally across Twitter/X
- `users` - Find Twitter accounts
- `profile_tweets` - Search within specific profiles (requires usernames)

#### Profile Options

- **Include Replies** - Get reply tweets along with regular tweets
- **Include Retweets** - Include retweeted content in results
- **Media Only** - Extract only tweets containing images or videos
- **Scrape Profile Info** - Extract complete profile details (default: enabled)

#### Content Filters

**Media Filters:**

- Only Images - Tweets with images only
- Only Videos - Tweets with videos only
- Only Tweets with Links - Filter tweets containing external links

**Account Filters:**

- Only Verified Users - Limit to verified accounts
- Only Quote Tweets - Extract quote tweets exclusively
- Include Native Retweets - Include native retweet format (default: enabled)
- Safe Search - Filter sensitive content

#### Time & Location Filters

- **Start Date** - Filter from date (YYYY-MM-DD format)
- **End Date** - Filter until date (YYYY-MM-DD format)
- **Location** - Filter by geographic location (e.g., "Los Angeles", "New York")

#### Engagement Filters

Set minimum thresholds:

- **Minimum Likes** - Filter tweets by minimum like count
- **Minimum Replies** - Filter tweets by minimum reply count
- **Minimum Retweets** - Filter tweets by minimum retweet count

#### Language & Limits

- **Tweet Language** - Filter by language code (e.g., `en`, `es`, `fr`, `de`)
- **Max Items** - Maximum items to scrape (0 = unlimited, default: 100)
- **Max Pages** - Maximum pages per handle/query (default: 100, max: 100)
- **Max Retries** - Retry attempts for failed requests (1-10, default: 5)

***

### Output Data Structure

All scraped data is saved to the Apify dataset in structured JSON format with multiple pre-configured views:

#### Available Views

1. **All Results** - Complete dataset with all fields
2. **Profiles Only** - Profile information view
3. **Tweets Only** - Tweet/post data view
4. **Tweets with Media** - Media-rich tweets
5. **Top Engagement** - Tweets sorted by engagement metrics

#### Data Fields

**Profile Objects Include:**

- `username`, `fullname`, `bio`, `location`, `website`
- `joined` date, `avatar`, `banner` URLs
- `stats` object with tweets, followers, following counts
- `scrape_date` timestamp

**Tweet Objects Include:**

- `tweet_url`, `tweet_id`, `text`, `date`
- `author` object with profile details
- `hashtags`, `mentions`, `urls` arrays
- `stats` object with engagement metrics
- `media` array with image/video URLs and metadata
- `is_retweet` boolean flag
- `scrape_date` timestamp

**Search Results Include:**

- `search_query` field for search context
- `type` field indicating data type (profile, tweet, user, etc.)

***

### Usage Examples

#### Example 1: Extract Profile Tweets

```json
{
  "twitterHandles": ["elonmusk", "apify"],
  "maxItems": 100
}
```

Scrapes the latest 100 tweets from each profile.

#### Example 2: Media-Only Scraping

```json
{
  "twitterHandles": ["apify"],
  "mediaOnly": true,
  "maxItems": 50
}
```

Extracts only tweets containing images or videos.

#### Example 3: Search with Filters

```json
{
  "searchTerms": ["LLM", "AI"],
  "searchType": "tweets",
  "startDate": "2024-01-01",
  "endDate": "2024-12-31",
  "onlyVerified": true,
  "minLikes": 10,
  "tweetLanguage": "en",
  "maxItems": 200
}
```

Searches for verified English tweets about LLM/AI from 2024 with at least 10 likes.

#### Example 4: Profile Search

```json
{
  "twitterHandles": ["apify"],
  "searchTerms": ["scraping", "automation"],
  "searchType": "profile_tweets",
  "maxItems": 100
}
```

Searches for specific keywords within a profile's tweets.

#### Example 5: User Discovery

```json
{
  "searchTerms": ["data scientist", "ML engineer"],
  "searchType": "users",
  "maxItems": 50
}
```

Finds Twitter accounts matching the search terms.

#### Example 6: List Scraping

```json
{
  "listUrls": ["https://x.com/i/lists/78783491"],
  "maxItems": 200
}
```

Scrapes tweets from a Twitter List.

#### Example 7: Engagement-Based Filtering

```json
{
  "searchTerms": ["web scraping"],
  "searchType": "tweets",
  "minLikes": 50,
  "minRetweets": 10,
  "onlyImages": true,
  "maxItems": 100
}
```

Finds high-engagement image tweets about web scraping.

#### Example 8: Complete Profile with Replies

```json
{
  "twitterHandles": ["elonmusk"],
  "includeReplies": true,
  "includeRetweets": true,
  "scrapeProfileInfo": true,
  "maxItems": 500
}
```

Comprehensive profile extraction including replies and retweets.

***

### Output Views Explained

The dataset includes 5 pre-configured views for easy data analysis:

#### 📊 All Results View

Displays complete dataset with all available fields. Perfect for comprehensive data export.

#### 👤 Profiles Only View

Filtered view showing only profile information. Ideal for lead generation and user research.

#### 🐦 Tweets Only View

Tweet-focused view sorted by date (newest first). Best for content analysis and trend monitoring.

#### 🎬 Tweets with Media View

Media-rich tweets only. Perfect for visual content analysis and media asset collection.

#### 📈 Top Engagement View

Tweets sorted by engagement metrics (likes + retweets). Great for identifying viral content.

***

### Technical Specifications

#### Supported Operations

- Profile timeline scraping (normal, with replies, media-only modes)
- Twitter List extraction
- Global tweet search
- User account search
- Profile-specific search
- Advanced filtering and pagination

#### Data Extraction

- Automatic pagination handling
- Real-time streaming results
- Structured JSON output
- Multiple data views for different use cases

#### Reliability Features

- Built-in residential proxy support (automatic)
- Configurable retry logic (1-10 attempts)
- Error handling and graceful degradation
- Rate limiting protection

***

### Use Cases

#### 🎯 Lead Generation

Extract verified users, filter by engagement, and collect profile data for outreach campaigns.

#### 📊 Social Media Monitoring

Track mentions, hashtags, and trending topics with date and location filters.

#### 🔍 Competitive Analysis

Monitor competitor profiles, analyze engagement patterns, and track content strategies.

#### 📈 Content Research

Find high-performing tweets, analyze media content, and identify trending topics.

#### 🎬 Media Collection

Extract images and videos from profiles or search results with direct URLs.

#### 👥 User Discovery

Search for users by keywords, industry, or interests for targeted outreach.

***

### Best Practices

1. **Start Small** - Test with `maxItems: 50` before scaling up
2. **Use Filters** - Apply engagement filters to get high-quality results
3. **Date Ranges** - Use date filters to focus on specific time periods
4. **Residential Proxy** - Enabled by default for reliable extraction
5. **Monitor Limits** - Set reasonable `maxPages` and `maxItems` to control costs
6. **Language Filtering** - Use `tweetLanguage` to target specific regions
7. **Profile Info** - Enable `scrapeProfileInfo` for complete user data

***

### Data Privacy & Compliance

This tool is designed for public data extraction. Users are responsible for:

- Compliance with Twitter/X Terms of Service
- Adherence to data protection regulations (GDPR, CCPA, etc.)
- Ethical use of scraped data
- Respect for user privacy and consent requirements

***

### Support & Resources

For technical support, feature requests, or questions:

- Check the Actor logs for detailed error messages
- Review input parameters in the Actor interface
- Consult Apify documentation for platform-specific questions

***

### Version

**Current Version:** 1.0\
**Categories:** Social Media, Lead Generation, Automation\
**Platform:** Apify

***

**Ready to extract Twitter/X data?** Configure your inputs, set your filters, and start scraping. The Actor handles everything automatically! 🚀

# Actor input Schema

## `twitterHandles` (type: `array`):

Enter Twitter usernames to scrape (with or without @). Example: elonmusk, apify

## `listUrls` (type: `array`):

Enter Twitter/X list URLs to scrape. Format: https://x.com/i/lists/123456 or https://twitter.com/i/lists/123456

## `searchTerms` (type: `array`):

Search queries to find tweets or users. For profile search, provide both username and search term. Examples: 'web scraping', 'AI', '#python'

## `searchType` (type: `string`):

When using Search Terms, what to search for

## `includeReplies` (type: `boolean`):

Include reply tweets from profiles (automatically switches to 'with replies' mode)

## `includeRetweets` (type: `boolean`):

Include retweets in profile results (automatically switches to 'with replies' mode)

## `mediaOnly` (type: `boolean`):

Only scrape tweets that contain images or videos

## `maxItems` (type: `integer`):

Maximum number of items (tweets/users) to scrape. 0 = unlimited

## `maxPages` (type: `integer`):

Maximum number of pages to scrape per handle/query

## `startDate` (type: `string`):

Filter tweets from this date onwards (YYYY-MM-DD format)

## `endDate` (type: `string`):

Filter tweets up to this date (YYYY-MM-DD format)

## `location` (type: `string`):

Filter tweets by location (e.g., 'Los Angeles', 'New York', 'London')

## `onlyImages` (type: `boolean`):

Filter tweets that contain images

## `onlyVideos` (type: `boolean`):

Filter tweets that contain videos

## `onlyVerified` (type: `boolean`):

Only include tweets from verified accounts

## `onlyQuotes` (type: `boolean`):

Only include quote tweets

## `includeNativeRetweets` (type: `boolean`):

Include native retweets in search results

## `includeLinks` (type: `boolean`):

Filter tweets that contain external links

## `safeSearch` (type: `boolean`):

Enable safe search to filter sensitive content

## `minLikes` (type: `integer`):

Minimum number of likes a tweet must have

## `minReplies` (type: `integer`):

Minimum number of replies a tweet must have

## `minRetweets` (type: `integer`):

Minimum number of retweets a tweet must have

## `tweetLanguage` (type: `string`):

Filter by language code (e.g., 'en' for English, 'es' for Spanish, 'fr' for French)

## `scrapeProfileInfo` (type: `boolean`):

Extract detailed profile information (bio, stats, avatar, etc.)

## `useResidentialProxy` (type: `boolean`):

Enable Apify residential proxy for better reliability (requires Apify paid plan)

## `proxyConfiguration` (type: `object`):

Apify proxy configuration (requires paid plan)

## `maxRetries` (type: `integer`):

Number of retry attempts for failed requests

## Actor input object example

```json
{
  "twitterHandles": [
    "elonmusk"
  ],
  "listUrls": [],
  "searchTerms": [],
  "searchType": "tweets",
  "includeReplies": false,
  "includeRetweets": true,
  "mediaOnly": false,
  "maxItems": 100,
  "maxPages": 100,
  "startDate": "2024-01-01",
  "endDate": "2024-12-31",
  "location": "",
  "onlyImages": false,
  "onlyVideos": false,
  "onlyVerified": false,
  "onlyQuotes": false,
  "includeNativeRetweets": true,
  "includeLinks": false,
  "safeSearch": false,
  "minLikes": 0,
  "minReplies": 0,
  "minRetweets": 0,
  "tweetLanguage": "en",
  "scrapeProfileInfo": true,
  "useResidentialProxy": true,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  },
  "maxRetries": 5
}
```

# Actor output Schema

## `allResults` (type: `string`):

Complete dataset with all scraped data

## `profilesOnly` (type: `string`):

Filtered view showing only profile information

## `tweetsOnly` (type: `string`):

Tweet-focused view sorted by date (newest first)

## `tweetsWithMedia` (type: `string`):

Media-rich tweets only (images and videos)

## `topEngagement` (type: `string`):

Tweets sorted by engagement metrics (likes + retweets)

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "twitterHandles": [
        "elonmusk"
    ],
    "listUrls": [],
    "searchTerms": []
};

// Run the Actor and wait for it to finish
const run = await client.actor("forge-api/x-scraper-advance").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "twitterHandles": ["elonmusk"],
    "listUrls": [],
    "searchTerms": [],
}

# Run the Actor and wait for it to finish
run = client.actor("forge-api/x-scraper-advance").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "twitterHandles": [
    "elonmusk"
  ],
  "listUrls": [],
  "searchTerms": []
}' |
apify call forge-api/x-scraper-advance --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,forge-api/x-scraper-advance"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/4Yw1UWCgMNRGuWbFN/builds/jFBHfB2iMTPdA5QA1/openapi.json