# Twitter / X Profile Scraper (`datapilot/twitter-x-profile-scraper`) Actor

usernames, display names, bios, follower and following counts, profile images, and profile URLs from usernames or links. Supports , Apify Proxy, batch processing, and structured dataset exports.

- **URL**: https://apify.com/datapilot/twitter-x-profile-scraper.md
- **Developed by:** [Data Pilot](https://apify.com/datapilot) (community)
- **Categories:** Other
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 scraped results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

🐦 **Twitter / X Profile** Scraper is a powerful Apify Actor designed to discover, track, and collect public **Twitter / X Profile** data directly from X's own profile pages using a real headless browser. This tool provides comprehensive **Twitter / X Profile** intelligence including full name, bio/description, follower count, following count, and profile picture for any accessible **Twitter / X Profile**. Whether you're building a social media dashboard, tracking competitor accounts, or conducting audience research, the Twitter / X Profile Scraper delivers actionable **Twitter / X Profile** insights efficiently.

With -powered rendering, OpenGraph/meta-tag extraction, in-page DOM fallback search, and reliable Apify Dataset delivery, the Twitter / X Profile Scraper ensures comprehensive **Twitter / X Profile** coverage across any list of usernames or profile links. It focuses on key **Twitter / X Profile** signals including followers, following, and bio, making it an essential tool for social research and account intelligence.

***

### 📋 Table of Contents

- [Features](#-features)
- [Data Source](#-data-source)
- [How It Works](#-how-it-works)
- [Input](#-input)
- [Output](#-output)
- [Technical Stack](#-technical-stack)
- [Data Fields](#-data-fields-explained)
- [Use Cases](#-use-cases)
- [Quick Start](#-quick-start)
- [Configuration](#-configuration)
- [Performance](#-performance)
- [Important Notes](#-important-notes)
- [License & Legal](#-license--legal)

***

### 🔥 Features

- **Real Browser Rendering** – Uses  (Chromium) to load each **Twitter / X Profile** page exactly as a real visitor would see it.
- **Flexible Input Handling** – Accepts plain usernames, `@handles`, or full **Twitter / X Profile** URLs and normalizes them automatically.
- **OpenGraph Meta Extraction** – Pulls full name, description, and profile picture directly from X's `og:title`, `og:description`, and `og:image` meta tags.
- **Regex Follower Parsing** – Extracts follower and following counts from the meta description using targeted pattern matching.
- **In-Page DOM Fallback** – Automatically searches rendered follower/following links on the page when meta data is incomplete.
- **404 Detection** – Automatically detects and skips **Twitter / X Profile** pages that no longer exist.
- **Residential Proxy Support** – Apify residential proxy wired directly into  for reliable **Twitter / X Profile** access.
- **Batch Processing** – Scrape any number of **Twitter / X Profile** entries in a single run.
- **Human-Like Pacing** – Built-in delay between **Twitter / X Profile** requests to reduce blocking.
- **Real-Time Dataset Push** – Pushes all collected **Twitter / X Profile** records to Apify Dataset in a single batch.
- **Graceful Error Handling** – Skips a failing **Twitter / X Profile** without stopping the whole run.

***

### 📊 Data Source

#### **X (Twitter) Public Profile Pages**

- **Authority**: X's own public-facing profile pages
- **Access Method**: Headless Chromium via , rendering `x.com/{username}`
- **Coverage**: Any public **Twitter / X Profile** reachable without login
- **Data**: Rendered OpenGraph meta tags and in-page follower/following DOM elements
- **Access**: Public pages, no API key required
- **Update Frequency**: Reflects each **Twitter / X Profile**'s current live state at time of run

***

### ⚙️ How It Works

The Twitter / X Profile Scraper accepts a list of usernames or profile URLs, normalizes each into a clean username, and loads the profile page in a real headless browser session. It parses the rendered page's OpenGraph meta tags to extract full name, description, and profile picture. Follower and following counts are first parsed from the meta description using regex; if either is still missing, the Actor evaluates the live DOM to read follower/following links directly off the page. Every successfully scraped **Twitter / X Profile** record is collected and pushed to the Apify Dataset in a single batch at the end of the run.

**Key Processing Steps:**

1. **Input Parsing** – Accept a list of usernames, `@handles`, or full **Twitter / X Profile** URLs
2. **Username Normalization** – Strip `@` symbols and extract the username from full URLs
3. **Proxy Setup** – Configure Apify residential proxy credentials for
4. **Browser Launch** – Start a headless Chromium instance with a realistic user agent
5. **Profile Loop** – Iterate through each normalized **Twitter / X Profile** username
6. **Page Navigation** – Load `x.com/{username}` and check for 404s
7. **Meta Tag Parsing** – Extract full name, description, and profile picture from OpenGraph tags
8. **Regex Extraction** – Parse follower/following counts from the meta description
9. **DOM Fallback** – Read follower/following counts directly from rendered page links if still missing
10. **Timestamp Tagging** – Record scrape time per entry
11. **Rate Limiting** – Apply a short delay between **Twitter / X Profile** requests
12. **Dataset Push** – Push all collected records from the run in a single batch

**Key Benefits:**

- Pull **Twitter / X Profile** data for any public account without manual visits
- Detect missing or suspended **Twitter / X Profile** pages automatically
- Monitor creator and brand accounts' **Twitter / X Profile** metrics over time
- Feed social dashboards, research pipelines, or influencer-tracking tools
- Automate recurring **Twitter / X Profile** checks as follower counts change

***

### 📥 Input

The Actor accepts the following input parameters:

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `profiles` | array | *(required)* | List of **Twitter / X Profile** usernames, `@handles`, or full profile URLs. |
| `useApifyProxy` | boolean | `true` | Enable Apify residential proxies. |
| `apifyProxyGroups` | array | `["RESIDENTIAL"]` | Proxy group configuration. |

**Example Input:**

```json
{
  "profiles": [
    "elonmusk",
    "@nasa",
    "https://x.com/natgeo"
  ],
  "useApifyProxy": true,
  "apifyProxyGroups": ["RESIDENTIAL"]
}
```

**Scrape a Single Twitter / X Profile:**

```json
{
  "profiles": ["natgeo"]
}
```

**Scrape from Full Profile URLs:**

```json
{
  "profiles": [
    "https://x.com/nasa",
    "https://x.com/natgeo"
  ]
}
```

***

### 📤 Output

The Actor pushes **Twitter / X Profile** records with the following structure:

| Field | Type | Description |
|-------|------|-------------|
| `username` | string | The normalized **Twitter / X Profile** username used for the request |
| `full_name` | string | Display name extracted from the `og:title` meta tag |
| `profile_url` | string | The full **Twitter / X Profile** URL |
| `description` | string | Profile bio/description text |
| `followers` | string/null | Follower count as displayed on the **Twitter / X Profile** (e.g., "1.2M Followers") |
| `following` | string/null | Following count as displayed on the **Twitter / X Profile** (e.g., "342 Following") |
| `profile_pic` | string | URL of the **Twitter / X Profile** picture |
| `scraped_at` | string | ISO 8601 timestamp of when the record was scraped |

**Example Twitter / X Profile Record:**

```json
{
  "username": "natgeo",
  "full_name": "National Geographic",
  "profile_url": "https://x.com/natgeo",
  "description": "Taking the world's best photos daily since 1888.",
  "followers": "23.1M Followers",
  "following": "312 Following",
  "profile_pic": "https://pbs.twimg.com/profile_images/...",
  "scraped_at": "2026-08-06T09:00:00+00:00"
}
```

***

### 🧰 Technical Stack

- **Browser Automation:** \`\` (Chromium) for real, rendered access to **Twitter / X Profile** pages
- **HTML Parsing:** `BeautifulSoup` for OpenGraph and meta tag extraction
- **Pattern Matching:** `re` for follower/following count extraction from meta descriptions
- **Date Handling:** `datetime` with timezone-aware UTC timestamps
- **Async:** `asyncio` for non-blocking, multi-profile processing
- **Proxy:** Apify Proxy with `RESIDENTIAL` configuration, wired into 's launch options
- **Logging:** Apify Actor logging system
- **Platform:** Apify Actor serverless environment

***

### 📊 Data Fields Explained

#### **Identity**

- **username**: The normalized username used to build the **Twitter / X Profile** URL
- **full\_name**: The public display name shown on the **Twitter / X Profile**

#### **Profile Details**

- **description**: The account's bio/description text
- **profile\_url**: The canonical link to the **Twitter / X Profile**
- **profile\_pic**: Direct link to the profile's avatar image

#### **Engagement Metrics**

- **followers**: Follower count as publicly displayed on the **Twitter / X Profile**
- **following**: Following count as publicly displayed on the **Twitter / X Profile**

***

### 🎯 Use Cases

- **Creator Research** – Evaluate follower and engagement metrics before an influencer partnership
- **Competitor Tracking** – Monitor a competitor's **Twitter / X Profile** growth over time
- **Content Auditing** – Verify profile bios and branding are up to date
- **Marketing Analytics** – Feed **Twitter / X Profile** metrics into campaign dashboards
- **Media Monitoring** – Track how partner or affiliate accounts present themselves
- **Academic Research** – Study public **Twitter / X Profile** trends at scale
- **OSINT Research** – Gather publicly available **Twitter / X Profile** metadata for investigations

***

### 🚀 Quick Start

#### **1. Prepare Input**

Go to Apify Console and enter:

```json
{
  "profiles": ["nasa", "natgeo"]
}
```

#### **2. Run the Actor**

Click **Start**. The Actor will:

- Normalize each username or URL into a clean **Twitter / X Profile** username
- Launch a headless browser and load each profile page
- Extract meta data and fall back to DOM parsing as needed
- Push all results to the Dataset

#### **3. Monitor Progress**

Console shows:

```
Initializing Twitter Scraper for 2 account(s)...
[Twitter Scraper] Navigating to profile: '@nasa'
[Twitter Scraper] Navigating to profile: '@natgeo'
Done! Successfully saved 2 Twitter profile(s) to Dataset.
```

#### **4. View & Download Results**

- **Results Tab**: All **Twitter / X Profile** records
- **Export**: JSON, CSV, Excel, or HTML
- **Filter**: By follower count or username
- **API Access**: Available via the Apify API

***

### ⚙️ Configuration

Single Twitter / X Profile:

```json
{
  "profiles": ["natgeo"]
}
```

Mixed usernames and URLs:

```json
{
  "profiles": [
    "@nasa",
    "https://x.com/natgeo"
  ]
}
```

Run without proxy:

```json
{
  "profiles": ["natgeo"],
  "useApifyProxy": false
}
```

***

### 📈 Performance

#### **Processing Speed**

- One page load per **Twitter / X Profile**, plus in-page DOM evaluation when needed
- 2 second delay between profiles to reduce blocking
- Slower than raw HTTP scraping since a full browser is launched, but far more resilient to X's client-rendered pages

#### **Resource Usage**

- Memory: Moderate, since a Chromium browser instance is launched
- Network: One page navigation per **Twitter / X Profile**
- Proxy: One residential proxy tunnel shared across the browser context for the run

***

### ⚠️ Important Notes

#### **Legal & Compliance**

- **Fair Use**: Respects X's public pages and applies pacing delays between requests
- **Public Data Only**: Retrieves only publicly visible **Twitter / X Profile** data exposed on the rendered page
- **Privacy**: Collecting profile data may be subject to privacy regulations and X's Terms of Service — use only for legitimate purposes
- **Legal**: Not legal advice — consult qualified professionals before using this data for compliance-sensitive decisions

#### **Data Quality**

- **Freshness**: Reflects each **Twitter / X Profile**'s live state at time of run
- **Completeness**: Follower/following counts depend on which extraction layer succeeds; some profiles may return partial data
- **Accuracy**: Sourced directly from X's own rendered pages
- **Verification**: Cross-check high-stakes figures against the live X app or website

#### **Best Practices**

- Always configure a residential proxy for the most reliable **Twitter / X Profile** access
- Use full profile URLs when a username alone is ambiguous
- Re-run periodically to track changes in follower or following counts
- Expect `followers` or `following` to occasionally be `null` if all extraction layers fail
- Batch multiple usernames together in one run to reduce overhead

***

### 📄 License & Legal

**Terms of Use:**

- Use for legitimate research, marketing, and analytics purposes
- Respect X's Terms of Service
- Do not use for unsolicited contact, harassment, or unlawful surveillance
- Use **Twitter / X Profile** data responsibly and in compliance with applicable privacy laws

**Disclaimer:**
Twitter / X Profile Scraper is provided as-is for research and analytics purposes. Users are responsible for compliance with X's ToS and all applicable laws. This is not legal advice.

***

### ⚖️ Twitter / X Profile Excellence

This Actor is optimized for **Twitter / X Profile** research with:

- ✅ Real, browser-rendered data accuracy
- ✅ Multi-layer fallback extraction (meta tags → regex → DOM search)
- ✅ Flexible username/URL input handling
- ✅ 404 and missing-profile detection
- ✅ Real-time Dataset push
- ✅ Production-ready code

# Actor input Schema

## `profiles` (type: `array`):

Enter Twitter usernames or URLs (e.g. elonmusk or https://x.com/elonmusk).

## `useApifyProxy` (type: `boolean`):

Recommended to bypass Twitter rate limits.

## `apifyProxyGroups` (type: `array`):

Select proxy networks (RESIDENTIAL is highly recommended).

## Actor input object example

```json
{
  "profiles": [
    "elonmusk"
  ],
  "useApifyProxy": true,
  "apifyProxyGroups": [
    "RESIDENTIAL"
  ]
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "profiles": [
        "elonmusk"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("datapilot/twitter-x-profile-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "profiles": ["elonmusk"] }

# Run the Actor and wait for it to finish
run = client.actor("datapilot/twitter-x-profile-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "profiles": [
    "elonmusk"
  ]
}' |
apify call datapilot/twitter-x-profile-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,datapilot/twitter-x-profile-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/kQPdPm8LZhZbqbkNA/builds/wmrdjBOV5NIHM73pX/openapi.json