# X (Twitter) Scraper - Tweets, Users & Search (`titan_network/dataclawd-twitter-data-collector`) Actor

Scrape X/Twitter tweets, users, followers, search, lists, communities, trends, notifications, and DMs through 57 API endpoints with automatic pagination. Powered by Titan Network.

- **URL**: https://apify.com/titan\_network/dataclawd-twitter-data-collector.md
- **Developed by:** [Titan Network](https://apify.com/titan_network) (community)
- **Categories:** Developer tools
- **Stats:** 2 total users, 1 monthly users, 84.2% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.18 / 1,000 item collecteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Dataclawd X/Twitter Data Collector

A reliable Actor that collects **X/Twitter social data** through the **Dataclawd API** — a managed social-data API with API-key authentication, usage reporting and prepaid credits. This Actor is a thin client: it calls the endpoint you choose (or maps a friendly **task** to the right endpoint automatically), follows pagination cursors automatically, extracts each item, and writes one row per item into the default **Dataset** (each successful row is also the pay-per-event billing event).

It covers users, tweets, search & trends, followers/following, lists, communities, notifications and DM inboxes — 57 read/data endpoints in total (see the [endpoint list](#supported-endpoints)).

> Powered by Dataclawd API · Not affiliated with X Corp. or Twitter.

### Quick start

1. Open the Actor's **Input** tab.
2. Pick a **task** (collection task), e.g. `userTweets` (a user's tweets) — the endpoint is selected automatically.
3. Fill the endpoint's parameters in **params**, e.g.:
   ```json
   { "userId": "44196397" }
   ```
4. Optionally set `maxItems` / `maxPages` to control volume, or add multiple objects to **targets** to batch-collect.
5. Click **Run** and open the **Dataset** tab when it finishes — every collected item is one row with the raw JSON in the `data` field.

### Input

| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `task` | enum | — | **Recommended.** Collection intent, auto-maps to an endpoint (see Tasks below). Leave empty only if you set `endpoint` explicitly. |
| `endpoint` | enum | `auto` | Advanced. `auto` = resolve from `task`; or set an explicit endpoint to override. |
| `params` | object | — | Endpoint-specific request parameters as JSON, e.g. `{"screenName":"elonmusk"}`. Applied to every target. |
| `targets` | array of object | *(empty)* | Batch mode: each element is a param set (overrides `params`), collected one after another. |
| `authToken` | string | *(empty)* | X web-login `auth_token` cookie. Required for account-scoped endpoints (notifications, DMs). |
| `ct0` | string | *(empty)* | X web-login `ct0` cookie (some account-scoped endpoints). |
| `proxyUrl` | string | *(empty)* | Proxy used by Dataclawd's server when fetching X. |
| `httpMethod` | enum | `POST` | `POST` or `GET` (both are defined in the Dataclawd OpenAPI). |
| `maxItems` | integer | `100` | Stop after this many items in total (max 100 000). |
| `maxPages` | integer | `10` | Max pages per target (max 100). |
| `startCursor` | string | *(empty)* | Resume from a previous cursor instead of page 1. |
| `itemsPath` | string | *(empty)* | Dot-path to the item list in the response, e.g. `data.items`. Leave empty for auto-detection (largest array). |
| `nextCursorPath` | string | *(empty)* | Dot-path to the next-page cursor. Leave empty for auto-detection (`bottom`/`next`/`cursor`/`after`…). |
| `paginationParam` | enum | `auto` | Query param used for paging: `auto` (cursor), `after`, or `none` (no paging). |

#### Supported endpoints

#### Tasks

Pick a `task` and the Actor selects the endpoint for you. Fill the endpoint's parameters in `params` (or `targets` for batch).

| Task | Endpoint | Required params |
| --- | --- | --- |
| `userProfile` User profile | `userByScreenNameV2` | `screenName` |
| `userTweets` User tweets | `userTweetsV2` | `userId` |
| `userMedia` User media | `userMedia` | `userId` |
| `userLikes` User likes | `userLikeV2` | `userId` |
| `tweetDetail` Tweet detail | `tweetSimple` | `id` |
| `search` Search tweets | `search` | `words` |
| `trending` Global trends | `trending` | — |
| `followers` User followers | `followersListV2` | `userId` |
| `following` User following | `followingsListV2` | `userId` |
| `listTweets` List tweets | `listLatestTweetsTimelineV2` | `listId` |
| `communityTweets` Community tweets | `CommunitiesTweetsTimelineV2` | `communityId` |
| `notifications` Notifications | `notificationsAll` | needs `authToken` |
| `dmInbox` DM inbox | `getDMSListV2` | needs `authToken` |

| Group | Endpoints |
| --- | --- |
| Users | `userByScreenNameV2`, `uerByIdRestIdV2`, `usersByIdRestIds`, `usernameChanges`, `highlightsV2`, `UserArticlesTweets` |
| Tweets | `tweetSimple`, `tweetTimeline`, `tweetResultsByRestIds`, `userTweetsV2`, `userTimeline`, `userMedia`, `userLikeV2`, `userTweetReply`, `favoritesList` |
| Tweet interactions | `favoritersV2`, `retweetersV2`, `retweetersIds`, `quotesV2` |
| Search & trends | `search`, `searchBox`, `trends`, `trending`, `explore`, `news`, `sports`, `entertainment` |
| Followers / following | `followersListV2`, `followingsListV2`, `followersIds`, `followingsIds`, `friendshipsShow`, `blueVerifiedFollowersV2` |
| Lists | `listByRestIdV2`, `listByUserIdOrScreenName`, `listFollowersByListIdV2`, `listLatestTweetsTimelineV2`, `listMembersByListIdV2`, `listSearchTimeline` |
| Communities | `CommunitiesSearchV2`, `CommunitiesTweetsTimelineV2`, `CommunitiesMemberV2`, `CommunitiesTimelineV2`, `CommunityQueryV2`, `CommunitiesFetchOneQuery`, `TopicListV2`, `getCommunitiesByScreenName` |
| Space (optional auth) | `AudioSpaceById` |
| Notifications (auth) | `notificationsAll`, `notificationsMentions`, `notificationsVerified`, `notificationsV2` |
| DM inbox (auth) | `getDMSListV2`, `getUpdateDMSV2`, `getDMSInitIdV2`, `getDMS`, `getCanSendDMS` |

Each endpoint's required parameters are listed in the **endpoint** dropdown description and in `docs/dataclawd-api-reference.md` of the project repository.

### Output (Dataset — one row per collected item)

| Field | Description |
| --- | --- |
| `platform` | Always `twitter`. |
| `endpoint` | The endpoint that produced this row. |
| `status` | `completed` (data collected) or `failed` (request-level failure). |
| `target` | Human-readable target summary, e.g. `userId=44196397`. |
| `page` | Page number this item came from (1-based). |
| `itemIndex` | Sequential index within the run (1-based). |
| `cursor` | Cursor used to fetch that page (`-1` = first page). |
| `data` | **The collected item** — the raw JSON returned by Dataclawd (tweet, user, etc.). |
| `errorStage` | Where a failure happened: `connect_failed` / `server_error` / `auth_error` / `rate_limited` / `request_error` / `parse_error` / `empty_result`. |
| `errorCode` | HTTP status code (if any), e.g. `429`. |
| `error` | Error message on failed rows. |

#### Run status & status message

While running, the Actor's **status message** is updated periodically (`Collecting …` → `Collecting …: wrote n items` → `Done: …`). The run **succeeds** (`SUCCEEDED`) as soon as **at least one item** is collected. If every target fails or returns nothing, the run ends `FAILED` with an explanatory message. Billing happens only for `completed` rows.

### How to verify the results

1. **Run status**: the run is `SUCCEEDED` when at least one item was written; the status message shows live progress.
2. **Per-item state**: open the run's **Dataset** — every collected item has `status=completed` with the raw JSON in `data`; failed rows explain why (`errorStage` / `errorCode` / `error`).
3. **Spot-check against the API**: re-run one target in the Dataclawd API Console and compare with the row's `data`.
4. **Billing**: only `completed` rows carry the `item-collected` pay-per-event charge.

> Developer note: for automated checks (run status, row schema, optional Dataclawd API replay), use `python3 tools/verify_output.py --run <RUN_ID> [--api]`.

### Pricing

Pay-per-event (PPE): **one `item-collected` event per successfully collected item**. Failed rows do **not** carry this event.

> Publishing note: define an `item-collected` event with your price in the Actor's **Monetization** tab. The platform's built-in synthetic event `apify-default-dataset-item` is charged automatically per default-dataset item when PPE is enabled; set its price to `0` if you don't want a per-item base fee.

### Configuration (publisher)

The following environment variables are configured on the Actor's version settings and are not visible to end users:

- `DATACLAWD_API_HOST` — Dataclawd API base URL (e.g. `https://api.dataclawd.example`).
- `DATACLAWD_API_KEY` — Dataclawd API key (sent as `X-API-Key`).

### Disclaimer

This Actor is not affiliated with, endorsed by, or sponsored by X Corp. or Twitter. Collect only data you are authorized to collect, and comply with X's Terms of Service, the Dataclawd service terms, and applicable law.

***

# Actor input Schema

## `task` (type: `string`):

The X/Twitter data scenario to collect. Pick one and the Actor maps it to the corresponding endpoint; parameters go in "Request params" below. For example, User tweets → {"userId":"44196397"}. Leave it empty to manually specify the endpoint under "Advanced options".

## `endpoint` (type: `string`):

Advanced options: default auto (the endpoint is auto-selected by "Collection task"); you can also manually specify a specific endpoint to override task.

## `params` (type: `object`):

Endpoint-specific request params (JSON object), applied to all targets as defaults. For example {"screenName": "elonmusk"} or {"userId": "44196397"}.

## `targets` (type: `array`):

Batch collection: each element is a set of request params (overriding the default params), and the Actor collects each target one by one. Leave empty to collect only once using params. For example \[{"screenName":"elonmusk"},{"screenName":"SpaceX"}].

## `authToken` (type: `string`):

auth\_token from the X web login cookie. Required for account-level endpoints such as notifications/DMs; can be left empty for public endpoints. How to get it: from the Dataclawd login API, or copy it from the x.com logged-in cookie.

## `ct0` (type: `string`):

ct0 from the X web login cookie (some account-level endpoints require it together with auth\_token).

## `proxyUrl` (type: `string`):

Proxy address used by the Dataclawd server when requesting X/Reddit.

## `httpMethod` (type: `string`):

HTTP method used when calling the Dataclawd endpoint (OpenAPI defines both POST and GET; default POST).

## `maxItems` (type: `integer`):

Maximum number of items written to the Dataset during the entire run; collection stops once this is reached.

## `maxPages` (type: `integer`):

Maximum number of pages to fetch per target (prevents endless pagination).

## `startCursor` (type: `string`):

Start paginating from the given cursor (continue collecting data left unfinished last time). Leave empty to start from the first page.

## `itemsPath` (type: `string`):

Dot path that specifies the location of the item list in the response, e.g. data.items. Leave empty to auto-detect (prefers the largest array).

## `nextCursorPath` (type: `string`):

Dot path that specifies the location of the next-page cursor in the response. Leave empty to auto-detect by key name (bottom/next/cursor/after, etc.).

## `paginationParam` (type: `string`):

Query param used for pagination: auto selects automatically by endpoint (default cursor); after is for Reddit-style pagination; none disables pagination.

## Actor input object example

```json
{
  "task": "userProfile",
  "endpoint": "auto",
  "params": {
    "screenName": "elonmusk"
  },
  "targets": [],
  "httpMethod": "POST",
  "maxItems": 5,
  "maxPages": 1,
  "paginationParam": "auto"
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset of collected X/Twitter records — one item per record.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "task": "userProfile",
    "endpoint": "auto",
    "params": {
        "screenName": "elonmusk"
    },
    "targets": [],
    "maxItems": 5,
    "maxPages": 1
};

// Run the Actor and wait for it to finish
const run = await client.actor("titan_network/dataclawd-twitter-data-collector").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "task": "userProfile",
    "endpoint": "auto",
    "params": { "screenName": "elonmusk" },
    "targets": [],
    "maxItems": 5,
    "maxPages": 1,
}

# Run the Actor and wait for it to finish
run = client.actor("titan_network/dataclawd-twitter-data-collector").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "task": "userProfile",
  "endpoint": "auto",
  "params": {
    "screenName": "elonmusk"
  },
  "targets": [],
  "maxItems": 5,
  "maxPages": 1
}' |
apify call titan_network/dataclawd-twitter-data-collector --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,titan_network/dataclawd-twitter-data-collector"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/OMyqLDGyuRgBLRWPL/builds/q3sXDebh7r8UjznQ6/openapi.json