# X (Twitter) Post & Tweet Scraper-MCP & API for AI Agents, LLM's (`codeperfection/x-twitter-scraper`) Actor Scrape posts and tweets from X (Twitter): search, timelines, URLs, and threads, no official API key. Structured JSON for apps, AI agents, and LLM/RAG. You pay per unique tweet delivered, no duplicates; empty runs cost nothing, with no mock or placeholder data. Unofficial, not affiliated with X Corp. - **URL**: https://apify.com/codeperfection/x-twitter-scraper.md - **Developed by:** [CodePerfection](https://apify.com/codeperfection) (community) - **Categories:** Social media, AI, Agents - **Stats:** 3 total users, 0 monthly users, 95.7% runs succeeded, 1 bookmarks - **User rating**: 5.00 out of 5 stars ## Pricing $0.49 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README **Real tweets from X (Twitter) — with billing you can audit.** Search posts and tweets, expand threads, pull profiles and single tweets, and get clean structured JSON. You pay **per unique real tweet delivered** — every run ends with a receipt where **accepted = charged = dataset items**. Empty runs cost **$0**. No mock data, no duplicates, no phantom charges — ever. No official Twitter/X API key required. Output is ready for apps, spreadsheets, OSINT and research workflows, AI agents, and LLM/RAG pipelines. > **Unofficial.** This Actor is not affiliated with, endorsed by, or sponsored by X Corp or Twitter. "X" and "Twitter" are trademarks of their respective owners. --- ### Twitter (X) scraper with an auditable per-tweet bill Most X scrapers lead on price and speed and stay quiet about *what you actually get billed for*. This one leads on the opposite: - **Pay per unique real tweet.** One charge per delivered tweet — deduplicated by tweet ID across pages, sources, and search terms. You are never billed for the same tweet twice. - **$0 on an empty run.** No results found? You pay nothing. No minimum, no "we returned some placeholder rows so we could charge you." - **A receipt every run.** The run writes a reconciliation record where `accepted = charged = dataset items`. If those three don't independently match, the run fails instead of billing you for something it can't prove. - **No mock/placeholder data.** Every row is a real tweet. Filtered-out, deduplicated, and unroutable inputs are disclosed in the run log — not padded into your billed dataset. ### Tweets you are never charged for | Not billed | Why | |---|---| | Empty runs (0 results) | You pay for tweets, and there weren't any | | Duplicate tweets | Deduplicated by ID before delivery | | Filtered-out tweets | Removed by your filters, never delivered | | Unroutable inputs | Disclosed in `SKIPPED_SEEDS`, never charged | | The free dry-run | Validate your output shape for $0 (see below) | | Output shaping | Presets, projections, and jq transforms are included | ### Pricing: $0.49 per 1,000 tweets **$0.49 per 1,000 tweets** — that's `$0.00049` per unique tweet delivered. Platform usage (compute, proxy, storage) is included, so your invoice is one clean per-tweet line. | Tweets delivered | You pay | |---|---| | 0 (empty run) | **$0.00** | | 100 | $0.049 | | 1,000 | $0.49 | | 10,000 | $4.90 | ### Scrape tweets by keyword, hashtag, handle, URL, or thread No 3-Actor split, no bouncing between listings. One input covers every X search mode, and everything is merged and deduplicated into one billed result set: - **Search** — full X advanced-search syntax (`from:`, `filter:`, `since:`/`until:`, `min_faves:`, language, media, hashtags, and more). This is a full Twitter search scraper. - **Handles** — scrape a list of accounts' tweets (a Twitter profile scraper: pull a user's tweets by handle). - **URLs** — a tweet URL (that exact tweet), a profile URL, a search URL, a list URL, or a hashtag URL. - **Threads** — expand a conversation into its replies. ### Filter tweets by date, likes, language, and media Every filter you set is applied as an X advanced-search operator **and re-checked on each returned tweet**. A tweet that doesn't genuinely match is dropped (counted as filtered, never delivered, never charged) rather than leaked through. If the data can't verify a filter on a returned tweet, that's disclosed in the run log — it's never silently ignored. Date range, from/reply-to handle, mentions, minimum likes/retweets/replies, language, media type, verified / blue-verified, and quote-only are all enforced this way. Tweet IDs are returned as strings, so 64-bit IDs are never rounded; quoted and retweeted tweets are fully expanded. ### Export tweets to CSV or JSON Shape and download the data however you need. It never changes your bill — you always get exactly **one row per tweet**: - **Ready-made formats** — `full` (complete nested JSON), `compact` (the fields most people use), or `csvFlat` (CSV columns for spreadsheets and Excel). - **Pick the fields you want (no code)** — keep only the fields you need, rename them, reorder them, flatten nested fields into columns, and add your own tag columns. - **Custom formulas (advanced)** — a `jq` expression for calculated fields like total engagement, ratios, and conditionals. - **Free preview** — set `validateTransformOnly: true` to see exactly what one row will look like against a sample tweet, with **no scrape, no rows, and no charge.** ### No Twitter or X API key required Scrape Twitter without an API key. There is no official X API access, developer account, app registration, or bearer token to set up, and no rate-limit tier to buy. Provide your search terms, handles, or URLs and run — the Actor handles collection and hands back clean tweets. ### Twitter data for AI agents, OSINT, and LLM/RAG Clean, consistently-shaped JSON per tweet — author, engagement counts, media, and fully-expanded quoted & retweeted tweets — or projected/jq'd down to exactly the fields you need. No official API key to manage. As a standard pay-per-event Actor with limited permissions, it's discoverable and callable by AI agents and assistants — Claude, ChatGPT, Gemini, Codex, and any MCP client — via MCP and agentic payments, as well as from the API, UI, and scheduler. It's a clean building block for OSINT and research, monitoring, and real-time LLM/RAG workflows. ### Scrape tweets from your AI agent (MCP) Call this scraper straight from Claude, ChatGPT, Cursor, or any MCP client — no code. Your agent finds it as a tool, runs it, and reads the tweets back for you. **1. Connect.** Point your MCP client at this URL: `https://mcp.apify.com/?tools=codeperfection/x-twitter-scraper` In Claude Desktop (Settings → Developer → Edit Config), that is: ```json { "mcpServers": { "x-twitter-scraper": { "url": "https://mcp.apify.com/?tools=codeperfection/x-twitter-scraper" } } } ```` Claude Desktop signs you in to Apify through your browser on first use. For Cursor, VS Code, and other token-based clients, add the same `url` plus a header: `"Authorization": "Bearer YOUR_APIFY_TOKEN"`. **2. Just ask.** Talk to your agent in plain language: > "Get the 20 latest tweets from @NatGeo." > > "Search X for #AI posts in English with at least 100 likes." > > "Pull the replies in this thread: https://x.com/NatGeo/status/…" Your agent runs the scraper and fetches the results for you — no steps to manage. Billing is identical to the UI and API: **$0.49 / 1,000 tweets**, and empty results cost **$0**. ### Quick start Search tweets by keyword or advanced-search operators: ```json { "searchTerms": ["\"climate change\" min_faves:100 lang:en", "#ai filter:images"], "maxTweets": 100, "sort": "Latest" } ``` Scrape a user's tweets (one or more handles): ```json { "twitterHandles": ["NatGeo", "BBCEarth"], "maxTweets": 50 } ``` A single tweet, a profile, and a thread, by URL / conversation ID: ```json { "startUrls": ["https://x.com/NatGeo/status/2075663637319217267", "https://x.com/NatGeo"], "conversationIds": ["2073460397902827575"] } ``` A hashtag or a saved search, by URL: ```json { "startUrls": ["https://x.com/hashtag/ai", "https://x.com/search?q=climate%20min_faves%3A100&f=live"], "maxTweets": 200 } ``` Advanced filters (all provably enforced): ```json { "searchTerms": ["climate"], "fromUser": "NatGeo", "startDate": "7 days", "minFaves": 100, "language": "en", "media": "images" } ``` ### Output Each item is one real tweet (the `full` preset shown trimmed): ```json { "id": "2075663637319217267", "url": "https://x.com/NatGeo/status/2075663637319217267", "text": "...", "created_at": "2022-10-26T13:00:00.000Z", "lang": "en", "like_count": 12043, "retweet_count": 1876, "reply_count": 402, "quote_count": 55, "view_count": 980112, "hashtags": [], "mentions": [], "urls": [], "media": [], "author": { "username": "NatGeo", "followers": 27926325, "verified": false, "blue_verified": true }, "quoted": null, "retweeted": null } ``` Prefer columns? Use `outputPreset: "csvFlat"`, or a projection / jq to keep exactly the fields you want. ### The receipt At the end of every run, the key-value store holds a `RECEIPT` record: ```json { "billingEvent": "apify-default-dataset-item", "acceptedUniqueTweets": 100, "chargedTweets": 100, "datasetItems": 100, "reconciled": true, "reconciliationStatus": "ok" } ``` `datasetItems` is read back independently from the dataset — so the receipt genuinely reconciles what you were charged against what was stored, rather than restating one number twice. Per-term `SUMMARY_*` records break down what each search delivered vs. fetched. ### Free plan Free-plan (non-paying Apify) accounts get a capped preview of up to **5 tweets per run** so you can try the Actor end to end — the full receipt is still written. Upgrade to any paid Apify plan for uncapped runs. ### FAQ **How do I scrape tweets without a Twitter/X API key?** Provide search terms, handles, or X URLs and run. There is no official X API, developer account, app, or bearer token to set up — the Actor collects the tweets for you. **Can I export tweets to CSV or JSON?** Yes. Set `outputPreset: "csvFlat"` for flat spreadsheet columns, keep `full` or `compact` for nested JSON, or use the field picker / jq to choose exactly the columns you want. Shaping the output never changes your bill. **How do I scrape all tweets from a user?** Add the account to `twitterHandles` (or pass its profile URL in `startUrls`) and raise `maxTweets`. Each handle runs as a `from:` search, so your date, engagement, and language filters still apply. **How do I scrape tweets from a hashtag?** Put the term in `searchTerms` (for example `#ai`), or pass a hashtag URL like `https://x.com/hashtag/ai` in `startUrls`. **How much does it cost?** $0.49 per 1,000 unique tweets delivered, which is `$0.00049` each. Empty runs cost **$0**, and platform usage (compute, proxy, storage) is included. **Do you charge for tweets you didn't return?** No. Billing is one charge per unique real tweet actually delivered to your dataset. Empty runs, duplicates, and filtered-out tweets cost nothing. **What if a run finds nothing?** It costs **$0** and reports `no results` — no minimum, no placeholder rows. **How do I control cost?** `maxTweets` caps per search term; `maxTotalTweets` caps the whole run; and Apify's per-run max-cost limit is respected. **Why did a run say "partial"?** You hit a cap (e.g. `maxTweets`) and more results existed — the run tells you exactly how many were left so you can raise the cap. It's honesty, not an error. ### Support Found a bug or need a field added? Open an issue on the Actor and it'll get looked at. # Actor input Schema ## `searchTerms` (type: `array`): One or more searches to run. Each term supports full X advanced-search syntax (for example "web scraping", "#ai", "apify lang:en"). Terms are paginated independently, and results are deduplicated by tweet ID across all terms, so you are not charged for the same tweet twice in a run. Provide at least one of: searchTerms, twitterHandles, startUrls, or conversationIds. ## `twitterHandles` (type: `array`): Scrape tweets authored by these accounts (handle without the @). Each handle runs as a from: search, so your date, engagement, and language filters still apply. Deduplicated with everything else in the run. ## `startUrls` (type: `array`): Scrape directly from X URLs. Supported: a tweet URL (x.com/user/status/ID for that exact tweet), a profile URL (x.com/user for that user's tweets), a search URL (x.com/search?q=... for that query), a list URL (x.com/i/lists/ID), or a hashtag URL (x.com/hashtag/tag). An unrecognized URL is recorded in the run's SKIPPED\_SEEDS record and not charged. ## `conversationIds` (type: `array`): Expand one or more X threads. Give a conversation ID (the thread root tweet's ID) to get the replies in that conversation. Your filters still apply, and replies are deduplicated across the run like everything else. ## `sort` (type: `string`): "Latest" is reverse-chronological (newest first). "Top" is X's relevance ranking. ## `maxTweets` (type: `integer`): Hard cap on real tweets returned per search term. You are charged only for the real tweets delivered, so a search that finds fewer than this (or none) costs proportionally less (or nothing). ## `maxTotalTweets` (type: `integer`): Optional global cap on real tweets delivered across all search terms combined (maxTweets is the per-term cap). The run stops once this total is reached. Leave empty for no global cap. ## `startDate` (type: `string`): Only tweets at or after this date (UTC). Accepts an absolute date (YYYY-MM-DD or ISO-8601 timestamp) or a relative value like "7 days" or "1 month". Applied server-side and re-checked on every returned tweet. ## `endDate` (type: `string`): Only tweets strictly before this date (UTC). Accepts an absolute date (YYYY-MM-DD or ISO-8601 timestamp) or a relative value like "1 day". ## `fromUser` (type: `string`): Only tweets authored by this account (handle without the @). Applied server-side (from:) and re-checked against the tweet author. ## `toUser` (type: `string`): Only tweets that are a direct reply to this account (handle without the @). Matches the reply target, not arbitrary mentions. ## `minFaves` (type: `integer`): Only tweets with at least this many likes. ## `minRetweets` (type: `integer`): Only tweets with at least this many retweets. ## `minReplies` (type: `integer`): Only tweets with at least this many replies. ## `language` (type: `string`): Only tweets in this language, as an ISO 639-1 code (for example en, ru, es, de). A tweet with no detected language is dropped rather than returned. ## `media` (type: `string`): Restrict to tweets that carry media of the chosen kind. ## `onlyVerified` (type: `boolean`): Only tweets from verified accounts (legacy verified or X Premium/blue). ## `onlyBlueVerified` (type: `boolean`): Only tweets from X Premium (blue) accounts. Stricter than 'Verified authors only'. ## `onlyQuote` (type: `boolean`): Only quote tweets (a tweet that embeds another tweet). Applied with filter:quote and re-checked on every returned tweet. ## `mentioning` (type: `string`): Only tweets that mention this account (handle without the @). Applied server-side and re-checked against each tweet's mentions. ## `includeSearchTerms` (type: `boolean`): Add a `search_term` field to every tweet recording which of your search terms found it. Useful when running several terms in one run. ## `outputPreset` (type: `string`): The base shape of every delivered tweet, before any advanced transform. "Full" is the complete nested JSON (all fields). "Compact" is a lean high-value JSON subset (author kept nested). "CSV flat" is flat scalar columns for CSV and spreadsheet tools (author flattened, arrays joined; structurally flat, but not formula-injection-sanitized, so see the docs if you import into a formula-evaluating app). One item out per tweet: the preset never changes how many tweets you get or what you are charged. ## `flattenAuthor` (type: `boolean`): Legacy shortcut: replace the nested `author` object with flat `author_*` columns (author\_followers, author\_verified, ...). Applies to the Full and Compact presets; the CSV flat preset already flattens the author. Leave off to keep the nested JSON shape. ## `transformMode` (type: `string`): Optional advanced step applied on top of the format. "None" delivers the format unchanged. "Pick the fields you want" lets you choose, order, rename, and flatten fields and add constant columns without code. "Custom formula (jq)" runs an advanced jq program for calculated and derived fields (sums, ratios, conditionals). None of them can change the number of items: every accepted tweet still produces exactly one output item. Settings for the inactive modes are checked for validity but not applied. ## `projection` (type: `object`): Choose, order, rename, flatten, and add string constants without changing the number of output items. Ignored unless Transform mode is 'Pick the fields you want'. ## `jqExpression` (type: `string`): An advanced jq expression that reshapes each tweet into exactly one output object, for calculated and derived fields (engagement sums, ratios, conditionals, renames) beyond what the field picker can express. It runs on the preset-shaped tweet and must return exactly one JSON object per tweet: it cannot filter, drop, or fan out. An expression that returns nothing, a scalar, null, or multiple values is rejected before any paid run (a free dry-run is available). Module loading and environment or extra-input builtins are not permitted. Ignored unless Transform mode is jq. Example: {id, text, engagement: (.like\_count + .retweet\_count + .reply\_count)} ## `jqVariables` (type: `array`): Optional string variables passed to the jq expression as $name (for example a campaign tag reused across the expression). Ignored unless Transform mode is jq. ## `outputCardinality` (type: `string`): Contract guarantee: exactly one dataset item is produced for each delivered unique tweet. Only "oneToOne" is supported. ## `validateTransformOnly` (type: `boolean`): Validate the selected preset and transform against a fixed canonical sample tweet and write a preview to the run's Output, with no scrape, no dataset items, and no charge. Use it to check your transform before a paid run. ## Actor input object example ```json { "searchTerms": [ "web scraping", "#ai" ], "twitterHandles": [ "NatGeo", "BBCEarth" ], "startUrls": [ { "url": "https://x.com/NatGeo/status/2075663637319217267" }, { "url": "https://x.com/NatGeo" } ], "conversationIds": [ "2073460397902827575" ], "sort": "Latest", "maxTweets": 10, "media": "", "onlyVerified": false, "onlyBlueVerified": false, "onlyQuote": false, "includeSearchTerms": false, "outputPreset": "full", "flattenAuthor": false, "transformMode": "none", "projection": { "selection": "onlyListed", "fields": [], "flatten": false, "constants": [], "arrays": "keep", "arrayJoinSeparator": ",", "missingFields": "omit" }, "jqExpression": ".", "jqVariables": [], "outputCardinality": "oneToOne", "validateTransformOnly": false } ``` # Actor output Schema ## `tweets` (type: `string`): The scraped tweets stored in the default dataset, one item per unique real tweet delivered. See the dataset schema for per-field titles, descriptions, and examples (id, url, text, created\_at, engagement counts, author, media, quoted, retweeted, and more). ## `receipt` (type: `string`): A per-run reconciliation record in the default key-value store proving you were billed only for tweets actually delivered: acceptedUniqueTweets equals chargedTweets equals datasetItems, and reconciled is true only when the three counts independently match. Also reports userPlan and any free-plan cap. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "searchTerms": [ "apify" ], "maxTweets": 10, "jqExpression": "." }; // Run the Actor and wait for it to finish const run = await client.actor("codeperfection/x-twitter-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "searchTerms": ["apify"], "maxTweets": 10, "jqExpression": ".", } # Run the Actor and wait for it to finish run = client.actor("codeperfection/x-twitter-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "searchTerms": [ "apify" ], "maxTweets": 10, "jqExpression": "." }' | apify call codeperfection/x-twitter-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=codeperfection/x-twitter-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "X (Twitter) Post & Tweet Scraper-MCP & API for AI Agents, LLM's", "description": "Scrape posts and tweets from X (Twitter): search, timelines, URLs, and threads, no official API key. Structured JSON for apps, AI agents, and LLM/RAG. You pay per unique tweet delivered, no duplicates; empty runs cost nothing, with no mock or placeholder data. Unofficial, not affiliated with X Corp.", "version": "1.0", "x-build-id": "EvK0V1jwxxlNiolpS" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/codeperfection~x-twitter-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-codeperfection-x-twitter-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/codeperfection~x-twitter-scraper/runs": { "post": { "operationId": "runs-sync-codeperfection-x-twitter-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/codeperfection~x-twitter-scraper/run-sync": { "post": { "operationId": "run-sync-codeperfection-x-twitter-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "searchTerms": { "title": "Search terms", "type": "array", "description": "One or more searches to run. Each term supports full X advanced-search syntax (for example \"web scraping\", \"#ai\", \"apify lang:en\"). Terms are paginated independently, and results are deduplicated by tweet ID across all terms, so you are not charged for the same tweet twice in a run. Provide at least one of: searchTerms, twitterHandles, startUrls, or conversationIds.", "items": { "type": "string", "minLength": 1 } }, "twitterHandles": { "title": "Twitter/X handles", "type": "array", "description": "Scrape tweets authored by these accounts (handle without the @). Each handle runs as a from: search, so your date, engagement, and language filters still apply. Deduplicated with everything else in the run.", "items": { "type": "string", "minLength": 1 } }, "startUrls": { "title": "Start URLs", "type": "array", "description": "Scrape directly from X URLs. Supported: a tweet URL (x.com/user/status/ID for that exact tweet), a profile URL (x.com/user for that user's tweets), a search URL (x.com/search?q=... for that query), a list URL (x.com/i/lists/ID), or a hashtag URL (x.com/hashtag/tag). An unrecognized URL is recorded in the run's SKIPPED_SEEDS record and not charged.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "conversationIds": { "title": "Conversation IDs (threads)", "type": "array", "description": "Expand one or more X threads. Give a conversation ID (the thread root tweet's ID) to get the replies in that conversation. Your filters still apply, and replies are deduplicated across the run like everything else.", "items": { "type": "string" } }, "sort": { "title": "Sort by", "enum": [ "Latest", "Top" ], "type": "string", "description": "\"Latest\" is reverse-chronological (newest first). \"Top\" is X's relevance ranking.", "default": "Latest" }, "maxTweets": { "title": "Maximum tweets per search term", "minimum": 1, "maximum": 10000, "type": "integer", "description": "Hard cap on real tweets returned per search term. You are charged only for the real tweets delivered, so a search that finds fewer than this (or none) costs proportionally less (or nothing).", "default": 100 }, "maxTotalTweets": { "title": "Maximum tweets total (across all terms)", "minimum": 1, "type": "integer", "description": "Optional global cap on real tweets delivered across all search terms combined (maxTweets is the per-term cap). The run stops once this total is reached. Leave empty for no global cap." }, "startDate": { "title": "From date", "type": "string", "description": "Only tweets at or after this date (UTC). Accepts an absolute date (YYYY-MM-DD or ISO-8601 timestamp) or a relative value like \"7 days\" or \"1 month\". Applied server-side and re-checked on every returned tweet." }, "endDate": { "title": "To date", "type": "string", "description": "Only tweets strictly before this date (UTC). Accepts an absolute date (YYYY-MM-DD or ISO-8601 timestamp) or a relative value like \"1 day\"." }, "fromUser": { "title": "From this @handle", "type": "string", "description": "Only tweets authored by this account (handle without the @). Applied server-side (from:) and re-checked against the tweet author." }, "toUser": { "title": "Replying to this @handle", "type": "string", "description": "Only tweets that are a direct reply to this account (handle without the @). Matches the reply target, not arbitrary mentions." }, "minFaves": { "title": "Minimum likes", "minimum": 0, "type": "integer", "description": "Only tweets with at least this many likes." }, "minRetweets": { "title": "Minimum retweets", "minimum": 0, "type": "integer", "description": "Only tweets with at least this many retweets." }, "minReplies": { "title": "Minimum replies", "minimum": 0, "type": "integer", "description": "Only tweets with at least this many replies." }, "language": { "title": "Language", "type": "string", "description": "Only tweets in this language, as an ISO 639-1 code (for example en, ru, es, de). A tweet with no detected language is dropped rather than returned." }, "media": { "title": "Media", "enum": [ "", "any", "images", "videos" ], "type": "string", "description": "Restrict to tweets that carry media of the chosen kind.", "default": "" }, "onlyVerified": { "title": "Verified authors only", "type": "boolean", "description": "Only tweets from verified accounts (legacy verified or X Premium/blue).", "default": false }, "onlyBlueVerified": { "title": "Blue-verified authors only", "type": "boolean", "description": "Only tweets from X Premium (blue) accounts. Stricter than 'Verified authors only'.", "default": false }, "onlyQuote": { "title": "Quote tweets only", "type": "boolean", "description": "Only quote tweets (a tweet that embeds another tweet). Applied with filter:quote and re-checked on every returned tweet.", "default": false }, "mentioning": { "title": "Mentioning this @handle", "type": "string", "description": "Only tweets that mention this account (handle without the @). Applied server-side and re-checked against each tweet's mentions." }, "includeSearchTerms": { "title": "Tag each tweet with its search term", "type": "boolean", "description": "Add a `search_term` field to every tweet recording which of your search terms found it. Useful when running several terms in one run.", "default": false }, "outputPreset": { "title": "Output preset", "enum": [ "full", "compact", "csvFlat" ], "type": "string", "description": "The base shape of every delivered tweet, before any advanced transform. \"Full\" is the complete nested JSON (all fields). \"Compact\" is a lean high-value JSON subset (author kept nested). \"CSV flat\" is flat scalar columns for CSV and spreadsheet tools (author flattened, arrays joined; structurally flat, but not formula-injection-sanitized, so see the docs if you import into a formula-evaluating app). One item out per tweet: the preset never changes how many tweets you get or what you are charged.", "default": "full" }, "flattenAuthor": { "title": "Flatten author into columns (for CSV/Excel)", "type": "boolean", "description": "Legacy shortcut: replace the nested `author` object with flat `author_*` columns (author_followers, author_verified, ...). Applies to the Full and Compact presets; the CSV flat preset already flattens the author. Leave off to keep the nested JSON shape.", "default": false }, "transformMode": { "title": "Transform mode", "enum": [ "none", "projection", "jq" ], "type": "string", "description": "Optional advanced step applied on top of the format. \"None\" delivers the format unchanged. \"Pick the fields you want\" lets you choose, order, rename, and flatten fields and add constant columns without code. \"Custom formula (jq)\" runs an advanced jq program for calculated and derived fields (sums, ratios, conditionals). None of them can change the number of items: every accepted tweet still produces exactly one output item. Settings for the inactive modes are checked for validity but not applied.", "default": "none" }, "projection": { "title": "Field-picker settings -- used only when Transform mode is 'Pick the fields you want'", "type": "object", "description": "Choose, order, rename, flatten, and add string constants without changing the number of output items. Ignored unless Transform mode is 'Pick the fields you want'.", "properties": { "selection": { "title": "Field selection", "type": "string", "description": "Output only the listed fields (plus constants), or keep all preset fields while applying the listed renames and promotions.", "editor": "select", "enum": [ "onlyListed", "allFields" ], "enumTitles": [ "Only listed fields", "All fields, with listed overrides" ], "default": "onlyListed" }, "fields": { "title": "Fields", "type": "array", "description": "Ordered source paths to select or rename, as JSON: a list of {\"from\",\"to\"} objects. `from` is a documented field or dotted object path (for example author.username); a blank or omitted `to` uses the final path segment. Example: [{\"from\":\"author.username\",\"to\":\"handle\"}]", "editor": "json", "default": [], "maxItems": 200, "items": { "type": "object", "additionalProperties": false, "required": [ "from" ], "properties": { "from": { "title": "Source path", "type": "string", "description": "Documented preset field or dotted object path, for example author.username.", "editor": "textfield" }, "to": { "title": "Output name", "type": "string", "description": "Optional literal top-level output key; blank uses the final source-path segment.", "editor": "textfield", "default": "" } } } }, "flatten": { "title": "Flatten nested objects", "type": "boolean", "description": "Recursively flatten remaining nested objects into dot-named keys; arrays are not expanded into rows.", "editor": "checkbox", "default": false }, "constants": { "title": "String constants", "type": "array", "description": "Add constant string columns such as run tags. Keys must be unique and cannot overwrite a projected field.", "editor": "keyValue", "default": [], "maxItems": 100, "items": { "type": "object", "additionalProperties": false, "required": [ "key", "value" ], "properties": { "key": { "type": "string", "title": "Output name", "description": "Top-level output key for this constant column." }, "value": { "type": "string", "title": "Value", "description": "Constant string value placed in that column on every row." } } } }, "arrays": { "title": "Array handling", "type": "string", "description": "How to handle top-level array fields: keep them as JSON arrays, encode them as compact JSON strings, or join their values into one string (nested arrays inside kept objects are unaffected).", "editor": "select", "enum": [ "keep", "json", "join" ], "enumTitles": [ "Keep arrays", "JSON-encode arrays", "Join array values" ], "default": "keep" }, "arrayJoinSeparator": { "title": "Array join separator", "type": "string", "description": "Separator used when array handling is Join; ignored otherwise.", "editor": "textfield", "default": "," }, "missingFields": { "title": "Missing optional fields", "type": "string", "description": "Omit a listed field that is absent from a tweet, or emit it with a null value.", "editor": "select", "enum": [ "omit", "null" ], "enumTitles": [ "Omit the field", "Output null" ], "default": "omit" } }, "default": { "selection": "onlyListed", "fields": [], "flatten": false, "constants": [], "arrays": "keep", "arrayJoinSeparator": ",", "missingFields": "omit" }, "additionalProperties": false }, "jqExpression": { "title": "jq expression -- used only in jq mode", "type": "string", "description": "An advanced jq expression that reshapes each tweet into exactly one output object, for calculated and derived fields (engagement sums, ratios, conditionals, renames) beyond what the field picker can express. It runs on the preset-shaped tweet and must return exactly one JSON object per tweet: it cannot filter, drop, or fan out. An expression that returns nothing, a scalar, null, or multiple values is rejected before any paid run (a free dry-run is available). Module loading and environment or extra-input builtins are not permitted. Ignored unless Transform mode is jq. Example: {id, text, engagement: (.like_count + .retweet_count + .reply_count)}", "default": "." }, "jqVariables": { "title": "jq variables", "maxItems": 50, "type": "array", "description": "Optional string variables passed to the jq expression as $name (for example a campaign tag reused across the expression). Ignored unless Transform mode is jq.", "items": { "type": "object", "additionalProperties": false, "required": [ "key", "value" ], "properties": { "key": { "type": "string", "title": "Variable name", "description": "Variable name, referenced in the jq expression as $name." }, "value": { "type": "string", "title": "Value", "description": "String value bound to the variable for this run." } } }, "default": [] }, "outputCardinality": { "title": "Output cardinality", "enum": [ "oneToOne" ], "type": "string", "description": "Contract guarantee: exactly one dataset item is produced for each delivered unique tweet. Only \"oneToOne\" is supported.", "default": "oneToOne" }, "validateTransformOnly": { "title": "Validate transform only (free dry-run)", "type": "boolean", "description": "Validate the selected preset and transform against a fixed canonical sample tweet and write a preview to the run's Output, with no scrape, no dataset items, and no charge. Use it to check your transform before a paid run.", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```