X (Twitter) Tweet Scraper

Scrape public X (Twitter) tweets from search queries, hashtags, profiles, lists, exact tweet URLs, and reply conversations. This Twitter scraper returns clean, deduplicated tweet data with author profiles, engagement metrics, images, videos, quotes, reposts, and replies. Download the results as JSON, CSV, Excel, XML, RSS, or use them through the Apify API.

No X API key, developer account, browser, or Twitter login is required from the user.

What can you scrape?

• Keyword searches and hashtags, including X advanced-search syntax
• Tweets from one or many public profile handles
• X and Twitter profile, search, list, and exact tweet URLs
• Public replies in a conversation or thread
• Latest results, Top results, or both result streams together
• Tweets filtered by date, language, author, mentions, media, verification, location, and minimum engagement
• Strict author exclusion, with local verification of every returned tweet
• Tweet text, author data, likes, reposts, replies, quotes, views, bookmarks, media, cards, places, and article metadata

The Actor is focused on tweets. It does not return follower lists, following lists, private profiles, or unrelated profile-only rows.

Why use this Twitter scraper?

Zero duplicate tweet IDs

Every run uses exact tweet-ID deduplication across queries, endpoints, pages, Latest and Top results, and profile timelines. If the same tweet is discovered five ways, it is saved once.

One stable output schema

Every dataset row is a tweet with the same top-level shape. The schema is an additive superset of API Dojo's Tweet Scraper field names and types, making migration easier for existing integrations. Structured media details are added without changing the compatible media URL array.

More unique tweets per profile

For profile inputs, Maximize unique results can combine search, tweet, media, and replies timelines. The results are merged and deduplicated, improving coverage without making users reconcile several datasets.

Clear run-wide limits

maxItems is the maximum number of unique tweets for the whole run. It is shared across every submitted query, profile, URL, and conversation—not multiplied by the number of inputs.

Auditable historical coverage

Long date ranges are searched in calendar-month windows. Busy windows are divided into smaller ranges when they saturate, while exact UTC date boundaries and excluded authors are enforced again after parsing. The run stores a COVERAGE manifest in its key-value store so automations can distinguish completed, truncated, and failed windows.

Built for automation and AI agents

The Actor includes documented input, output, and dataset schemas. Run it manually, schedule it, call it through the API, connect it to Make or Zapier, or use it from an MCP-compatible AI workflow.

Input examples

Scrape tweets from a profile

{
  "twitterHandles": ["NASA"],
  "maxItems": 1000,
  "sort": "Latest",
  "maximizeResults": true
}

Search Twitter by keyword or hashtag

{
  "searchTerms": ["artificial intelligence", "#MachineLearning lang:en"],
  "maxItems": 500,
  "sort": "Latest + Top"
}

Filter by date and engagement

{
  "searchTerms": ["climate technology"],
  "start": "2026-01-01",
  "end": "2026-09-01",
  "tweetLanguage": "en",
  "minimumFavorites": 100,
  "minimumRetweets": 20,
  "maxItems": 2000
}

Monitor a brand while excluding its official account

{
  "searchTerms": ["ChangeNOW OR changenow.io OR @ChangeNOW_io"],
  "excludeAuthors": ["ChangeNOW_io"],
  "start": "2025-09-01",
  "end": "2026-09-23",
  "sort": "Latest",
  "maxItems": 10000,
  "includeSearchTerms": true
}

Scrape exact tweets, profiles, searches, or lists from URLs

{
  "startUrls": [
    "https://x.com/NASA",
    "https://x.com/search?q=%23AI&f=live",
    "https://x.com/i/lists/78783491",
    "https://x.com/NASA/status/2036789109363532163"
  ],
  "maxItems": 500
}

Scrape replies from a conversation

{
  "conversationIds": ["2036789109363532163"],
  "maxItems": 500,
  "sort": "Latest"
}

Twitter scraper input

Field	Type	What it does
searchTerms	string[]	Keywords, hashtags, or X advanced-search queries.
startUrls	string[]	X/Twitter profile, search, list, or exact tweet URLs.
twitterHandles	string[]	Profile handles with or without @.
conversationIds	string[]	Conversation IDs for collecting public thread replies.
maxItems	integer	Run-wide maximum number of unique tweets. Default: 1000.
sort	string	Latest, Top, or Latest + Top.
maximizeResults	boolean	Combines additional profile timelines for broader unique coverage.
includeSearchTerms	boolean	Adds the first matching source query to each result.
tweetLanguage	string	ISO 639-1 language code such as en, es, or de.
onlyVerifiedUsers	boolean	Only tweets from verified authors.
onlyTwitterBlue	boolean	Only tweets from X Premium authors.
onlyImage / onlyVideo / onlyQuote	boolean	Restrict results by tweet content type.
author / inReplyTo / mentioning	string	Build author, reply, and mention filters without writing query syntax.
excludeAuthors	string[]	Exclude these authors using both X operators and strict local checks.
minimumRetweets / minimumFavorites / minimumReplies	integer	Minimum engagement thresholds.
start / end	YYYY-MM-DD	Add inclusive since: and exclusive until: search boundaries.
geotaggedNear / withinRadius / geocode / placeObjectId	string	Restrict compatible searches by location.

You must provide at least one of searchTerms, startUrls, twitterHandles, or conversationIds. Multiple sources can be combined; their results share maxItems and are deduplicated.

X (Twitter) scraper output

Each dataset item represents one unique tweet. The Output tab provides a readable Tweets view and a compact Analysis view. The complete JSON record remains available through downloads and the dataset API.

Historical runs also write a COVERAGE key-value-store record. It reports each leaf date window, whether it completed, how many tweets and pages it produced, and whether the overall run stopped because of max-items, budget, or rate-limit.

{
  "type": "tweet",
  "id": "2036789109363532163",
  "url": "https://x.com/NASA/status/2036789109363532163",
  "twitterUrl": "https://twitter.com/NASA/status/2036789109363532163",
  "text": "A new view of our universe.",
  "fullText": "A new view of our universe.",
  "source": "Twitter Web App",
  "createdAt": "Fri Sep 18 16:12:00 +0000 2026",
  "lang": "en",
  "retweetCount": 125,
  "replyCount": 18,
  "likeCount": 842,
  "quoteCount": 12,
  "viewCount": 24890,
  "bookmarkCount": 31,
  "isReply": false,
  "isRetweet": false,
  "isQuote": false,
  "conversationId": "2036789109363532163",
  "author": {
    "type": "user",
    "id": "11348282",
    "userName": "NASA",
    "name": "NASA",
    "url": "https://x.com/NASA",
    "description": "There's space for everybody.",
    "followers": 85000000,
    "following": 180,
    "isVerified": true,
    "isBlueVerified": false,
    "profilePicture": "https://pbs.twimg.com/profile_images/example_normal.jpg"
  },
  "media": ["https://pbs.twimg.com/media/example.jpg"],
  "mediaDetails": [
    {
      "type": "photo",
      "url": "https://pbs.twimg.com/media/example.jpg",
      "videoUrl": null,
      "width": 2048,
      "height": 1365
    }
  ],
  "quote": null,
  "retweet": null
}

Important output fields include:

• Tweet identity: id, url, twitterUrl, conversationId
• Content: text, fullText, lang, createdAt, source
• Engagement: likeCount, retweetCount, replyCount, quoteCount, viewCount, bookmarkCount
• Relationships: isReply, inReplyToId, isQuote, quote, isRetweet, retweet
• Author: handle, name, bio, location, follower/following counts, verification, profile and cover images
• Media: compatibility URL arrays plus structured images, video variants, GIFs, dimensions, duration, bitrate, and thumbnails
• Rich data: entities, extended entities, cards, places, and X Articles

Tweet and user IDs are strings, not numbers, so JavaScript and spreadsheet tools do not lose precision.

Twitter scraper API

Use the synchronous endpoint when you want to start the Actor and receive dataset items in one response:

curl -X POST \
  "https://api.apify.com/v2/acts/smart_albatross~x-tweet-scraper/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchTerms":["from:NASA"],"maxItems":100}'

JavaScript with apify-client:

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('smart_albatross/x-tweet-scraper').call({
  searchTerms: ['from:NASA'],
  maxItems: 100,
  sort: 'Latest',
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python with apify-client:

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("smart_albatross/x-tweet-scraper").call(run_input={
    "searchTerms": ["from:NASA"],
    "maxItems": 100,
    "sort": "Latest",
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

Twitter scraping use cases

• Brand monitoring and mention tracking
• Market, public-opinion, and academic research
• Hashtag and trend discovery
• Competitor content and engagement analysis
• Influencer research using follower and performance metrics
• News monitoring and event tracking
• Content strategy and viral-post analysis
• AI, sentiment-analysis, and retrieval pipelines

Limits and responsible use

• Only publicly accessible tweets are in scope. Protected, deleted, suspended, withheld, or otherwise unavailable content cannot be returned.
• X decides which tweets appear in search and timeline responses. No scraper can guarantee a complete historical archive of content X does not expose.
• Narrow filters, inactive profiles, exact tweet URLs, and exhausted sources can return fewer rows than maxItems.
• Large date-range searches work best when both start and end are provided. The Actor partitions them by calendar month and can subdivide saturated windows without weakening the run-wide item or spending limits.
• Use scraped personal data only when you have a lawful purpose. Follow applicable privacy, data-protection, copyright, and platform rules.

FAQ

Do I need an X API key or Twitter login?

No. Users only provide the tweets, profiles, URLs, or searches they want to collect. The Actor manages access and proxy routing internally.

Does the Actor return duplicate tweets?

No duplicate tweet IDs are written within a run. Exact ID deduplication covers overlapping inputs, pages, endpoint unions, and Latest + Top searches.

Can I scrape one exact tweet?

Yes. Put its x.com or twitter.com URL in startUrls. The run can naturally return one item even if maxItems is higher.

Can I scrape replies or a complete thread?

Use the original tweet's numeric ID in conversationIds, or use an advanced query such as conversation_id:2036789109363532163 in searchTerms.

How do I get more tweets from a profile?

Use twitterHandles or profile URLs with maximizeResults: true. For large historical jobs, use search queries with explicit date ranges and split very broad periods into smaller ranges if X stops exposing older pages.

Why did I receive fewer tweets than maxItems?

maxItems is a ceiling, not a promise. The source may contain fewer matching public tweets, X may stop returning pages, a filter may exclude most results, or a run may reach the account pool's bounded rate-limit window. The Actor exits with the unique results already collected instead of waiting for hours.

Which export formats are supported?

Apify datasets can be downloaded as JSON, JSONL, CSV, Excel, XML, RSS, or HTML Table. You can also read results through the Dataset API or an Apify client library.

Local development

The project uses Node.js 22, TypeScript strict mode, ESM, Apify SDK, and pure HTTP requests.

npm install
npm test
npm run build

Offline tests use committed X response fixtures. The live smoke test skips automatically unless the required private environment is configured.