Twitter (X) Reply Scraper

Twitter (X) Reply Scraper is an Apify actor that allows you to extract detailed information about tweets and their replies from Twitter.com / X.com. This scraper can collect data on tweet & comment details, user information, and reply content, providing valuable insights for social media analysis, sentiment tracking, or trend monitoring.

Why use Twitter (X) Reply Scraper?

• Comprehensive Data Collection: Gather extensive information about tweets, including text, engagement metrics, and timestamps.
• Reply Insights: Extract detailed information about replies & comments, including user data and engagement metrics.
• Flexible Scraping Options: Choose to focus on specific tweets or users based on your research needs.
• Authentication Support: Utilize cookie-based authentication to access tweets that might be restricted to logged-in users.

How to use Twitter (X) Reply Scraper

1. Set up your Input:

   • Provide start URLs from X.com (e.g., specific search URLs).
   • Set the maximum number of requests per crawl.
   • Enter your X.com cookies for authentication.

2. Run the Actor: Start the actor and let it crawl the specified X pages.

3. Retrieve Results: Once the run is complete, download the scraped data in your preferred format (JSON, CSV, or Excel).

Input Configuration

• startUrls: Provide a list of URLs from Twitter.com (e.g., specific tweet URLs).
• maxRequestsPerCrawl: Set the maximum number of requests per crawl, default is 100.
• cookies: Enter your Twitter.com cookies for authentication.

Example Input

1{
2	"startUrls": [
3		{
4			"url": "https://x.com/search?q=%23journorequest+min_replies%3A10&src=typed_query&f=live"
5		}
6	],
7	"maxRequestsPerCrawl": 10000,
8	"cookies": "COOKIE_STRING"
9}

Obtaining Start URLs

You can copy any X.com search URL and use it as a start URL. You can select the Latest tab to get the latest tweets or the Top tab to get the most popular tweets.

There is also an advanced search option available at https://x.com/search-advanced where you can use various parameters to refine your search results more precisely.

You can set these parameters in the advanced search page:

• Words:

  • Use keywords or phrases to search for specific content.
  • Example: "breaking news" will find tweets containing the exact phrase "breaking news".
  • Use "OR" to search for tweets containing either of two terms (e.g., "cats OR dogs").
  • Use "-" before a word to exclude tweets containing that word (e.g., "-cats" excludes tweets with "cats").
  • Use "#" to search for specific hashtags (e.g., "#ThrowbackThursday").

• Accounts:

  • Use "from:" to find tweets from a specific account (e.g., "from:@X").
  • Use "to:" to find replies to a specific account (e.g., "to:@X").
  • Use "@" to find tweets mentioning specific accounts (e.g., "@SFBART OR @Caltrain").

• Filters:

  • Select "Replies" to include or exclude reply tweets.
  • Choose "Links" to include or focus on tweets with links.

• Engagement:

  • Use "min_replies:", "min_faves:", or "min_retweets:" followed by a number to set minimum engagement thresholds.

• Date Range:

  • Specify a date range to narrow down the search period.

• Language:

  • Select a specific language to filter tweets.

• Location:

  • Specify a location to find geo-tagged tweets from that area.

Obtaining Cookies

1. Install the Copy Cookies browser extension from the Chrome Web Store (link also works for other Chrome-based browsers such as Edge & Brave).
2. Go to X.com.
3. Log in to your account.
4. Click on the Copy Cookies extension icon to copy your cookies.
5. Paste the copied cookies into the cookies field in the input configuration.

Output Data

The actor will output the following data for each tweet and its replies:

• url: The URL of the tweet.
• username: The username of the user who posted the tweet.
• handle: The handle of the user who posted the tweet.
• tweetText: The text of the tweet.
• time: The timestamp of the tweet.
• views: The number of views the tweet has received.

Example Output

1{
2	"url": "https://x.com/davidbyers26/status/1844376177722708098",
3	"username": "David Byers",
4	"handle": "davidbyers26",
5	"tweetText": "Are you a landlord trying to sell a flat that has been caught up in the cladding scandal and valued at zero, rendering it unsellable? I'd be interested to hear from you at david.byers@thetimes.co.uk #journorequest",
6	"time": "2024-10-10T13:55:22.000Z",
7	"views": 8069,
8	"replies": 11,
9	"reposts": 58,
10	"likes": 55,
11	"bookmarks": 4,
12	"repliesData": [
13		{
14			"username": "EDITORIELLE",
15			"handle": "editorielle",
16			"text": "We've shared this for you David! Have a lovely day! x",
17			"time": "2024-10-10T14:23:01.000Z",
18			"replies": 0,
19			"reposts": 0,
20			"likes": 0,
21			"bookmarks": 0,
22			"views": 254
23		},
24		{
25			"username": "NLC - LEASEHOLD CAMPAIGNER - #reLEASE us",
26			"handle": "NLC_2019",
27			"text": "I will post in the NLC Facebook page.\nWe have over 30k members\nhttps://facebook.com/groups/nationalleaseholdcampaign/?ref=share_group_link…",
28			"time": "2024-10-10T15:39:33.000Z",
29			"replies": 0,
30			"reposts": 1,
31			"likes": 2,
32			"bookmarks": 1,
33			"views": 268
34		},
35		{
36			"username": "Property4Media",
37			"handle": "property4media",
38			"text": "Hello David, could you please confirm the publication and the deadline? We would love to share this with our property network, thank you!",
39			"time": "2024-10-10T14:04:16.000Z",
40			"replies": 0,
41			"reposts": 1,
42			"likes": 1,
43			"bookmarks": 0,
44			"views": 255
45		},
46		{
47			"username": "Olympic Park Homes Action Group",
48			"handle": "OlympicHomes_AG",
49			"text": "Must be thousands in this position!",
50			"time": "2024-10-10T21:57:31.000Z",
51			"replies": 0,
52			"reposts": 1,
53			"likes": 1,
54			"bookmarks": 0,
55			"views": 127
56		},
57		{
58			"username": "Eastreadingcommunity",
59			"handle": "Eastreadingcom",
60			"text": "you could move it into a ltd company  if its valued at zero, zero stamp duty  then let it out and move on?  if its already a btl then do the same,  zero capital gains tax and report the capital gain loss to HMRC Red book valuation and roll capital gain loss to future gains?",
61			"time": "2024-10-10T15:43:11.000Z",
62			"replies": 0,
63			"reposts": 0,
64			"likes": 0,
65			"bookmarks": 2,
66			"views": 234
67		},
68		{
69			"username": "Kate P",
70			"handle": "katesineed",
71			"text": "There's a simple fix in that case. Advertise properties (Eng./Wal flats on freehold) for LEASE, not for SALE. Have the lease contract terms defined in great detail, not the woolly nonsense as current. Ensure the lessee is can see all the faults in a building  with a surveyor pack",
72			"time": "2024-10-11T06:36:24.000Z",
73			"replies": 2,
74			"reposts": 0,
75			"likes": 4,
76			"bookmarks": 0,
77			"views": 119
78		},
79		{
80			"username": "Charles Newall",
81			"handle": "ChasNewall",
82			"text": "Just think of the CGT losses",
83			"time": "2024-10-11T12:38:12.000Z",
84			"replies": 0,
85			"reposts": 0,
86			"likes": 0,
87			"bookmarks": 0,
88			"views": 32
89		},
90		{
91			"username": "rj",
92			"handle": "richard_jm",
93			"text": "Reluctant landlord of 1 apartment here.",
94			"time": "2024-10-10T21:45:08.000Z",
95			"replies": 0,
96			"reposts": 0,
97			"likes": 0,
98			"bookmarks": 0,
99			"views": 116
100		},
101		{
102			"username": "Service charge Campaigner",
103			"handle": "bobbycarver88",
104			"text": "Yes, bought through Nationwide mortgage, unable to sell when buyer also used nationwide. Also shared owner",
105			"time": "2024-10-12T05:04:18.000Z",
106			"replies": 0,
107			"reposts": 0,
108			"likes": 0,
109			"bookmarks": 0,
110			"views": 14
111		},
112		{
113			"username": "Mel Bartley",
114			"handle": "melb4886",
115			"text": "We are not landlords, just older leaseholders, wondering if our heirs will have to pay IHT on a property that is unsellable because of (a) cladding & (b) service charges of £8000 p.a.",
116			"time": "2024-10-10T18:24:46.000Z",
117			"replies": 0,
118			"reposts": 1,
119			"likes": 3,
120			"bookmarks": 0,
121			"views": 162
122		},
123		{
124			"username": "Rainbowland",
125			"handle": "rainbowland1979",
126			"text": "Must be a few contenders @McrCladiators @HelpSkyline @EOCS_Official",
127			"time": "2024-10-10T15:24:42.000Z",
128			"replies": 0,
129			"reposts": 0,
130			"likes": 1,
131			"bookmarks": 0,
132			"views": 180
133		},
134		{
135			"username": "Jimmy Rushton",
136			"handle": "JimmySecUK",
137			"text": "Mass executions of surrendering Ukrainian soldiers is happening too frequently and too systematically for it to be anything other than an official or semi official policy - either sanctioned directly by Russian commanders or going unpunished.",
138			"time": "2024-10-13T15:33:01.000Z",
139			"replies": 44,
140			"reposts": 1110,
141			"likes": 3060,
142			"bookmarks": 81,
143			"views": 266361
144		},
145		{
146			"username": "Free Leaseholders",
147			"handle": "FreeLeasehlders",
148			"text": "Good news for leaseholders. These legislators behaved appallingly in trying to thwart the Leasehold and Freehold Reform Act 2024. If still in the Upper House, they would be blockers of the promised Leasehold and Commonhold Reform Bill.",
149			"time": "2024-10-13T16:37:04.000Z",
150			"replies": 2,
151			"reposts": 10,
152			"likes": 38,
153			"bookmarks": 1,
154			"views": 980
155		},
156		{
157			"username": "NLC - LEASEHOLD CAMPAIGNER - #reLEASE us",
158			"handle": "NLC_2019",
159			"text": "You need to cap ground rents for EXISITING leaseholders @mtpennycook \n\nThe last government were about to do this, do not let all this work go to waste. And DO NOT listen to those in vested interests !!!\n\nRemember \"People before party\" @Keir_Starmer",
160			"time": "2024-10-13T20:49:01.000Z",
161			"replies": 1,
162			"reposts": 10,
163			"likes": 22,
164			"bookmarks": 1,
165			"views": 844
166		}
167	]
168}