Twitter (X) Comment Scraper
1 day trial then $14.99/month - No credit card required now
Twitter (X) Comment Scraper
1 day trial then $14.99/month - No credit card required now
This Apify actor is designed to scrape comments from a given tweet. It collects information such as username, tweet content, timestamp, URL, image URL, quote tweet URL, number of replies, number of retweets, number of likes, and number of bookmarks.
Twitter (X) Comment Scraper
This Apify actor is designed to scrape comments from a specific tweet.
Features
- Scrapes comments from a specific tweet.
- Collects information such as username, tweet content, timestamp, URL, image URL, quote tweet URL, reply count, retweet count, like count, and bookmark count from the comments.
Obtaining Cookie Information
You can use the Cookie Editor extension to obtain Twitter cookie information. Follow these steps:
- Add the Cookie Editor extension to your Chrome browser.
- Log in to Twitter.
- Ensure your language settings are set to English from here.
- Click on the Cookie Editor icon in the top right corner of your browser.
- Click the "Export" button to export your cookie information in JSON format.
- Use this JSON string as the cookieJson input.
Note: If you encounter errors or if 0 comments are processed, ensure that your account is functioning normally and log in again to obtain new cookies to use as input.
Usage
- Run this actor in the Apify console.
- Provide the desired inputs:
- tweetUrl: The URL of the tweet you want to scrape comments from.
- cookieJson: A JSON string containing your Twitter account's cookie information.
Output
The scraped comments are saved to the Apify dataset. The output data includes:
username: Usernametweet: Tweet contenttimestamp: Timestampurl: Tweet URLimage_url: Image URLquote_url: Quote tweet URLreply_count: Reply countretweet_count: Retweet countlike_count: Like countbookmark_count: Bookmark count
Example Output
1{ 2 "username": "user123", 3 "tweet": "This is an example tweet content.", 4 "timestamp": "2024-02-07T14:30:00", 5 "url": "https://x.com/user123/status/1234567890", 6 "image_url": "https://x.com/image.jpg", 7 "quote_url": "https://x.com/user456/status/0987654321", 8 "reply_count": 10, 9 "retweet_count": 5, 10 "like_count": 20, 11 "bookmark_count": 2 12}
This example output shows the structure of a single comment. The actual output will be a list of similar objects for all scraped comments.
Notes
- This actor uses the Selenium framework for web scraping.
- It is designed to scrape comments visible on the tweet screen. Therefore, it cannot process nested comments and can retrieve a maximum of 200 comments due to limitations.
- The actor runs slowly to avoid X blocking due to request rates, which may result in higher than expected usage costs.
- It cannot process URLs of community posts.
- The collected data is stored in Apify’s default data store.
Muhammet Akkurt- 11 monthly users
- 1 star
- 42.9% runs succeeded
- Created in Sep 2024
- Modified 9 days ago
