Twitter (X) Profiles and Tweets Scraper
2 hours trial then $15.00/month - No credit card required now
Twitter (X) Profiles and Tweets Scraper
2 hours trial then $15.00/month - No credit card required now
This actor allows you to scrape Twitter/X profiles, extract tweets, key metrics, metadata and gather insights such as tweet content, engagement metrics (likes, retweets, replies), and author details. It is handy for social media analysis, marketing research, and tracking public figures or brands.
Twitter Profiles and Tweets Scraper
Overview
This powerful scraper allows you to extract comprehensive data from Twitter (X) profiles and their tweets. It's designed to work with various input formats, including full profile URLs, usernames, and even Instagram profile URLs.
Features
- Scrape detailed profile information
- Extract tweets with comprehensive metadata
- Supports multiple input URLs, including Twitter handles and profile URLs.
- Username-only input support
- Configurable concurrency and retry mechanisms
- Optional inclusion of posts/tweets
- Proxy configuration for enhanced reliability
- Extracts tweets, user information, engagement metrics, tweet sources, and more.
Input Configuration
-
Start URLs:
- Full profile URLs:
https://x.com/elonmusk
orhttps://twitter.com/elonmusk
- Usernames:
adrian_horning_
- Instagram URLs (for cross-platform profiles):
https://www.instagram.com/robertsyfert/
- Full profile URLs:
-
Max Concurrency: Set maximum concurrent requests (default: 10)
-
Min Concurrency: Set minimum concurrent requests (default: 1)
-
Max Request Retries: Set maximum retry attempts for failed requests (default: 100)
Input Data
Here's an example of how to set up a Twitter profile scraping task:
1{ 2 "startUrls": [ 3 "https://x.com/elonmusk", 4 "elonmusk", 5 "https://www.instagram.com/robertsyfert/" 6 ], 7 "maxConcurrency": 10, 8 "minConcurrency": 1, 9 "maxRequestRetries": 100 10}
Output Structure
1{ 2 "type": "tweet", 3 "id": "1519480761749016577", 4 "url": "https://x.com/elonmusk/status/1519480761749016577", 5 "twitterUrl": "https://twitter.com/elonmusk/status/1519480761749016577", 6 "text": "Next I'm buying Coca-Cola to put the cocaine back in", 7 "retweetCount": 620127, 8 "replyCount": 179263, 9 "likeCount": 4434571, 10 "quoteCount": 166686, 11 "createdAt": "Thu Apr 28 00:56:58 +0000 2022", 12 "lang": "en", 13 "quoteId": null, 14 "bookmarkCount": 21208, 15 "isReply": false, 16 "card": {}, 17 "place": {}, 18 "source": "<a href=\"http://twitter.com/download/iphone\" rel=\"nofollow\">Twitter for iPhone</a>", 19 "author": { 20 "type": "user", 21 "userName": "elonmusk", 22 "url": "https://x.com/elonmusk", 23 "twitterUrl": "https://twitter.com/elonmusk", 24 "id": "44196397", 25 "name": "Elon Musk", 26 "isVerified": true, 27 "isBlueVerified": true, 28 "verifiedType": "none", 29 "hasNftAvatar": false, 30 "profilePicture": "https://pbs.twimg.com/profile_images/1845482317860450309/OrD0ovmf_normal.jpg", 31 "coverPicture": "https://pbs.twimg.com/profile_banners/44196397/1726163678", 32 "description": "Read @America to understand why I'm supporting Trump for President", 33 "location": "", 34 "followers": 201720551, 35 "following": 787, 36 "protected": false, 37 "status": "", 38 "canDm": false, 39 "canMediaTag": false, 40 "advertiserAccountType": "promotable_user", 41 "analyticsType": "enabled", 42 "createdAt": "Tue Jun 02 20:12:29 +0000 2009", 43 "entities": { 44 "description": { 45 "urls": [] 46 }, 47 "url": { 48 "urls": [ 49 { 50 "display_url": "TheAmericaPAC.org", 51 "expanded_url": "http://TheAmericaPAC.org", 52 "url": "https://t.co/DjyKIO6ePx", 53 "indices": [ 54 0, 55 23 56 ] 57 } 58 ] 59 } 60 }, 61 "fastFollowersCount": 0, 62 "favouritesCount": 81232, 63 "geoEnabled": false, 64 "hasCustomTimelines": true, 65 "hasExtendedProfile": false, 66 "isTranslator": false, 67 "mediaCount": 2592, 68 "profileBackgroundColor": "C0DEED", 69 "statusesCount": 54347, 70 "translatorTypeEnum": "None", 71 "withheldInCountries": [], 72 "affiliatesHighlightedLabel": { 73 "label": { 74 "url": { 75 "url": "https://twitter.com/X", 76 "urlType": "DeepLink" 77 }, 78 "badge": { 79 "url": "https://pbs.twimg.com/profile_images/1683899100922511378/5lY42eHs_bigger.jpg" 80 }, 81 "description": "X", 82 "userLabelType": "BusinessLabel", 83 "userLabelDisplayType": "Badge" 84 } 85 } 86 }, 87 "extendedEntities": {}, 88 "isRetweet": false, 89 "isQuote": false, 90 "media": [] 91}
Output Format
The scraper provides rich, structured data for both profiles and tweets. Here's a detailed breakdown of the output fields:
Tweet Object
Field | Type | Description |
---|---|---|
type | String | Always "tweet" for tweet objects |
id | String | Unique identifier of the tweet |
url | String | Full URL to the tweet on X.com |
twitterUrl | String | Full URL to the tweet on Twitter.com |
text | String | Content of the tweet |
retweetCount | Number | Count of retweets |
replyCount | Number | Count of replies |
likeCount | Number | Count of likes |
quoteCount | Number | Count of quote tweets |
createdAt | String | Timestamp of tweet creation |
lang | String | Language code of the tweet |
quoteId | String | null |
bookmarkCount | Number | Count of bookmarks |
isReply | Boolean | Indicates if the tweet is a reply |
card | Object | Additional card information (e.g., for links) |
place | Object | Location information, if available |
source | String | Application used to post the tweet |
isRetweet | Boolean | Indicates if the tweet is a retweet |
isQuote | Boolean | Indicates if the tweet is a quote tweet |
media | Array | Contains media objects (images, videos) |
extendedEntities | Object | Additional entities (e.g., extended media info) |
Author Object (User Profile)
Field | Type | Description |
---|---|---|
type | String | Always "user" for profile objects |
userName | String | Twitter handle of the user |
url | String | Profile URL on X.com |
twitterUrl | String | Profile URL on Twitter.com |
id | String | Unique identifier of the user |
name | String | Display name of the user |
isVerified | Boolean | Legacy verification status |
isBlueVerified | Boolean | Twitter Blue verification status |
verifiedType | String | Type of verification |
hasNftAvatar | Boolean | Indicates if user has an NFT avatar |
profilePicture | String | URL of the profile picture |
coverPicture | String | URL of the profile banner/cover |
description | String | User's bio text |
location | String | User's location (if provided) |
followers | Number | Count of followers |
following | Number | Count of accounts followed |
protected | Boolean | Indicates if the account is private |
status | String | Current status (if set) |
canDm | Boolean | Indicates if user can receive DMs |
canMediaTag | Boolean | Indicates if user can be tagged in media |
advertiserAccountType | String | Type of advertiser account |
analyticsType | String | Type of analytics enabled |
createdAt | String | Timestamp of account creation |
entities | Object | URLs and mentions in the bio |
fastFollowersCount | Number | Count of fast followers |
favouritesCount | Number | Count of liked tweets |
geoEnabled | Boolean | Indicates if geolocation is enabled |
hasCustomTimelines | Boolean | Indicates custom timeline usage |
hasExtendedProfile | Boolean | Indicates extended profile features |
isTranslator | Boolean | Indicates if user is a Twitter translator |
mediaCount | Number | Count of media posts |
profileBackgroundColor | String | Hex code of profile background color |
statusesCount | Number | Total count of tweets/posts |
translatorTypeEnum | String | Type of translator status |
withheldInCountries | Array | Countries where account is withheld |
affiliatesHighlightedLabel | Object | Information about affiliated labels |
Usage Tips
- For bulk scraping, use the "Bulk edit" feature to input multiple profile URLs or usernames.
- Adjust concurrency settings based on your network capabilities and Twitter's rate limits.
- Use proxy configuration for large-scale scraping to avoid IP blocks.
Explore More Scrapers
If you found this Apify Smartbuyglasses Scraper useful, be sure to check out our other powerful scrapers and actors at memo23's Apify profile. We offer a wide range of tools to enhance your web scraping and automation needs across various platforms and use cases.
Support
- For issues or feature requests, please use the Issues section of this actor.
- If you need customization or have questions, feel free to contact the author:
- Author's website: https://muhamed-didovic.github.io/
- Email: muhamed.didovic@gmail.com
Additional Services
- Request customization or whole dataset: muhamed.didovic@gmail.com
- If you need anything else scraped, or this actor customized, email: muhamed.didovic@gmail.com
- For API services of this scraper (no Apify fee, just usage fee for the API), contact: muhamed.didovic@gmail.com
- Email: muhamed.didovic@gmail.com
Actor Metrics
62 monthly users
-
1 star
97% runs succeeded
Created in Oct 2024
Modified 14 days ago