
X (Twitter) User Scraper
Pricing
$3.00/month + usage

X (Twitter) User Scraper
This project is a web scraper designed to extract user data and tweets from X (formerly known as Twitter) using Crawlee and Playwright.
0.0 (0)
Pricing
$3.00/month + usage
3
Total users
161
Monthly users
19
Runs succeeded
98%
Issues response
2.1 days
Last modified
4 months ago
Failed Scrapes
Open
Hi, we're running into an issue with failed scrapes. For example, "qb_fernando" does not get scraped. Can you please advise?
lumen limitless (lumen_limitless)
Hello! Taking a look at the account you provided as an example, it appears that user has protected their posts and only approved followers are able to view them. The scraper will not be able to get any results from users with protected posts.
Also after taking a look at the logs from your run, it appears some of your other failed scrapes are for accounts that do not exist. You may want to double check your input and ensure all of the usernames in your list are active accounts.
Let me know if you have any other questions!
excelsm
Thanks for looking into this. We really only need the follower count from that specific account. So even if the follower count is revealed on the protected account, that info can't be scraped?
lumen limitless (lumen_limitless)
Let me take a look at what information is available in that case. I may be able to change the logic to return partial data in situations such as this. I'll get back to you once I get a chance to look into this.
excelsm
Great, thank you. Much appreciated. Since we're on the topic of partial data, there's not a way to get the scraper to only scrape the followers count, right? It would take a lot less time for us to run the scraper if there was a way to target only that data.
lumen limitless (lumen_limitless)
It is technically possible, but it wouldn't save time. All of the data returned by the scraper is received in the same api response, so the difference between returning just a subset of the data and the entire dataset would be minimal.
excelsm
Got it. Have you had any time to look into the partial data return?