X (Twitter) User Scraper avatar
X (Twitter) User Scraper

Pricing

$3.00/month + usage

Go to Store
X (Twitter) User Scraper

X (Twitter) User Scraper

Developed by

lumen limitless

lumen limitless

Maintained by Community

This project is a web scraper designed to extract user data and tweets from X (formerly known as Twitter) using Crawlee and Playwright.

0.0 (0)

Pricing

$3.00/month + usage

3

Total users

161

Monthly users

19

Runs succeeded

98%

Issues response

2.1 days

Last modified

4 months ago

EX

Failed Scrapes

Open

excelsm opened this issue
14 days ago

Hi, we're running into an issue with failed scrapes. For example, "qb_fernando" does not get scraped. Can you please advise?

lumen_limitless avatar

Hello! Taking a look at the account you provided as an example, it appears that user has protected their posts and only approved followers are able to view them. The scraper will not be able to get any results from users with protected posts.

Also after taking a look at the logs from your run, it appears some of your other failed scrapes are for accounts that do not exist. You may want to double check your input and ensure all of the usernames in your list are active accounts.

Let me know if you have any other questions!

EX

excelsm

14 days ago

Thanks for looking into this. We really only need the follower count from that specific account. So even if the follower count is revealed on the protected account, that info can't be scraped?

lumen_limitless avatar

Let me take a look at what information is available in that case. I may be able to change the logic to return partial data in situations such as this. I'll get back to you once I get a chance to look into this.

EX

excelsm

13 days ago

Great, thank you. Much appreciated. Since we're on the topic of partial data, there's not a way to get the scraper to only scrape the followers count, right? It would take a lot less time for us to run the scraper if there was a way to target only that data.

lumen_limitless avatar

It is technically possible, but it wouldn't save time. All of the data returned by the scraper is received in the same api response, so the difference between returning just a subset of the data and the entire dataset would be minimal.

EX

excelsm

4 days ago

Got it. Have you had any time to look into the partial data return?