Twitter Profile Scraper
3 days trial then $10.00/month - No credit card required now
Twitter Profile Scraper
3 days trial then $10.00/month - No credit card required now
Scrape any Twitter profile of any user. Extract tweets, replies, favorites, retweets, conversation threads and many more without any limit. Gather user related information such as verification, location, profile image, friends, followers, following and much more! No limits on Twitter!
Does not work
Hey there!
It seems like the actor is working properly and your data can be seen in the Storage Tab.
Please let us know if you need any further assistance, Best
Its not working,
"There are no items in the dataset."
Hey again!
That is the Output Schema and it shows up empty only when you use "onlyUserInfo" as true
. You can see the dataset by following the steps;
- Click on Output
- Click on "All fields"
This way, you can see all the output without any output schema. I am attaching a file for the reference.
Can you please let me know how it goes? Best
Ok well, that is confusing, why you don't have the basic there is really strange.
I exported the full list but unfortunately it is no good. The full twitter url is not showing, so we have no way of matching up the data.
Hey,
This attribute is a new one. Therefore the output schema is focused on tweet retrieval. For more detailed information, please reach out to the Apify team.
The data you get is; raw data that we are getting directly from Twitter. Do you need the URL of the account you put in the Input Schema to match up the results? Or something different? Best
"Do you need the URL of the account you put in the Input Schema to match up the results?"
Yes exactly, you have the outputted data, but without the url you inputted its pretty useless if its not matched up :)
Many thanks
Scott
Hey again!
Just deployed a new version for you which includes a new attribute called accountUrl
. Please use the latest version for the best experience.
Can you please check this up and let us know how it goes? Best
Looks great thanks
one last thing, we have a list of 7000 urls we want to do, I am 99% sure not all are working or correct urls, I tested your actor with a twitter url that does not exist but it seems to just keep going so I had to abort it,
https://console.apify.com/actors/runs/zon8sQIPCGy7zskA3#output
can you look into quickly flagging these and not charging too much and move onto the next url ?
Hey!
Unfortunately, that is not possible without opening the link. However, we just deployed a new version to quickly identify the non-existing user without retrying. I hope that helps.
Best
Ok thanks, I ran the two urls and see you only show the valid one in the output.
Is it possible to list the bad one as well in the output ? and flag it as a non existent twitter profile
When you click on this one for example
https://twitter.com/fragrancexwdqwed
it says
This account doesn’t exist Try searching for another.
This would be a very good feature for people who add large amounts of twitter urls. Maybe flag it if the twitter profile has zero followers ? I will leave it with you to best flag the dead url.
Many thanks
Scott
One other thing I have noticed, its quite slow due to the amount of data you are getting, even with "Only get the User Information" selected on.
On the output spreadsheet there are 80 fields ! If you can pick which fields you want, this would be great.
Many thanks
Scott
Hey again!
Sorry for the late reply. We just added a new feature called addNotFoundUsersToOutput
. If you enable this option and onlyUserInfo
at the same time, it will put the not found user in the output. Regarding the slowness, it is not actually because of the amount of data but mostly related to how Twitter behaves with the Datacenter IPs. Since the data is raw, there are not many things that can be done at that point.
However, if you still want to reduce the amount of data you are getting, you can customMapFunction
where you can map the output data and retrieve the data you want in any way.
Please let us know if you need any further assistance, Best
Hi! I appreciate this tool and the way it's set up - unfortunately, I'm unable to get any results for when I try to pull tweets. I'm using a residential proxy, etc, but I'm not getting any results when I use any options other than just getting the user data. Has twitter blocked this scraper for getting individual tweets, or is there something I'm missing?
Here is my attempt to get individual posts: https://console.apify.com/view/runs/x2TqspiL7uShZBMDo
and here is the attempt to get just profile information (which also doesn't seem to have any information?)
https://console.apify.com/view/runs/2nWNhnxwgJghvY8EP
Thank you!
Actor Metrics
104 monthly users
-
15 stars
>99% runs succeeded
Created in Jul 2022
Modified 19 hours ago