🏯 Tweet Scraper V2 (Pay Per Result) - X / Twitter Scraper
Pay $0.40 for 1,000 tweets
🏯 Tweet Scraper V2 (Pay Per Result) - X / Twitter Scraper
Pay $0.40 for 1,000 tweets
⚡️ Lightning-fast search, URL, list, and profile scraping, with customizable filters. At $0.30 per 1000 tweets, and 30-80 tweets per second, it is ideal for researchers, entrepreneurs, and businesses! Get comprehensive insights from Twitter (X) now!
Inconsistent results when searching for old tweets
Closed
I am trying to get Tweets from earlier this year but when I try running the scraper it stopped with 150 odd posts and the oldest one is in August. I tried using X advanced search and can find tweets from June/July. But I notice that both when using the 'top' tab and 'latest' tab when using the X web client it seems to stop loading tweets after a while so I am not sure if this is really a problem with the scraper or X. However it would be nice to know what is going on as other scrapes on recent events seem to getting better more complete coverage. Thanks.
Hello,
In order to fetch with dates, you need to use Twitter queries. Please refer to the actor's readme/documentation to learn more about how to use them.
https://apify.com/apidojo/tweet-scraper
Cheers
goldsmif
Thanks for the link, I looked at this and tried using the search term strings instead of the forms start and end dates but it still isn't working. Here is what I tried and the runs I performed and the search strings I used, can you tell me where I am going wrong. Many thanks.
Run using since with a UTC time
"#SpotifyWrapped Since:2024-12-04_13:00:00_UTC"
https://console.apify.com/actors/runs/yHJskkGkP2KVJ5yds#input
Gets tweets by they are just the most recent not from since the date.
Run using since but just date
"#SpotifyWrapped since:2024-12-04"
Gets tweets by they are just the most recent not from since the date.
https://console.apify.com/actors/runs/5Z8RrD6HBAvLl1UvA#input
Run using since_time and converting date and time to unix epoch
"#SpotifyWrapped since_time:1733317200"
https://console.apify.com/actors/runs/II9pGpNhmdhA4TYeo#output
Gets nothing
Also your documentation says twitter search only returns 800 tweets each time, but one of the runs above using just a search terms got 6,000 (which was my generic limit).
Many thanks
Hello,
-
Actually you are getting the tweets since that time, but you only fetch 100 so you can only see the most recent ones. When you are using Twitter queries, the sorting shows you the most recent ones. You can also use
until:2024-12-04if you want to see the tweets after that time. It is also adviced not to use epochs or UTC times as you may fall into themonitoringcase. Our rata-limiter is very sensitive with this. https://apify.com/apidojo/tweet-scraper#important-note -
Using
since_timeis forbidden, please check the Logs. I am also attaching a screenshot. -
We actually don't have any custom logic besides using your Twitter queries on Twitter so it is always recommended for you to test your query on Twitter web UI and see what you get, the actor will do the same and try to fetch those results to you. We don't have any control on that nor any logic.
-
Twitter queries are very non deterministic, sometimes they return even more than that. If you fetch from profiles with
from:, it will fetch ~800 per query.
I hope these help.
Cheers
API DojoActor Metrics
1.9k monthly users
-
355 stars
93% runs succeeded
3 hours response time
Created in Nov 2023
Modified 6 hours ago
