🏯 Tweet Scraper V2 (Pay Per Result) - X / Twitter Scraper
Pay $0.40 for 1,000 tweets
🏯 Tweet Scraper V2 (Pay Per Result) - X / Twitter Scraper
Pay $0.40 for 1,000 tweets
⚡️ Lightning-fast search, URL, list, and profile scraping, with customizable filters. At $0.30 per 1000 tweets, and 30-80 tweets per second, it is ideal for researchers, entrepreneurs, and businesses! Get comprehensive insights from Twitter (X) now!
Issue with Date Consistency in Results
Closed
Hello,
I'm experiencing inconsistencies with date filtering in the data results. Here are three test cases for reference:
Run ID: Ez6NJlluTMqz7aUTy – Using searchTerms to fetch tweets from x.com/mlb for dates Nov 11-15, 2024, I received 90 results. However, this seemed incomplete as 90 is not the number of tweets I am expecting from the handle in the given date range.
Run ID: lViFXBWxBeutYH1uR – Using start, end, and twitterHandles for dates Nov 11-16, I received 1,645 results, including duplicates and tweets outside the specified range.
Run ID: 46xWKUPH1qmpVzRzK – Using start, end, and twitterHandles for dates Oct 10-16, I got 848 results, with no duplicates but some tweets outside the range.
Filtering manually for tweets between Nov 11-15 from 2nd and 3rd run yields around 152 unique tweets, closer to my expectations.
Could you advise on the best way to ensure complete, accurate results within a specified date range, without duplicates or out-of-range tweets? I need consistent, reliable data and am open to using either searchTerm or start/end queries, as long as all tweets within the date range are captured.
Please review this and let me know if I’m missing anything in my query setup. Thanks.
Hello,
twitterHandles and start/end dates are not working together. Also if you are using Twitter queries, you shouldn't use any filters and do everything in the query. Have you checked our documentation? There is a very specific example that shows how you can fetch a profile's tweets with dates > https://apify.com/apidojo/tweet-scraper#fetching-the-tweets-of-a-profile
If you see less results than you expected, that is probably what twitter returns with that query and we cannot do anything to that. You can always test your query via Twitter web UI search and see check the results there.
Let me know if you have more questions.
Cheers
datapowa1
Hi,
If you look at the attached screenshots "mlb profile.png" shows the latest tweets, which is a retweet from "Seattle Mariners".
And in "mlb query.png" i tried the query to get a days data from:MLB since:2024-11-18 until:2024-11-19 As you can see it doesnt include retweets and many other tweets.
I understand its a twitter thing. But our use case includes getting tweets from the last day or last 3 days or using a start date and end date. We know only 700-1000 tweets are scrapable from an account. We can setup daily crawling for that. But we cant afford to miss any tweets, let it be quoted or retweeted or even a news article.
Is there any chance of implementing a date option ? it doesnt need to be a date range, but atleast like an option to say last n days or a last date. For example if i provide the date as "2024-11-15" and handle is "MLB" try to collect all the tweets back till the given date and return whatever tweets collected before hittign the limit.
Would appreciate if you could look into this. Thanks.
Hello,
Unfortunately that is not possible, you can only use Twitter queries with the actor.
Have you checked https://github.com/igorbrigadir/twitter-advanced-search which is linked from the documentation? There is a special part that explains how retweets work, some of them are excluded by default. Also as you mentioned we cannot do anything about what Twitter returns with your query.
Another thing is, what is your usecase? Are you aware that monitoring activities are not allowed as mentioned https://apify.com/apidojo/tweet-scraper#important-note? Seems like you are trying to monitor an account's tweets. Can you please let us know your use case in details? If it is private, you can reach out via Discord or email.
Thanks
datapowa1
Thank you for your reply.
i think something like this might work
(from:MLB) since:2023-01-01 include:nativeretweets
I am yet to test this. It does work in the Twitter frontend.
And thanks for the heads up on monitoring. I have read the documentation and we wont be monitoring any accounts for real time data as mentioned here:
". In other words: you should not try to initiate the same run in a very short period of time with time filters in order to understand if a profile tweeted or not. "
We will follow everything as mentioned in the documentation.
Thanks.
API DojoActor Metrics
1.7k monthly users
-
306 stars
99% runs succeeded
4.3 hours response time
Created in Nov 2023
Modified 17 hours ago
