
🏯 Tweet Scraper V2 ($0.4 / 1K tweets) - X / Twitter Scraper
Pricing
from $0.40 / 1,000 tweets

🏯 Tweet Scraper V2 ($0.4 / 1K tweets) - X / Twitter Scraper
⚡️ Lightning-fast search, URL, list, and profile scraping, with customizable filters. At $0.40 per 1000 tweets, and 30-80 tweets per second, it is ideal for researchers, entrepreneurs, and businesses! Get comprehensive insights from Twitter (X) now!
2.3 (76)
Pricing
from $0.40 / 1,000 tweets
737
Total users
18K
Monthly users
2.3K
Runs succeeded
>99%
Issues response
5.2 hours
Last modified
20 hours ago
Tweets are missing/Inconsistent results.
Closed
While scraping the query as per the attached run, we expected to obtain 70 tweets (excluding advertisements), however only 66 were returned and there doesn't appear to be any logic for the four tweets which were skipped, i.e., some are simple tweets with photos, while one directly follows an advertisement in the twitter advanced search.
I appreciate your advice on this matter.
-Tyler
durian.corp
Might this be related to the very large time span which our query covers? I noted your response to another user concerning breaking the query dates into smaller segment.
durian.corp
I went ahead and did month-by-month and week-by-week searches as per the below screenshots, but these produced even fewer results, Whole time range: 66 of 70 Month-by-Month: 50 of 70 Week-by-week: 44 of 70
Example of actor inputs:
{'customMapFunction': '(object) => { return {...object} }', 'includeSearchTerms': True, 'maxItems': 23810, 'maxTweetsPerQuery': 23810, 'onlyImage': False, 'onlyQuote': False, 'onlyTwitterBlue': False, 'onlyVerifiedUsers': False, 'onlyVideo': False, 'searchTerms': ['(ชายแดนใต้ OR ไฟใต้) (ไทยพุทธ) since:2022-01-01 until:2022-01-07\t', '(ชายแดนใต้ OR ไฟใต้) (ไทยพุทธ) since:2022-01-08 until:2022-01-14\t', '(ชายแดนใต้ OR ไฟใต้) (ไทยพุทธ) since:2022-01-15 until:2022-01-21\t', '(ชายแดนใต้ OR ไฟใต้) (ไทยพุทธ) since:2022-01-22 until:2022-01-28\t', ...], 'sort': 'Latest'}

Hey hey,
Can you send us how you get 70 results from Twitter so we can investigate further?
Best
durian.corp
Sure: I typed the original query into X and counted as I knew prior to this that this topic could be used for testing. :) (Literally (ชายแดนใต้ OR ไฟใต้) (ไทยพุทธ) since:2022-01-15 until:2022-01-21)
I know it isn't exactly the most programatic solution, but it works for test sets.
durian.corp
I can also indicate the missing four posts if this is of any assistance?

Hey hey,
Yes that would be great. Can you also send me your direct search URL? When I search these, I cannot get any results.
Cheers
durian.corp
- The post with the photo of the military person: 2nd in the list of returned tweets 2 & 3 - Middle Section of the results
- Very far towards the end.
durian.corp
Also, I note that there is a difference between how the search terms are ordered and we also experimented with this, both versions below produce the same results (66).
Version 1: ไทยพุทธ (ชายแดนใต้ OR ไฟใต้) until:2024-07-24 since:2022-01-01 Version 2: (ชายแดนใต้ OR ไฟใต้) ไทยพุทธ until:2024-07-24 since:2022-01-01
Although the order of the results did vary.

Hello,
We are checking the missing tweet issue. For the other one, that is an expected behavior since Twitter searches by keyword and it doesn't check the order. You can try to put these keywords between quotes if you want to get the exact result like "ไทยพุทธ (ชายแดนใต้ OR ไฟใต้) "
. I believe that is the suggested way of doing a search like this. You can refer to https://github.com/igorbrigadir/twitter-advanced-search to get more information
Cheers
durian.corp
Thanks for prioritizing the missing tweets. I am not too concerned with the order of results, as that is of no consequence. Although I question whether wrapping the boolean operators in double quotes is recommended? That is used for exact phrase matches, isn't it? Wouldn't that effectively render the implicit 'AND' and 'OR' as keywords?

Hey there,
Just got a reply from our engineering team. They investigated the missing tweet however couldn't find the real cause here. One of the possible reasons is that these tweets could be marked as sensitive. Our scrapers returning you whatever they can get from Twitter so sometimes there can be some inconsistencies or missing data. We cannot promise anything related to the data we get from Twitter.
For the Twitter queries, I think that github link has the most information out there. We don't have any more information on top of that.
I hope these will help.
Cheers!
durian.corp
That's completely understandable and I appreciate you looking into it. :)