Pay $0.30 for 1,000 tweets
🏯 Tweet Scraper V2 (Pay Per Result) - X / Twitter Scraper
Pay $0.30 for 1,000 tweets
⚡️ Lightning-fast search, URL, list, and profile scraping, with customizable filters. At $0.30 per 1000 tweets, and 30-80 tweets per second, it is ideal for researchers, entrepreneurs, and businesses! Get comprehensive insights from Twitter (X) now!
Scraper came back with 1500 identical rows.
Closed
Hey hey!
Can you please share your Run ID?
Cheers
https://console.apify.com/actors/runs/Cwl6Spb0H1xtVlTLH#output
Sincerely,
Viktor I. Chief Pitcherman Rated 5/5 on Upwork 2024 Clutch Award Winner +1 304 460 9725 viktori.co
[image: Mailtrack] https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality11& Sender notified by Mailtrack https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality11&
Hey again!
Thank you very much for your swift response. You retrieved the user timeline twice, that's why this happened. You passed the same username in both twitterHandles and startUrls, which you should use only one of them.
Cheers
Thanks just did that but it came back with just about 700 results and the profile I'm scraping has about 4k tweets. https://console.apify.com/actors/61RPP7dywgiy0JPD0/runs/PbksoCuXF0OzOcmwb#output
How do I get the rest of them?
Sincerely,
Viktor I. Chief Pitcherman Rated 5/5 on Upwork 2024 Clutch Award Winner +1 304 460 9725 viktori.co
[image: Mailtrack] https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality11& Sender notified by Mailtrack https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality11&
Hey hey!
Thanks for reaching out. This problem is happening because of Twitter itself. Twitter has a soft limitation no matter what account type you use. For each of the paginated lists, Twitter creates a stream. Every stream has its maximum limitation. The limitation is simply that Twitter does not give more results after an uncertain amount of pagination. No matter where you are checking that stream, or which account you use. For example, if you start pagination by using your account, after some time, the UI will not return more data. This limitation is not because of the account type but Twitter's global restrictions. This restriction also applies to all the products that you can find around the market.
In addition to this, each user timeline is a stream. Therefore, you cannot retrieve more than a specific amount of tweets if you proceed forward by the retrieval.
The best option option that you can use would be splitting your request into multiple chunks and trying to create multiple streams - by using Advanced Search. You can remove everything from your Input and include searchTerms only. In the Search Terms, you have to provide something like this: from:elonmusk since:2024-01-01 until:2024-04-04. In this way, you create a time window in your search functionality and retrieve the results accordingly. Then you can move this time window and retrieve more tweets. For reference you can check this page: https://github.com/igorbrigadir/twitter-advanced-search
Also, for the validation, you can paste the string you want to put on the Search Term into Twitter and see if it is working properly.
Cheers
API Dojo- 1k monthly users
- 95.9% runs succeeded
- 0.29 days response time
- Created in Nov 2023
- Modified about 7 hours ago
