Twitter Scraper

Hi, the issue is currently investigated. Thanks!

range is over ten months, it gives only 5 tweets!

Hi earthy_myrtle,

To pull more results, you can split the search URL based on close time ranges. Please take a look at this section of the documentation. I'd advise setting the date ranges based on weeks, not months (and not 10 months, as in your case). https://apify.com/microworlds/twitter-scraper#limitation-and-solution. Please let me know if this helps.

Thanks! 😊

Thank you, I still get only 5 results. I removed date range and kept it open.

No, removing date ranges could significantly lower the number of results. Please provide me with the search URL so I can generate a suitable set of input URLs for you.

I search by terms and keywords not URL.

Okay, can you share a sample run id - I can generate the relevant URLs that can pull more results

Here it is: XbeyOdmtq1kAtNT40

Hi Caleb, Any updates on the issue why its giving only one-week data even though I have provided the date range from 2021 Jan?

@earthy_myrtle: Sorry for the mix-up. The response above is for you (I'll paste it below this): "To pull more results, you can split the search URL based on close time ranges. Please take a look at this section of the documentation. I'd advise setting the date ranges based on weeks, not months (and not 10 months, as in your case). https://apify.com/microworlds/twitter-scraper#limitation-and-solution."

@optimistic_elixir: You see to be on Apify free plan - in which the actor limits the possibility of extracting results beyond 5. Please consider upgrading your plan on Apify here - https://apify.com/pricing or contact support@apify.com. Hope that helps

Hi caleb I want to crawl hastags, hastag crawl didn't require url right? and one more thing I want data from "Jan 2021 to today" not from 10 months. so I dont think setting date range based on week will be feasible for us. could you please suggest alternate way?

Hi earthy_myrtle, I have generated the relevant search URLs for your query. So instead of the input below:

1{
2  "addUserInfo": false,
3  "maxRequestRetries": 6,
4  "maxTweets": 20000,
5  "maxTweetsPerQuery": 2000,
6  "scrapeTweetReplies": false,
7  "searchMode": "live",
8  "searchTerms": [
9    "AlinmaBankSA"
10  ],
11  "sinceDate": "2021-01-01",
12  "untilDate": "2024-07-22"
13}

Now you can add these URLs to the actor in the "URLs" section https://shots.screenshotpal.com/twitter_actor_urls.txt

That will extract all the tweets AlinmaBankSA until:2024-07-22 since:2021-01-01

So here's how to inject it (see image):

Hope this helps

Running this will take some some time, so I recommend setting the timeout to 0.

Regarding hashtag crawls, they work 100% like searches - you can use keywords or URLs. If you provide the hashtags, I can generate the queries for you as well.

Hope that helps 😊

Hi Caleb, thank you for the reply this are the hashtags i want to crawl could you please generate the queries for this, from the date range since : 2021-01-01 to 2024-07-31 (today): AlinmaBankSA, snbalahli, SAIBLIVE, RiyadBank, alrajhibank, anb_bank, alawwalsab, BankAlbilad

I am getting 5 results for every search while i put 50 results desired

@earthy_myrtle: Sure, I'm on it.

@nurturing_knight: You see to be on Apify free plan - in which the actor limits the possibility of extracting results beyond 5. Please consider upgrading your plan on Apify here - https://apify.com/pricing or contact support@apify.com. Hope that helps

Hi @earthy_myrtle, sorry for the delayed response. Here are the generated hashtags from 2021-01-01 to 2024-08-06 (today):

• https://shots.screenshotpal.com/hashtags/AlinmaBankSA.txt
• https://shots.screenshotpal.com/hashtags/snbalahli.txt
• https://shots.screenshotpal.com/hashtags/SAIBLIVE.txt
• https://shots.screenshotpal.com/hashtags/RiyadBank.txt
• https://shots.screenshotpal.com/hashtags/alrajhibank.txt
• https://shots.screenshotpal.com/hashtags/anb_bank.txt
• https://shots.screenshotpal.com/hashtags/alawwalsab.txt
• https://shots.screenshotpal.com/hashtags/BankAlbilad.txt

Cheers!

Hi Caleb: I'm also getting the problem that the actor only returns 5 tweets, I want to scrape twitter for tweets #USElection2024 and especially from July to August, and I use Make to run it but it doesn't seem to work