Twitter Scraper avatar

Twitter Scraper

Deprecated
Go to Store
This Actor is deprecated

This Actor is unavailable because the developer has decided to deprecate it. Would you like to try a similar Actor instead?

See alternative Actors
Twitter Scraper

Twitter Scraper

microworlds/twitter-scraper

Twitter Scraper searches and extracts data from Twitter (with historic data from 2006) - parses and converts the data to structured formats: HTML table, JSON, CSV, Excel and XML.

EM

return few tweets than expected within data range

Open

earthy_myrtle opened this issue
5 months ago

I got only one-week data even though I have provided the date range from 2021 Jan

microworlds avatar

Hi, the issue is currently investigated. Thanks!

OE

optimistic_elixir

5 months ago

range is over ten months, it gives only 5 tweets!

microworlds avatar

Hi earthy_myrtle,

To pull more results, you can split the search URL based on close time ranges. Please take a look at this section of the documentation. I'd advise setting the date ranges based on weeks, not months (and not 10 months, as in your case). https://apify.com/microworlds/twitter-scraper#limitation-and-solution. Please let me know if this helps.

Thanks! 😊

OE

optimistic_elixir

5 months ago

Thank you, I still get only 5 results. I removed date range and kept it open.

microworlds avatar

No, removing date ranges could significantly lower the number of results. Please provide me with the search URL so I can generate a suitable set of input URLs for you.

OE

optimistic_elixir

5 months ago

I search by terms and keywords not URL.

microworlds avatar

Okay, can you share a sample run id - I can generate the relevant URLs that can pull more results

OE

optimistic_elixir

5 months ago

Here it is: XbeyOdmtq1kAtNT40

EM

earthy_myrtle

5 months ago

Hi Caleb, Any updates on the issue why its giving only one-week data even though I have provided the date range from 2021 Jan?

microworlds avatar

@earthy_myrtle: Sorry for the mix-up. The response above is for you (I'll paste it below this): "To pull more results, you can split the search URL based on close time ranges. Please take a look at this section of the documentation. I'd advise setting the date ranges based on weeks, not months (and not 10 months, as in your case). https://apify.com/microworlds/twitter-scraper#limitation-and-solution."

@optimistic_elixir: You see to be on Apify free plan - in which the actor limits the possibility of extracting results beyond 5. Please consider upgrading your plan on Apify here - https://apify.com/pricing or contact support@apify.com. Hope that helps

EM

earthy_myrtle

5 months ago

Hi caleb I want to crawl hastags, hastag crawl didn't require url right? and one more thing I want data from "Jan 2021 to today" not from 10 months. so I dont think setting date range based on week will be feasible for us. could you please suggest alternate way?

microworlds avatar

Hi earthy_myrtle, I have generated the relevant search URLs for your query. So instead of the input below:

1{
2  "addUserInfo": false,
3  "maxRequestRetries": 6,
4  "maxTweets": 20000,
5  "maxTweetsPerQuery": 2000,
6  "scrapeTweetReplies": false,
7  "searchMode": "live",
8  "searchTerms": [
9    "AlinmaBankSA"
10  ],
11  "sinceDate": "2021-01-01",
12  "untilDate": "2024-07-22"
13}

Now you can add these URLs to the actor in the "URLs" section https://shots.screenshotpal.com/twitter_actor_urls.txt

That will extract all the tweets AlinmaBankSA until:2024-07-22 since:2021-01-01

So here's how to inject it (see image):

Hope this helps

microworlds avatar

Running this will take some some time, so I recommend setting the timeout to 0.

Regarding hashtag crawls, they work 100% like searches - you can use keywords or URLs. If you provide the hashtags, I can generate the queries for you as well.

Hope that helps 😊

EM

earthy_myrtle

5 months ago

Hi Caleb, thank you for the reply this are the hashtags i want to crawl could you please generate the queries for this, from the date range since : 2021-01-01 to 2024-07-31 (today): AlinmaBankSA, snbalahli, SAIBLIVE, RiyadBank, alrajhibank, anb_bank, alawwalsab, BankAlbilad

NK

nurturing_knight

5 months ago

I am getting 5 results for every search while i put 50 results desired

microworlds avatar

@earthy_myrtle: Sure, I'm on it.

@nurturing_knight: You see to be on Apify free plan - in which the actor limits the possibility of extracting results beyond 5. Please consider upgrading your plan on Apify here - https://apify.com/pricing or contact support@apify.com. Hope that helps

PR

Providence

4 months ago

Hi Caleb: I'm also getting the problem that the actor only returns 5 tweets, I want to scrape twitter for tweets #USElection2024 and especially from July to August, and I use Make to run it but it doesn't seem to work

MP

mustard_poppy

4 months ago

I am getting the same issue. No matter which settings do I introduce, the number of returned tweets never exceeds 5.

Developer
Maintained by Community
Categories