Twitter Scraper avatar
Twitter Scraper

Deprecated

Pricing

$1.50 / 1,000 tweets

Go to Store
Twitter Scraper

Twitter Scraper

Deprecated

Developed by

Caleb David

Maintained by Community

Twitter Scraper searches and extracts data from Twitter (with historic data from 2006) - parses and converts the data to structured formats: HTML table, JSON, CSV, Excel and XML.

5.0 (1)

Pricing

$1.50 / 1,000 tweets

61

Monthly users

41

Runs succeeded

>99%

Last modified

7 months ago

EM

return few tweets than expected within data range

Open

earthy_myrtle opened this issue
9 months ago

I got only one-week data even though I have provided the date range from 2021 Jan

microworlds avatar

Hi, the issue is currently investigated. Thanks!

OE

optimistic_elixir

9 months ago

range is over ten months, it gives only 5 tweets!

microworlds avatar

Hi earthy_myrtle,

To pull more results, you can split the search URL based on close time ranges. Please take a look at this section of the documentation. I'd advise setting the date ranges based on weeks, not months (and not 10 months, as in your case). https://apify.com/microworlds/twitter-scraper#limitation-and-solution. Please let me know if this helps.

Thanks! 😊

OE

optimistic_elixir

9 months ago

Thank you, I still get only 5 results. I removed date range and kept it open.

microworlds avatar

No, removing date ranges could significantly lower the number of results. Please provide me with the search URL so I can generate a suitable set of input URLs for you.

OE

optimistic_elixir

9 months ago

I search by terms and keywords not URL.

microworlds avatar

Okay, can you share a sample run id - I can generate the relevant URLs that can pull more results

OE

optimistic_elixir

9 months ago

Here it is: XbeyOdmtq1kAtNT40

EM

earthy_myrtle

9 months ago

Hi Caleb, Any updates on the issue why its giving only one-week data even though I have provided the date range from 2021 Jan?

microworlds avatar

@earthy_myrtle: Sorry for the mix-up. The response above is for you (I'll paste it below this): "To pull more results, you can split the search URL based on close time ranges. Please take a look at this section of the documentation. I'd advise setting the date ranges based on weeks, not months (and not 10 months, as in your case). https://apify.com/microworlds/twitter-scraper#limitation-and-solution."

@optimistic_elixir: You see to be on Apify free plan - in which the actor limits the possibility of extracting results beyond 5. Please consider upgrading your plan on Apify here - https://apify.com/pricing or contact support@apify.com. Hope that helps

EM

earthy_myrtle

9 months ago

Hi caleb I want to crawl hastags, hastag crawl didn't require url right? and one more thing I want data from "Jan 2021 to today" not from 10 months. so I dont think setting date range based on week will be feasible for us. could you please suggest alternate way?

microworlds avatar

Hi earthy_myrtle, I have generated the relevant search URLs for your query. So instead of the input below:

1{
2  "addUserInfo": false,
3  "maxRequestRetries": 6,
4  "maxTweets": 20000,
5  "maxTweetsPerQuery": 2000,
6  "scrapeTweetReplies": false,
7  "searchMode": "live",
8  "searchTerms": [
9    "AlinmaBankSA"
10  ],
11  "sinceDate": "2021-01-01",
12  "untilDate": "2024-07-22"
13}

Now you can add these URLs to the actor in the "URLs" section https://shots.screenshotpal.com/twitter_actor_urls.txt

That will extract all the tweets AlinmaBankSA until:2024-07-22 since:2021-01-01

So here's how to inject it (see image):

Hope this helps

microworlds avatar

Running this will take some some time, so I recommend setting the timeout to 0.

Regarding hashtag crawls, they work 100% like searches - you can use keywords or URLs. If you provide the hashtags, I can generate the queries for you as well.

Hope that helps 😊

EM

earthy_myrtle

9 months ago

Hi Caleb, thank you for the reply this are the hashtags i want to crawl could you please generate the queries for this, from the date range since : 2021-01-01 to 2024-07-31 (today): AlinmaBankSA, snbalahli, SAIBLIVE, RiyadBank, alrajhibank, anb_bank, alawwalsab, BankAlbilad

NK

nurturing_knight

9 months ago

I am getting 5 results for every search while i put 50 results desired

microworlds avatar

@earthy_myrtle: Sure, I'm on it.

@nurturing_knight: You see to be on Apify free plan - in which the actor limits the possibility of extracting results beyond 5. Please consider upgrading your plan on Apify here - https://apify.com/pricing or contact support@apify.com. Hope that helps

PR

Providence

8 months ago

Hi Caleb: I'm also getting the problem that the actor only returns 5 tweets, I want to scrape twitter for tweets #USElection2024 and especially from July to August, and I use Make to run it but it doesn't seem to work

MP

mustard_poppy

8 months ago

I am getting the same issue. No matter which settings do I introduce, the number of returned tweets never exceeds 5.

Pricing

Pricing model

Pay per result 

This Actor is paid per result. You are not charged for the Apify platform usage, but only a fixed price for each dataset of 1,000 items in the Actor outputs.

Price per 1,000 items

$1.50