Twitter Scraper

If you add search terms, the scraper will find and extract tweets that mention those terms. If you want to search for hashtags, just use the # at the beginning of your search term, so instead of webscraping search for #webscraping. Alternatively, scroll further down to scrape by Twitter profile or URL.

This setting will change the way the scraper sorts Twitter data before extracting it: by latest or top tweets, people, photos or videos.

Limit the number of profiles to scrape. This is useful if you want to scrape a lot of tweets, but only from a few profiles.

This value lets you set the maximum number of tweets to retrieve per search query

This allows you to get the number of times a tweet has been viewed

Extends the tweets with user information. You can decrease the size of your dataset by turning this off.

Use Cheerio instead of Puppeteer to scrape the page. This has a very high chance to scrape all tweets and it's faster for multiple scrapes, but slower for single scrapes. It's only enabled if (content type is not `People`, and not logged in)

You can add the twitter handles of specific profiles you want to scrape. This is a shortcut so that you don't have to add a full username URLs such as https://twitter.com/username

You can either limit scraping to tweets or also include replies. Note that this only applies when scraping Twitter profiles.

This lets you tell the scraper where to start. You can enter Twitter URLs one by one. You can also link to or upload a text file with a list of URLs.

Scrape tweets newer than this date, format YYYY-MM-DD. You can use this in conjunction with 'Tweets older than' to create a limited time slice.

Scrape tweets from this date and before, format YYYY-MM-DD. You can use this in conjunction with 'Tweets newer than' to create a limited time slice.

Use advanced search instead of the default search. This is useful if you want to scrape tweets using a searchTerm, user handle, or using a date range. Note that enabling this option doesn't scrape `retweets`.

This is required if you want to use Apify Proxy.

Add or remove properties on the output object or omit the output returning null

Advanced function that allows you to extend the default scraper functionality, allowing you to manually perform actions on the page

Any data that you want to have available inside the Extend Output/Scraper Function

Max timeout for the handlePageFunction. Can be increased for long running processes

Set the max request retries

Configures how many seconds of no data received will be considered done

Enable debug log

Your login cookies will be used to bypass the login wall. Check the README for detailed instructions.