
Best Twitter Scraper
Twitter scraper that scrapes the latest Twitter content of any user. Since you need to be logged in to Twitter to crawl, you need to provide a Twitter account. Twitter 抓取器,可以抓取任意用户最新的Twitter内容。 因为需要先登录 Twitter 才能抓取,所以需要提供一个 Twitter 的账号。
The Email for logged in accounts
user name
The user name for logged in accounts
password
The password for logged in accounts
Specific Twitter Profiles
Provide a list of Twitter profiles to scrape. The profiles should be written as 'username'.Scrape from https://twitter.com/{username}
Specific Tweet URLs
List the twitter search URLs you want to scape such as 'https://twitter.com/elonmusk/status/1656898688739057665' if your url contains more such as a cursor like 'https://twitter.com/elonmusk/status/1656898688739057665?s=20' only the tweets on the current screen will be loaded.
Twitter Search URLs
List the full links to tweets you want to scape such as 'https://twitter.com/search?q=((MadMaxFuryRoad%20OR%20MadMax%20OR%20%22FuryRoad%22%20OR%20%22GeorgeMiller%22%20OR%20GeorgeMiller%20OR%20TomHardy%20OR%20%22CharlizeTheron%22%20OR%20CharlizeTheron)%20AND%20(Film%20OR%20Movie%20OR%20Review))&src=typed_query&f=top' if your url contains more such as a cursor only the tweets on the current screen will be loaded.
Maximum number of tweets (per profile)
Set the maximum number of tweets get from per request. Default:10
Default value of this property is
Max PageDown Event
Set the maximum number of a keyboard 'PageDown' event sent to the page. Default:100.
Default value of this property is
Concurrent Requests
Configure maximum concurrent requests performed by Scrapy (default: 2)
Default value of this property is
Locale
Specify user locale, for example en-GB
, de-DE
, etc. Locale will affect navigator.language
value, Accept-Language
request header value as well as number and date formatting rules. Default:en-US.
Default value of this property is
Timezone
Changes the timezone of the context. See ICU's metaZones.txt for a list of supported timezone IDs.
timeout
Default value of this property is
- 213 users
- 543 runs
- Modified 3 months ago