3 days trial then $29.00/month - No credit card required now
Twitter followers scraper
3 days trial then $29.00/month - No credit card required now
Scrape followers, following and subscriptions list of any twitter profile
Need advice.
Closed
If I have to scrape 20M followers each of 20 handles (each handle having 20M followers) then which plan to take? How many can be scraped per week and per month? Can i run 20 concurrent threads of the actor in the same plan? Please guide so that it takes minimum possible time to scrape.
You can't scrape too fast or simultaneously as this scraper uses cookie of your twitter account and twitter can block you for that. But you can run multiple instances with different accounts cookies and proxies to reduce time required to scrape. Other questions i don't have exact numbers or answers. You have to find out
extraordinary_muffin
Thanks!
extraordinary_muffin
-
Suppose X blocks the account/cookie session in 1-2 weeks. @100K scrapes per day, there will be 1M scrapes before it blocks. Is 1M safe "minimum" figure to consider for planning scraping schedule, by your practical experience?
-
After blocking and getting a store id with 1M scrapes from above run, when re-run is initiated with another account/cookie parameters, will the newly incorporated feature of inputting store id will begin with 0 or the 1st scrape after the 1Mth scrape i.e. the new csv will have count of 1 or 1000001?
-
If the new csv count is 1000001, will there be repeats i.e. will 1000001 onwards scraped data will be same as previously (0-1000000) scraped data or the system will auto check for duplicate scraping or the system will auto check for duplicate before generating new csv?
Please guide on above as there will be new cookies post blocking and i don't want duplicate data.
Curious Coder- 105 monthly users
- 9 stars
- 84.1% runs succeeded
- 0.51 hours response time
- Created in Jun 2023
- Modified 5 days ago
