Twitter Trends Scraper

Twitter Trends Scraper

The Twitter Trends Scraper is a fast, reliable Twitter (X) trends scraping tool that collects trending topics and tweet volume snapshots from public GetDayTrends pages. It solves the pain of manual refreshing by delivering structured, timestamped data in real time or from historical snapshots. Built for marketers, developers, data analysts, and researchers, this twitter trending topics scraper and x trending topics scraper scales from quick checks to automated pipelines so you can monitor conversations, track momentum, and act on insights at speed.

What data / output can you get?

Below are the exact fields pushed to the Apify dataset during each run, with examples:

Data type	Description	Example value
time	UTC timestamp when the item was captured	2026-04-15T12:34:56.789Z
timePeriod	Time snapshot label for the URL being scraped (e.g., Live, Yesterday, N hours ago, Day 3)	Live
trend	The trending topic/hashtag label parsed from the page	#LaCasaDeLosFamososMx3
volume	Tweet volume text as displayed on the page	35.2K tweets

Notes:

• Results are saved live, row-by-row, to the Apify dataset and can be exported to CSV or JSON.
• In addition to the dataset, a run summary is saved to the key-value store as RUN_SUMMARY.json with run metadata (start time, proxy used, counts per timePeriod, and failed URLs).

Key features

• ⚡️ Bold reliability with retries & backoff
  Robust HTTP fetching with multiple retries and exponential backoff ensures consistent scraping, even when network hiccups occur.

• 🛡️ Smart proxy handling with automatic fallback
  Uses your configured proxy and automatically falls back to Apify RESIDENTIAL when the selected proxy is rejected, improving resilience in production.

• 🌍 Country-specific or worldwide coverage
  Select a country (e.g., US, GB, IN, JP) to scrape country-specific trends or leave empty for worldwide trends—ideal for a twitter trends monitoring tool and twitter trend tracker workflows.

• 🕒 Real-time and historical snapshots
  Scrape Live, N hours ago (1–23), Yesterday, Week Ago, Month Ago, Year Ago, or Day 3 to build time-series trend analyses with this real time twitter trends scraper.

• 🔗 Custom URL inputs with date templating
  Provide your own startUrls and use built-in placeholders like {today}, {yesterday}, and {day3} to target any GetDayTrends URL structure.

• 📤 Flexible data export
  Download dataset results in JSON or CSV directly from Apify—perfect for dashboards, reporting, and analytics pipelines.

• 👩‍💻 Developer-friendly and automation-ready
  Works great with the Apify API and fits into twitter trends scraper Python workflows, CI scripts, and orchestration tools. Live item pushing supports streaming use cases.

• 🧠 Production-focused parsing
  Multiple XPath strategies parse unstable layouts to extract trend names and volume reliably—ideal for a twitter trends data scraper, twitter trends extractor, or twitter trends crawler.

How to use Twitter Trends Scraper - step by step

1. Sign in to Apify
   Create or log in to your Apify account to run the Twitter Trends Scraper.

2. Open the actor and choose coverage
   Decide whether to scrape worldwide trends or select a specific country code (e.g., US for United States).

3. Select time snapshots
   Toggle the desired flags like live, hour1–hour23 (e.g., hour1 for “1 hour ago”), yesterday, weekAgo, monthAgo, yearAgo, or day3. You can combine multiple to capture several snapshots in one run.

4. (Optional) Add custom URLs
   Use startUrls to target any GetDayTrends page. Templating tokens {today}, {yesterday}, and {day3} are supported for dynamic dates.

5. Configure proxy (optional)
   Use Apify Proxy or your own proxy settings. If a selected proxy is rejected, the actor automatically falls back to Apify RESIDENTIAL.

6. Start the run
   Click Start. The scraper will fetch each URL, parse trends, and push items live into the dataset with fields time, timePeriod, trend, and volume.

7. Download your data
   Export your dataset to JSON or CSV. A RUN_SUMMARY.json file with stats and proxy info is also saved in the key-value store.

Pro tip: Schedule recurring runs and connect the dataset to your data warehouse or BI tool to scrape Twitter trending topics continuously and power a twitter trend mining script.

Use cases

Use case name	Description
Marketing campaign planning	Identify trending hashtags and topics quickly to align content and messaging with what’s gaining traction right now.
Social listening & brand monitoring	Monitor momentum around brand terms or industry themes using a repeatable x trending topics scraper workflow.
News & editorial planning	Track what’s breaking by country and time snapshot to inform coverage and headlines.
Competitive & influencer monitoring	Observe which conversations competitors amplify and measure shifts over time with a twitter trend tracker.
Market & cultural research	Analyze how events trend across geographies and hours/days to study audience engagement patterns.
Academic studies	Collect structured snapshots for temporal analysis, cultural studies, and media research.
Data engineering pipeline	Feed a twitter trends extractor into ETL/ELT workflows, APIs, or dashboards for automated reporting.

Why choose Twitter Trends Scraper?

Built for precision, automation, and reliability, this twitter trends scraping tool is optimized for production use.

• ✅ Accurate parsing across layout variations with multiple selectors
• 🌐 Worldwide and country-specific snapshots for flexible coverage
• 📈 Real-time and historical flags enable time-series trend analysis
• 🔌 Developer-ready: works with the Apify API and python-based pipelines
• 🧩 Integrates into automation stacks and streaming pipelines easily
• 🛡️ Safer operations: respects public web pages and avoids authenticated data
• 💾 Easy export from the Apify dataset to CSV/JSON for downstream use

Compared to unstable browser add-ons or ad-hoc scripts, this production-grade twitter trends crawler delivers consistent output, proxy resilience, and structured data you can trust.

Is it legal / ethical to use Twitter Trends Scraper?

Yes—when used responsibly. This actor scrapes publicly available pages on GetDayTrends, which aggregates Twitter/X trending data. It does not log into Twitter/X or access private information.

Guidelines for compliant use:

• Only collect publicly available content.
• Avoid scraping private or authenticated pages and any personal data.
• Follow the terms of service for the websites you access.
• Ensure compliance with data protection regulations (e.g., GDPR, CCPA).
• Consult your legal team for edge cases or jurisdiction-specific requirements.

Input parameters & output format

Example JSON input

{
  "live": true,
  "hour1": true,
  "yesterday": true,
  "weekAgo": false,
  "monthAgo": false,
  "yearAgo": false,
  "day2": false,
  "day3": false,
  "country": "US",
  "proxy": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"],
    "apifyProxyCountry": "US"
  },
  "startUrls": [
    { "url": "https://getdaytrends.com/{today}/24/" },
    { "url": "https://getdaytrends.com/us/3/" }
  ],
  "requestTimeoutSecs": 30,
  "maxRetries": 3
}

Parameters (from input schema)

Field	Type	Description	Default	Required
live	boolean	Scrape the current live trends.	Not set (interprets as false if omitted)	No
hour1	boolean	Scrape trends from 1 hour ago.	Not set (interprets as false if omitted)	No
hour2	boolean	Scrape trends from 2 hours ago.	Not set (interprets as false if omitted)	No
hour3	boolean	Scrape trends from 3 hours ago.	Not set (interprets as false if omitted)	No
hour4	boolean	Scrape trends from 4 hours ago.	Not set (interprets as false if omitted)	No
hour5	boolean	Scrape trends from 5 hours ago.	Not set (interprets as false if omitted)	No
hour6	boolean	Scrape trends from 6 hours ago.	Not set (interprets as false if omitted)	No
hour7	boolean	Scrape trends from 7 hours ago.	Not set (interprets as false if omitted)	No
hour8	boolean	Scrape trends from 8 hours ago.	Not set (interprets as false if omitted)	No
hour9	boolean	Scrape trends from 9 hours ago.	Not set (interprets as false if omitted)	No
hour10	boolean	Scrape trends from 10 hours ago.	Not set (interprets as false if omitted)	No
hour11	boolean	Scrape trends from 11 hours ago.	Not set (interprets as false if omitted)	No
hour12	boolean	Scrape trends from 12 hours ago.	Not set (interprets as false if omitted)	No
hour13	boolean	Scrape trends from 13 hours ago.	Not set (interprets as false if omitted)	No
hour14	boolean	Scrape trends from 14 hours ago.	Not set (interprets as false if omitted)	No
hour15	boolean	Scrape trends from 15 hours ago.	Not set (interprets as false if omitted)	No
hour16	boolean	Scrape trends from 16 hours ago.	Not set (interprets as false if omitted)	No
hour17	boolean	Scrape trends from 17 hours ago.	Not set (interprets as false if omitted)	No
hour18	boolean	Scrape trends from 18 hours ago.	Not set (interprets as false if omitted)	No
hour19	boolean	Scrape trends from 19 hours ago.	Not set (interprets as false if omitted)	No
hour20	boolean	Scrape trends from 20 hours ago.	Not set (interprets as false if omitted)	No
hour21	boolean	Scrape trends from 21 hours ago.	Not set (interprets as false if omitted)	No
hour22	boolean	Scrape trends from 22 hours ago.	Not set (interprets as false if omitted)	No
hour23	boolean	Scrape trends from 23 hours ago.	Not set (interprets as false if omitted)	No
yesterday	boolean	Scrape yesterday's full-day summary.	Not set (interprets as false if omitted)	No
weekAgo	boolean	Scrape trends from 1 week ago.	Not set (interprets as false if omitted)	No
monthAgo	boolean	Scrape trends from 1 month ago.	Not set (interprets as false if omitted)	No
yearAgo	boolean	Scrape trends from 1 year ago.	Not set (interprets as false if omitted)	No
day2	boolean	Scrape yesterday's full-day summary (legacy option, same as 'yesterday').	Not set (interprets as false if omitted)	No
day3	boolean	Scrape the summary from 2 days ago.	Not set (interprets as false if omitted)	No
country	string	Select a country to scrape country-specific Twitter trends. Leave empty for worldwide trends.	"" (empty = worldwide)	No
proxy	object	Proxy configuration. Defaults to no proxy. If the selected proxy is rejected, the actor falls back to Apify RESIDENTIAL automatically.	None	No

Additional inputs supported by the actor (from source code)

• startUrls: array of objects with { "url": "…" } to scrape custom pages; supports placeholders {today}, {yesterday}, {day3}.
• requestTimeoutSecs: integer timeout in seconds (default 30).
• maxRetries: integer number of retries (default 3).

Example dataset output

[
  {
    "time": "2026-04-15T12:34:56.789Z",
    "timePeriod": "Live",
    "trend": "#Navratri2025",
    "volume": "23.8K tweets"
  },
  {
    "time": "2026-04-15T12:34:56.789Z",
    "timePeriod": "1 hour ago",
    "trend": "Giants",
    "volume": "100.8K tweets"
  },
  {
    "time": "2026-04-15T12:34:56.789Z",
    "timePeriod": "Yesterday",
    "trend": "Erika",
    "volume": "576.2K tweets"
  }
]

Note: A RUN_SUMMARY.json is also saved to the key-value store with keys runStartedAt, proxyUsed, and stats (total_items, per_timePeriod counts, failed_urls).

FAQ

Does the Twitter Trends Scraper pull data in real time?

Yes. Enable the live flag to scrape the current live snapshot. You can also combine live with hour-based flags (e.g., hour1) and daily snapshots like yesterday for historical comparisons.

Can I scrape by country or worldwide?

Yes. Set the country field to a supported code (e.g., US, GB, IN) for country-specific pages, or leave it empty to scrape worldwide trends.

Does this scrape Twitter/X directly?

No. It scrapes public pages on GetDayTrends, which aggregates Twitter/X trends. This approach avoids login and focuses on publicly visible trend information.

What fields are included in the output?

Each item contains time, timePeriod, trend, and volume. The volume is the on-page text (e.g., “Under 10K tweets” or “78.7K tweets”), providing a consistent, human-readable metric.

How does proxy configuration work?

You can use Apify Proxy or your own proxy. If a selected proxy is rejected, the actor automatically falls back to Apify RESIDENTIAL. This improves reliability for larger or repeated runs.

Can I add my own URLs or schedule runs?

Yes. Provide startUrls with optional placeholders {today}, {yesterday}, and {day3}. You can schedule runs in Apify to build a continuous twitter trends scraper python pipeline or scrape x trends regularly.

Is there a free trial?

Yes. The actor includes trial minutes, so you can test it before subscribing. Check the Apify listing for current trial availability and details.

How many results will I get per run?

The number of items depends on what appears on the target GetDayTrends pages for each selected time snapshot. The actor pushes every parsed row into the dataset and reports counts in RUN_SUMMARY.json.

Closing CTA / Final thoughts

The Twitter Trends Scraper is built for structured, reliable extraction of trending topics and volumes from public GetDayTrends pages. With resilient proxy handling, real-time and historical flags, and clean exports, it empowers marketers, analysts, researchers, and developers to monitor trends at scale. Connect it to the Apify API or your python automation to create a repeatable twitter trends monitoring tool or twitter explore trends scraper pipeline. Start extracting smarter trend insights today and keep your team ahead of the conversation.