Twitter Trends Scraper

Twitter Trends Scraper

Twitter Trends Scraper is a production-ready scraper that collects trending topics and tweet volume snapshots from getdaytrends.com and streams them directly into your Apify dataset. It automates live and historical trend captures with robust retries and smart proxy fallback, eliminating manual checks and flaky workflows. Built for marketers, developers, data analysts, and researchers, it enables reliable, scalable monitoring of worldwide or country-specific trends with clean, structured outputs.

What data / output can you get?

This actor pushes one dataset item per trend with a consistent schema optimized for analysis and time‑series tracking.

Data type	Description	Example value
time	UTC timestamp when the trend snapshot was parsed	2026-04-07T11:30:12.345678Z
timePeriod	Label for the snapshot window (e.g., Live, hour-based, or historical)	Live; 1 hour ago; Yesterday; Week Ago; Month Ago; Year Ago; Day 3
trend	Trend text (topic or hashtag) as listed on getdaytrends.com	#Navratri2025; Russell Wilson; Chiefs
volume	Tweet volume label as displayed on getdaytrends.com	65.3K tweets; 10.0K tweets; Under 10K tweets

Bonus: The run also saves a summary to the key-value store as RUN_SUMMARY.json with:

• runStartedAt
• proxyUsed
• stats.total_items
• stats.per_timePeriod
• stats.failed_urls

You can view and export the dataset in multiple formats (CSV, JSON, Excel) from the Apify platform.

Key features

• ⚡ Live, row‑by‑row dataset saving
  Results are pushed to the Apify dataset as they’re parsed, providing immediate visibility and low memory usage.

• 🧭 Flexible time snapshots
  Capture Live and historical windows using granular flags: hour1–hour23, Yesterday, Week Ago, Month Ago, Year Ago, and Day 3.

• 🌍 Country-specific or worldwide coverage
  Choose a country (e.g., US, GB, IN) for localized trends or leave the country empty for worldwide coverage.

• 🔁 Resilient HTTP with retries
  Built-in retry logic with exponential backoff handles transient failures gracefully.

• 🛡️ Smart proxy handling with automatic fallback
  Uses your selected proxy settings and falls back to Apify RESIDENTIAL automatically if the chosen proxy is rejected.

• 🧩 Robust parsing with multiple XPath strategies
  Multiple selectors improve stability across layout variations on getdaytrends.com.

• 🧑‍💻 Developer-ready, Python-powered
  Runs on apify/actor-python:3.11 with clean, maintainable code for straightforward extension.

• 💾 Run summary for auditing
  Saves RUN_SUMMARY.json with timing, proxy used, and per‑timePeriod stats for traceability.

How to use Twitter Trends Scraper - step by step

1. Sign in to Apify
   Log in to your Apify account to run Twitter Trends Scraper from the Apify platform.

2. Open the actor and configure time flags
   Toggle the time windows you need: live, hour1–hour23, yesterday, weekAgo, monthAgo, yearAgo, or day3.

3. Choose the country (optional)
   Set the country field to a supported code (e.g., US, GB, IN) for localized results. Leave it empty for worldwide trends.

4. Configure proxy (optional)
   Use the Proxy configuration input. If your selected proxy is rejected, the actor will automatically fall back to Apify RESIDENTIAL.

5. Start the run
   Click Start. The actor will fetch your selected pages, retry on failures, and push each parsed trend row directly to the dataset.

6. Monitor progress
   Follow logs to see how many URLs were processed, item counts per timePeriod, and which proxy was used.

7. Download results
   Open the Dataset tab for your run to view, filter, and export items in CSV, JSON, or Excel formats.

Pro tip: Combine multiple time flags (e.g., live + hour1 + yesterday) to build richer time series in a single run. Audit runs using RUN_SUMMARY.json in the key‑value store.

Use cases

Use case name	Description
Marketing trend tracking	Identify trending topics to guide campaigns and content calendars with up-to-date snapshots.
Competitor and influencer monitoring	Track spikes and shifts in attention by comparing hourly and daily snapshots.
Editorial planning	Align coverage with emerging conversations using time-stamped trend captures.
Data analysis & dashboards	Feed time, timePeriod, trend, and volume into BI tools for trendline visualizations.
Academic & media research	Collect reproducible public snapshots for studies on attention dynamics and information diffusion.
Automation pipelines	Schedule runs and integrate dataset outputs into downstream APIs or workflows for alerts and enrichment.

Why choose Twitter Trends Scraper?

Purpose-built for reliability and clean outputs, this actor focuses on structured trend snapshots without overhead.

• 🎯 Precision output: Minimal, consistent fields (time, timePeriod, trend, volume) ideal for analytics.
• 🔁 Built-in resilience: Automatic retries with exponential backoff to reduce transient failures.
• 🛡️ Proxy robustness: Falls back to Apify RESIDENTIAL automatically if your chosen proxy is rejected.
• 🌍 Global or localized: Select a country or scrape worldwide for flexible coverage.
• 🧑‍💻 Developer friendly: Python-based actor with row‑by‑row pushes for streaming‑friendly workflows.
• 💾 Auditable runs: RUN_SUMMARY.json captures stats and proxy used for each execution.
• 🧱 Stable parsing: Multiple XPath strategies minimize breakage from layout changes.
  In short: a production‑ready alternative to brittle extensions or ad‑hoc scripts.

Is it legal / ethical to use Twitter Trends Scraper?

Yes—when used responsibly. This actor scrapes publicly available pages on getdaytrends.com and does not require login or access to private resources.

Guidelines for compliant use:

• Only collect publicly available information.
• Respect the target website’s terms of service and applicable laws (e.g., GDPR, CCPA).
• Avoid scraping private, gated, or user‑specific data.
• Verify compliance with your legal team for edge cases.

Input parameters & output format

Example input

{
  "live": true,
  "hour1": true,
  "yesterday": true,
  "weekAgo": false,
  "monthAgo": false,
  "yearAgo": false,
  "day3": false,
  "country": "US",
  "proxy": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"],
    "apifyProxyCountry": "US"
  }
}

Input parameters

Field	Type	Description	Default	Required
live	boolean	Scrape the current live trends.	not specified	No
hour1	boolean	Scrape trends from 1 hour ago.	not specified	No
hour2	boolean	Scrape trends from 2 hours ago.	not specified	No
hour3	boolean	Scrape trends from 3 hours ago.	not specified	No
hour4	boolean	Scrape trends from 4 hours ago.	not specified	No
hour5	boolean	Scrape trends from 5 hours ago.	not specified	No
hour6	boolean	Scrape trends from 6 hours ago.	not specified	No
hour7	boolean	Scrape trends from 7 hours ago.	not specified	No
hour8	boolean	Scrape trends from 8 hours ago.	not specified	No
hour9	boolean	Scrape trends from 9 hours ago.	not specified	No
hour10	boolean	Scrape trends from 10 hours ago.	not specified	No
hour11	boolean	Scrape trends from 11 hours ago.	not specified	No
hour12	boolean	Scrape trends from 12 hours ago.	not specified	No
hour13	boolean	Scrape trends from 13 hours ago.	not specified	No
hour14	boolean	Scrape trends from 14 hours ago.	not specified	No
hour15	boolean	Scrape trends from 15 hours ago.	not specified	No
hour16	boolean	Scrape trends from 16 hours ago.	not specified	No
hour17	boolean	Scrape trends from 17 hours ago.	not specified	No
hour18	boolean	Scrape trends from 18 hours ago.	not specified	No
hour19	boolean	Scrape trends from 19 hours ago.	not specified	No
hour20	boolean	Scrape trends from 20 hours ago.	not specified	No
hour21	boolean	Scrape trends from 21 hours ago.	not specified	No
hour22	boolean	Scrape trends from 22 hours ago.	not specified	No
hour23	boolean	Scrape trends from 23 hours ago.	not specified	No
yesterday	boolean	Scrape yesterday's full-day summary.	not specified	No
weekAgo	boolean	Scrape trends from 1 week ago.	not specified	No
monthAgo	boolean	Scrape trends from 1 month ago.	not specified	No
yearAgo	boolean	Scrape trends from 1 year ago.	not specified	No
day2	boolean	Scrape yesterday's full-day summary (legacy option, same as 'yesterday').	not specified	No
day3	boolean	Scrape the summary from 2 days ago.	not specified	No
country	string	Select a country to scrape country-specific Twitter trends. Leave empty for worldwide trends.	""	No
proxy	object	Defaults to no proxy. If the selected proxy is rejected, the actor falls back to Apify RESIDENTIAL automatically.	not specified	No

Notes:

• The country field accepts a predefined list of two‑letter codes (see input editor), including "" (Worldwide), US, GB, IN, and many more. Leave it empty for worldwide trends.
• The proxy object uses Apify’s standard proxy editor (supports Apify Proxy and custom URLs).

Output format

Each item pushed to the dataset has the following fields:

[
  {
    "time": "2026-04-07T11:30:12.345678Z",
    "timePeriod": "Live",
    "trend": "#Navratri2025",
    "volume": "23.8K tweets"
  },
  {
    "time": "2026-04-07T10:30:12.345678Z",
    "timePeriod": "1 hour ago",
    "trend": "Chiefs",
    "volume": "78.7K tweets"
  }
]

Field notes:

• volume may be an empty string when the page does not display a volume label.

FAQ

Does this scrape Twitter directly?

No. The actor scrapes public trend listings from getdaytrends.com and outputs structured records for each trend.

What fields are returned in the dataset?

Each dataset item includes time, timePeriod, trend, and volume. This minimal schema is designed for clean analytics and time‑series use.

Can I capture historical snapshots?

Yes. Enable hour1–hour23, yesterday, weekAgo, monthAgo, yearAgo, or day3 to capture historical snapshots alongside Live.

Does it support country-specific trends?

Yes. Use the country field for localized trends or leave it empty for worldwide coverage. The actor logs whether it’s scraping a specific country or worldwide.

How does proxy fallback work?

If your selected proxy (custom or Apify Proxy) is rejected, the actor automatically falls back to Apify RESIDENTIAL. The run logs the proxy in use and stores proxyUsed in RUN_SUMMARY.json.

Is login or cookies required?

No. The scraper accesses public pages on getdaytrends.com and does not require authentication.

How reliable is it if some pages fail?

The actor performs HTTP retries with exponential backoff and logs failures. Items are pushed row‑by‑row, so partial progress is preserved even if some URLs fail.

Where can I find run diagnostics?

A RUN_SUMMARY.json file is saved in the key‑value store with runStartedAt, proxyUsed, total item counts, per‑timePeriod stats, and any failed URLs.

Is there a free trial?

Yes. This actor includes trial minutes to test the workflow before subscribing. Check the Apify listing for current trial allocation and pricing details.

Closing CTA / Final thoughts

Twitter Trends Scraper is built to capture clean, time‑stamped trend snapshots from getdaytrends.com reliably and at scale. With flexible time windows, optional country selection, resilient retries, and automatic proxy fallback, it’s ideal for analysts, marketers, developers, and researchers who need dependable trend data. Run it on Apify, stream structured results to your dataset, and integrate the output into your analytics or automation pipelines. Start extracting smarter trend insights today.