🚀 X (Twitter) Trending Topics Scraper

A high-speed, robust Puppeteer scraper to extract Trending Categories and Topics from X.com.

This Actor extracts the latest trending topics (e.g., "Business", "Sports", "Politics") and their associated keywords/hashtags for any country. It uses advanced coordinate-based interaction ("Nuclear Click") to bypass X's virtualized lists and complex event listeners, ensuring 100% data accuracy where standard scrapers fail.

✨ Features

• 🌍 Global & Local Support: Scrape trends for Global, India, United States, or any country available in the X location dropdown.
• ⚡ High-Speed Extraction:
  • Uses Smart DOM Listeners to detect changes instantly (no slow hard waits).
  • Blocks heavy assets (images/fonts) to reduce bandwidth and speed up runs.
• 🛡️ "Nuclear" Interaction Engine: Uses physical mouse coordinate calculations to click elements, bypassing React event blocking and anti-bot measures.
• 📂 Clean Data Export: Saves data to the Apify Dataset, easily downloadable as JSON, CSV, or Excel.

🛠️ Input Configuration

The Actor takes the following inputs:

Field	Type	Description	Required	Default
country	String	The country to scrape (e.g., India, United States, Global).	❌ No	Global
targetCategories	Array	Specific categories to scrape (e.g., ["Business", "Sports"]). Leave empty to scrape ALL.	❌ No	[]
blockAssets	Boolean	Block images/fonts for faster scraping?	❌ No	true

🔑 How to get your authToken

To access X trends, the scraper needs to be logged in. You must provide your session cookie:

1. Log in to X.com.
2. Open Developer Tools (F12 or Right Click -> Inspect).
3. Go to the Application tab.
4. In the left sidebar, click Cookies -> https://x.com.
5. Find the cookie named auth_token.
6. Copy its Value and paste it into the Actor input.

🔒 Security Note: Your auth_token is handled securely inside the Actor and is only used to authenticate the scraping session. It is never shared or stored externally.

📊 Output Data

The results are stored in the default Apify Dataset. Each item represents a Category and its Trending Topics.

JSON Example

{
  "category": "Business & Finance",
  "topics": [
    "Stock Market",
    "Crypto",
    "Nvidia",
    "Elon Musk"
  ],
  "topicsFormatted": "Stock Market, Crypto, Nvidia, Elon Musk"
}

CSV Export Example

If you download the dataset as CSV, it will look like this:

Category	TopicsFormatted
Business & Finance	"Stock Market, Crypto, Nvidia, Elon Musk"
Sports	"Cricket, IPL, Kohli, Football"

⚙️ How it works (Under the Hood)

1. Authentication: The Actor injects your auth_token into a stealth Puppeteer browser instance.
2. Country Switching: It navigates to the Trends page and uses the Nuclear Click method (Coordinate Calculation + Physical Mouse Event) to open the dropdown and select your target country, even if it requires scrolling.
3. Category Walking: It identifies the category carousel (Top Row) and iterates through every category.
4. Smart Waiting: After clicking a category, it snapshots the current topic list and waits specifically for the text to change. This makes it significantly faster than scrapers using fixed sleep timers.
5. Extraction: It captures the new topics and pushes the structured data to the dataset.

---

Built with ❤️ using Apify SDK and Puppeteer.