Telegram Channel Scraper

Telegram Channel Scraper

A Python async scraper that fetches posts from multiple Telegram channels concurrently using AWS Lambda API. Supports both standalone execution and Apify Actor deployment.

Features

• ✅ Concurrent scraping of multiple channels
• ✅ Auto-pagination for large datasets
• ✅ Extract posts with full metadata (date, views, reactions, polls)
• ✅ Media support (images, videos, link previews)
• ✅ Error handling and detailed logging
• ✅ Configurable limits per channel

Configuration

Edit to configure scraping parameters:

{
  "channels": [
    "bloomberg",
    "techcrunch",
    "wired",
    "theverge"
  ],
  "limit": 50,
  "before": null
}

Configuration Options

Parameter	Type	Required	Description
channels	array	Yes	List of Telegram channel usernames
limit	integer	No	Number of posts per channel (default: all available)
before	string	No	Post ID for pagination (rarely needed)

---

Output:

🚀 Starting scraping for 4 channel(s)...
📋 Channels: bloomberg, techcrunch, wired, theverge
📊 Limit per channel: 50

📡 Fetching data for channel: bloomberg
📡 Fetching data for channel: techcrunch
📡 Fetching data for channel: wired
📡 Fetching data for channel: theverge

⏱️  Scraping completed in 3.45 seconds

======================================================================
                         SCRAPING SUMMARY                          
======================================================================
✅ bloomberg                     |    50 posts
✅ techcrunch                    |    50 posts
✅ wired                         |    50 posts
✅ theverge                      |    50 posts
======================================================================
Total Channels: 4 | Successful: 4 | Failed: 0
Total Posts Scraped: 200
======================================================================

Apify Actor Usage

1. Deploy to Apify:

   • Upload the Actor to Apify platform
   • Set environment variables in Apify settings

2. Configure Input:

{
     "channels": ["bloomberg", "techcrunch"],
     "limit": 50
   }

3. Run the Actor

---

API Parameters

Query String Parameters (Lambda API)

Parameter	Type	Required	Description	Example
channel	string	Yes	Telegram channel username	bloomberg
limit	integer	No	Number of posts to retrieve	50
before	string	No	Post ID for pagination	12345

---

Output Format

JSON Output Structure

{
  "scrape_timestamp": "2026-02-09T10:30:00.000000",
  "total_channels": 4,
  "successful": 4,
  "failed": 0,
  "results": [
    {
      "channel": "bloomberg",
      "status": "success",
      "timestamp": "2026-02-09T10:30:05.123456",
      "data": {
        "success": true,
        "channel": "bloomberg",
        "posts_count": 50,
        "requested_limit": 50,
        "posts": [
          {
            "date": "February 09, 2026",
            "fulldate": "2026-02-09T10:15:00+00:00",
            "description": "Post content here...",
            "image": "https://cdn.telegram.org/...",
            "video": "",
            "preview": "",
            "link": "https://t.me/bloomberg/12345",
            "views": "10.5K",
            "reply": "",
            "poll": "",
            "reactions": [
              {"emoji": "👍", "count": "125"},
              {"emoji": "❤️", "count": "89"}
            ],
            "edited": false
          }
        ]
      }
    }
  ]
}

---

Examples

Example 1: Scrape 3 News Channels

{
  "channels": ["CNN", "BBCNews", "Reuters"],
  "limit": 100
}

Example 2: Scrape Tech Channels

{
  "channels": ["techcrunch", "wired", "theverge", "engadget"],
  "limit": 25
}

Troubleshooting

Issue: "HTTP 403 Forbidden"

Solution: Check your API key is correct and has not expired.

Issue: "Getting fewer posts than requested"

Reason: The channel may have fewer posts available, or Telegram's rate limiting is active.

Issue: "Timeout errors"

Solution: Reduce the number of concurrent channels or increase timeout:

timeout = aiohttp.ClientTimeout(total=600)  # 10 minutes

Happy Scraping! 🚀