Streem Article Scraper – Webhook Support

A fast and reliable tool for extracting full-text content and metadata from Streem article links. This scraper is designed to support media teams, analysts, and researchers who need consistent, structured article data—automatically delivered via webhook.

🚀 Features

• ✅ Multi-URL Support – Process one or many Streem article links in a single run
• ⚙️ Concurrent Scraping – Handle multiple articles in parallel for faster results
• 🔄 Webhook Integration – Automatically send results to your endpoint after completion
• 🧹 Clean Content Output – Removes excess HTML/CSS, returns plain, readable text
• 🔒 Optional Webhook Auth – Add a token to secure your webhook endpoint
• 📄 Consistent Output – Each result includes success/error status, article data, and input URL

🔧 How It Works

1. Provide Input

You can supply either:

• urls: A list of Streem article links
• url: A single Streem article link

Additional options:

• concurrency: Number of articles to process at the same time (default: 3)
• webhook_url: (Optional) Where the data will be sent after scraping
• webhook_method: POST or PUT (default: POST)
• webhook_token: (Optional) Used as a bearer token to secure your webhook

2. Processing

For each URL, the scraper will:

• Fetch the page
• Extract the clean article text and metadata
• Package the data into a consistent JSON result
• Flag errors while keeping the structure intact

3. Delivery

When a webhook_url is provided, all results are sent automatically to your system—no manual downloading or polling required.

📥 Input Example

{
  "urls": [
    "https://links.streem.com.au/dubbo-photo-news-20250702-XVg7X1h8q7U4Hnh7EbRnI8cEjRRve",
    "https://links.streem.com.au/the-kimberley-echo-20250702-2716altLN5CQHRh"
  ],
  "concurrency": 3,
  "webhook_url": "https://your-api.com/webhook/streem-articles",
  "webhook_token": "my-secret-token-123",
  "webhook_method": "POST"
}