Tech Blog Aggregator

Aggregate articles from top technology blogs and news sources into a unified dataset. Supports multiple sources including Hacker News, dev.to, TechCrunch, The Verge, and Ars Technica.

Features

• Multi-source aggregation from the most popular tech news and blog platforms
• Structured data extraction including title, summary, author, date, tags, and URL
• Hacker News API integration for reliable top story extraction with point scores
• dev.to API support for developer community content with tags and reactions
• Configurable source selection to choose exactly which blogs to aggregate
• Deduplication of articles that appear across multiple sources

Use Cases

• Build a personalized tech news feed for your team or organization
• Monitor technology trends across multiple publications simultaneously
• Create datasets for NLP analysis of tech industry coverage
• Feed content into recommendation engines or content curation tools
• Track emerging technologies and developer tool announcements

Input Configuration

Parameter	Type	Default	Description
sources	array	["hackernews", "devto"]	Blog sources to scrape
maxArticles	integer	200	Maximum articles per source

Output Format

Each article produces a dataset item with:

• source - The originating blog/platform name
• title - Article title
• summary - Article summary or description
• author - Author name when available
• date - Publication date
• url - Direct link to the article
• tags - Array of associated tags or categories
• score - Engagement score (points, reactions) when available
• scrapedAt - ISO timestamp of extraction

Supported Sources

Currently supported sources include Hacker News, dev.to, TechCrunch, The Verge, and Ars Technica. Additional sources can be added by providing the source name.

Limitations

• Some sources use heavy JavaScript rendering and may return limited data
• Rate limiting is automatically handled but may slow down large requests
• Article summaries are truncated to maintain reasonable dataset sizes