AI News Content Crawler

Multi-Source News & Content Scraper

moving_beacon-owner1/rss-feeds----multi-source-news-content-scraper

Multi-Source News & Content Scraper. Aggregates articles from multiple RSS/Atom feeds simultaneously. Includes 60+ pre-built news source presets and supports custom feed URLs. No API key required.

Web Content Extractor API — URL to JSON

george.the.developer/web-content-extractor-api

Extract structured JSON from any webpage. Articles, products, recipes, jobs. Auto-detects content type. Returns metadata, headings, images, links. For AI agents and RAG.

Awesome Google News Scraper

sync-network/awesome-google-news-scraper

This tool scrapes content from Google News, streamlining the collection of latest the information on any topic. Its key feature is the ability to extract full-length articles, not just headlines. Customize results from brief summaries to complete content, revolutionizing your news gathering process.

Brave Search API — Web, News, Image & Video Search

junipr/brave-search

Search the web using the Brave Search API. Extract organic results, news, videos, and related searches for any query. Fast, private, and independent search results without Google dependencies.

RSS & News Intelligence - Real-Time AI Monitor

viralanalyzer/rss-news-intelligence

Monitor RSS feeds and Google News with AI-powered summarization.

📡 RSS & Atom Feed Extractor

taroyamada/rss-feed-aggregator

Aggregate public RSS feeds into structured JSON datasets to discover fresh website URLs from blogs and newsrooms for downstream web scrapers.

Train Your Local LLM for Business & Finance - DataPro

omissive_aurora/train-your-local-llm-for-business-finance---datapro

Train your local LLM for business and finance with Ultimate DataPro. Scrapes live stock prices, SEC EDGAR filings, options chains, and financial news - then auto-builds Alpaca/ShareGPT fine-tuning datasets. Export as JSONL, CSV, or Parquet. Push to HuggingFace Hub.

Aggregate and parse RSS/Atom feeds from any source. Extract articles with titles, descriptions, authors, dates, images. Optionally fetch full article content. Perfect for news monitoring and AI pipelines. $0.0005/article.

📄 Article Content Scraper & Extractor

taroyamada/article-content-extractor

Scrape clean article bodies, authors, and metadata from messy newsrooms. Built for AI models, NLP datasets, and SEO audits requiring pristine text.

📰 Hacker News Scraper

taroyamada/hacker-news-intelligence

Extract trending tech discussions, nested comment hierarchies, and post scores from Hacker News directly into structured JSON for custom RAG pipelines.

📄 Website Content Extractor

taroyamada/website-content-extractor

Strip noise from general website pages to extract clean markdown and structured text. Perfect for building LLM datasets from docs, pricing, and product pages.