Faceted Navigation Auditor

Website URL Crawler & Link Extractor

maximedupre/website-url-crawler

Crawl public websites and collect URLs from rendered navigation and sitemaps. Export a link map with source pages, depth, anchor text, HTTP facts, crawl status, and available sitemap metadata.

LinkedIn Company Jobs Monitor

apixyz/linkedin-company-jobs-monitor

Monitor new, updated, and confirmed-removed job postings for up to 20 companies on LinkedIn's public job search. Faceted enumeration covers large employers; confirmed removals require a complete scan. No login or cookies.

G2 Scraper - Reviews, Products, Analytics & Discussions

kaix/g2-scraper

🔥 ~$0.05/1K reviews 🔥 Scrape G2.com software reviews, products, analytics, comparisons, and discussions. Faceted filtering, facet-value discovery, 72 review fields, NLP sentiment, switching intelligence.

Site QA Indexability AI Crawler Report Scraper

taroyamada/site-qa-indexability-ai-crawler-report-scraper

Unofficially audit user-supplied public pages, robots.txt, and llms.txt signals for AI crawler indexability issues and source-linked report rows.

BeiDou Navigation System News Scraper

jungle_synthesizer/beidou-gov-cn-bds-navigation-news-scraper

Scrapes official news, announcements, and policies from beidou.gov.cn — China's BDS satellite navigation authority. Covers 5 sub-channels: news, announcements, policy, data exchange, and applications. Outputs zh-CN articles with title, content, publish date, source, images, and attachments.

Sitemap & Robots SEO Index Auditor

glowing_glove/sitemap-robots-index-auditor

Audit robots.txt, sitemap discovery, indexability signals, canonical tags, and SEO crawl readiness for business websites.

Crawl any website and extract clean article text and Markdown, ready to feed into LLMs, ChatGPT, vector databases and RAG pipelines. Removes navigation, ads and boilerplate. Configurable crawl depth and page limits. Export to JSON, CSV, Excel.

website-change-monitor

desmond-dev/website-change-monitor

Monitor any website for changes. If content updates, convert it to clean Markdown. Perfect for RAG, AI Agents, and competitor tracking. Ignores ads and navigation.

Extract clean text from any news site or blog. Removes ads, navigation, and HTML. Returns structured JSON ready for AI training, ChatGPT, RAG pipelines. Fast & Free proxy supported.

Indexability Audit

zerobreak/indexability-audit

Indexability audit tool that checks robots.txt, meta robots tags, X-Robots-Tag headers, and canonical URLs for any list of pages, so SEO teams know which ones Google can actually crawl and index.