Website Metadata Scraper

List of URLs whose HTTP redirect chains will be analyzed

Universal Web Scraper & Data Extractor

motivational_nickel/universal-scraper

Fast AI-ready webpage extraction with structured JSON output for automation workflows, AI agents, RAG pipelines, SEO analysis, and website intelligence.

URL Metadata Crawler

easyapi/url-metadata-crawler

Extracting comprehensive metadata from web pages. Gather vital information like meta tags, favicons, Open Graph tags, and more, all while enjoying flexible options for customization. Perfect for SEO specialists, developers, and content creators looking to enhance their web presence! 🌐

Extract all URLs from any website's sitemap. Auto-discovers sitemaps from robots.txt, supports sitemap index files and .gz compression. Filter by URL pattern, date range.

Firecrawl Site Mapper

alizarin_refrigerator-owner/firecrawl-site-mapper

Fast URL Discovery for Site Audits & Competitor Analysis Discover all URLs on a website using Firecrawl's Map endpoint. Perfect for competitor analysis, site audits, and content gap discovery.

Link Extractor

automation-lab/link-extractor

This actor extracts all hyperlinks from web pages. For each link, it captures the anchor text, href, rel attributes (nofollow, ugc, sponsored), target attribute, and classifies links as internal or external. It also detects the link's location in the page (nav, header, footer, main content,...

Answer Engine Optimization & GEO Audit - ChatGPT Fix Plan

tidytools/geo-readiness-audit

AEO with a fix plan, not just a score: for each buyer question, see which page ChatGPT, Perplexity or Google AI Overviews would quote, why competitors win, and what to add. Plus a GEO audit of AI crawler access and firewalls. $0.01/question, $0.03/site.

TikTok Comments Scraper

socialdataapi/tiktok-comments-scraper

TikTok Comments Scraper — Extract TikTok comments with usernames, comment text, likes, timestamps, and related video data. Useful for audience research, sentiment analysis, content insights, competitor analysis, and social media monitoring.

llms.txt Generator & Checker - llms-full.txt, AI SEO (GEO)

tidytools/llms-txt-generator

Generate llms.txt and llms-full.txt for any website from its sitemap or a crawl, with sections and AI descriptions, or validate existing llms.txt files in bulk. $2 per 1,000 pages or sites.

ConsumerAffairs Reviews Scraper

parsebird/consumeraffairs-scraper

Scrape ConsumerAffairs reviews for any company: rating, full text, author, location, date, topics, verified badge, and company replies, plus company rating, review count, and profile. Export JSON, CSV, Excel.