Site QA Indexability AI Crawler Report

Wayback Machine Checker — Snapshot Diff Alerts

taroyamada/wayback-machine-checker

Compare Wayback snapshots and detect changes. No API key / no login. Open Input → leave defaults → Start (1 URL ≈ USD 0.008): https://console.apify.com/actors/fy1UnkQNIzGyy8HIs/input From USD 8/1k.

Web Crawler & Semantic Schema-Enhanced Extractor

devil_port369-owner/web-crawler

Depth-controlled web crawler that transforms websites into structured analytics-ready data. Starting from one or more URLs, it crawls internal links up to a configurable depth and outputs detailed JSON records per page

Firecrawl Site Mapper

alizarin_refrigerator-owner/firecrawl-site-mapper

Fast URL Discovery for Site Audits & Competitor Analysis Discover all URLs on a website using Firecrawl's Map endpoint. Perfect for competitor analysis, site audits, and content gap discovery.

Contact Page Audit & Business Channel Extractor

taroyamada/contact-details-extractor

Audit public contact and support pages. No API key / no login. Open Input → leave defaults → Start (1 contact URL ≈ USD 0.01/page): https://console.apify.com/actors/1DsM6JuHN1ySAcgPm/input

Bulk URL Health Checker — Status and Redirects

taroyamada/bulk-url-health-checker

Check HTTP status, redirects, and response time across URLs. No API key / no login. Open Input → leave defaults → Start: https://console.apify.com/actors/hHvfTzycQXbARYlZs/input

Sitemap Analyzer — Nested XML URL Audit

taroyamada/sitemap-analyzer

Analyze XML sitemaps. No API key / no login. Open Input → leave defaults → Start (1 sitemap + maxUrls 3 ≈ USD 0.005 total): https://console.apify.com/actors/wJILhGwhAXWPfqBA2/input

Meta Tag & OpenGraph Scraper

taroyamada/meta-tag-analyzer

Audit meta tags, OpenGraph, SEO signals. No API key / no login. Open Input → leave defaults → Start (1 page ≈ USD 0.003/row): https://console.apify.com/actors/pRIbTQnVnGcacOLMv/input From USD 3/1k.

robots.txt Parser & AI Crawler Block Checker

taroyamada/robotstxt-ai-checker

Check AI crawler access. No API key / no login. Open Input → leave defaults → Start (1 domain ≈ USD 0.011/row): https://console.apify.com/actors/Mkf2MMndaW7UdtuOP/input

Algolia Website Indexer

apify/algolia-website-indexer

The Indexer crawls recursively a website using the Puppeteer browser (headless Chrome) and indexes the selected pages to the Algolia index.

tsboi index

trim_flag/tsboi-index

Indexing for LLMs. This application crawls specified websites, processes their content for LLMs.