Pricing

Pay per usage

AI Crawler Log Analyzer — GPTBot, Claude & Perplexity

Analyze Nginx, Apache, Cloudflare, and JSON server logs for AI search, user-fetch, and training crawlers. Get bot traffic, errors, bandwidth, top pages, and actionable recommendations without exposing raw lines in dataset output.

Pricing

Pay per usage

Rating

0.0

(0)

Developer

Luigy Gabriel

Actor stats

Bookmarked

Total users

Monthly active users

8 hours ago

Last modified

Categories

Developer tools

SEO tools

You can access the AI Crawler Log Analyzer — GPTBot, Claude & Perplexity programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1import { ApifyClient } from 'apify-client';
2
3// Initialize the ApifyClient with your Apify API token
4// Replace the '<YOUR_API_TOKEN>' with your token
5const client = new ApifyClient({
6    token: '<YOUR_API_TOKEN>',
7});
8
9// Prepare Actor input
10const input = {};
11
12// Run the Actor and wait for it to finish
13const run = await client.actor("plenteous_humidifier/ai-crawler-log-analyzer").call(input);
14
15// Fetch and print Actor results from the run's dataset (if any)
16console.log('Results from dataset');
17console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
18const { items } = await client.dataset(run.defaultDatasetId).listItems();
19items.forEach((item) => {
20    console.dir(item);
21});
22
23// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

AI Crawler Log Analyzer for GPTBot & AI Search API in JavaScript

The Apify API client for JavaScript is the official library that allows you to use AI Crawler Log Analyzer — GPTBot, Claude & Perplexity API in JavaScript or TypeScript, providing convenience functions and automatic retries on errors.

Install the apify-client

$npm install apify-client

Other API clients include:

AI Crawler Log Analyzer — GPTBot, Claude & Perplexity API in Python

AI Crawler Log Analyzer — GPTBot, Claude & Perplexity API through CLI

AI Crawler Log Analyzer — GPTBot, Claude & Perplexity OpenAPI definition

AI Crawler Log Analyzer — GPTBot, Claude & Perplexity API

AI Crawler Robots.txt Audit & llms.txt Checker

davidbenittah/ai-crawler-robots-audit

Check robots.txt and llms.txt on up to 50 sites and see which AI crawlers are blocked: OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-SearchBot, GPTBot and ClaudeBot. Returns a 0-100 AI search access score, the rule matched per bot, and a paste-ready robots.txt fix.

David

AI Search Readiness Audit (AEO/GEO)

burly_bat/ai-search-readiness-audit

Bulk-audit URLs for AI search readiness (AEO/GEO). Checks 13 AI crawlers against robots.txt with quoted evidence, llms.txt, JSON-LD structured data, no-JS renderability and citation signals. PASS/WARN/FAIL + 0-100 score per URL and a client-ready HTML report. Input: URLs, sitemap or dataset.

Burly Bat

GEO Competitive Benchmark — AI Search Readiness vs Competitors

foxlabs/geo-benchmark

Benchmark your site vs up to 5 competitors for AI-search readiness in one run. See where rivals out-rank you for ChatGPT, Perplexity, Gemini & Claude citation — AI-crawler access, schema, extractability, speed, trust — as a ranked scoreboard + per-signal gap list. Deterministic GEO/AEO.

Berkan Kaplan

GEO Auditor — AI Search Readiness & Citability Audit

foxlabs/geo-auditor

Audit how ready your site is to be found, read & cited by AI search (ChatGPT, Perplexity, Gemini, Claude). Checks AI-crawler access, structured data, content extractability, speed & trust — scored 0-100 with a prioritized fix list. GEO / AEO technical audit.

Berkan Kaplan

LinkedIn Scraper: Posts, Reactions, Comments & Engagement

scrapio/linkedin-company-page-scraper

Scrapes full data from any public LinkedIn company page, including About info, employees, posts, insights, specialties, locations, size, and industry. Ideal for lead enrichment, competitor analysis, market research, and automated company intelligence at scale

Scrapio

100

5.0

(1)

Facebook Comments Scraper: Sentiment & Toxicity Analysis

simpleapi/facebook-comments-scraper

💬 Facebook Comments Scraper scrapes comments & replies from public posts — author names, text, timestamps, reactions & permalinks. 🔍 Clean, dedupe, and export to CSV/JSON. 📊 Ideal for social listening, sentiment, marketing, and research. 🚀 Fast, reliable, API-ready.

SimpleAPI

Twitter X Reply Scraper: Reply Sentiment & Tone Breakdown

scrapio/twitter-x-reply-scraper

💬 Twitter/X Reply Scraper extracts tweet replies at scale—usernames, text, timestamps, IDs & thread context. 🔎 Perfect for social listening, sentiment, competitor research & engagement tracking. ⚡ CSV/JSON export for dashboards, BI & automation. 🚀

Scrapio

Google Maps Scraper — Reviews & Sentiment Analysis

scraper-engine/google-maps-scraper

Scrape business listings from Google Maps to collect names, addresses, phone numbers, websites, and more. Perfect for generating local B2B leads or enriching your CRM with location-based contact data.

Scraper Engine

278

5.0

(4)

Robots Txt Audit

apage/robots-txt-audit

Audit robots.txt files for AI crawler access. Get an AI Readiness Score (0-100), analyze 61+ AI crawlers (ChatGPT, Claude, Perplexity, Gemini), detect syntax errors, security concerns, and get actionable recommendations. Batch audit multiple domains at once with optional subdomain scanning.

Andy Page

AI Visibility Checker API

dev00/visibility-checker-api

Analyze domain visibility, brand authority, and search visibility across search engines and AI crawlers.

dev00

AI Readiness Checker - Website Scanner

alizarin_refrigerator-owner/ai-readiness-checker

Analyze any website for AI optimization readiness. Check robots.txt, llms.txt, structured data, meta tags & content quality. Get actionable recommendations to improve AI crawler accessibility.