Under maintenance

Pricing

Pay per usage

Try for free

Go to Apify Store

AI Crawler Log Analyzer — GPTBot, Claude & Perplexity

Under maintenance

Try for free

Analyze Nginx, Apache, Cloudflare, and JSON server logs for AI search, user-fetch, and training crawlers. Get bot traffic, errors, bandwidth, top pages, and actionable recommendations without exposing raw lines in dataset output.

Pricing

Pay per usage

Rating

0.0

(0)

Developer

Luigy Gabriel

Actor stats

Bookmarked

Total users

Monthly active users

20 days ago

Last modified

Categories

Developer tools

SEO tools

You can access the AI Crawler Log Analyzer — GPTBot, Claude & Perplexity programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in API & Integrations in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1import { ApifyClient } from 'apify-client';
2
3// Initialize the ApifyClient with your Apify API token
4// Replace the '<YOUR_API_TOKEN>' with your token
5const client = new ApifyClient({
6    token: '<YOUR_API_TOKEN>',
7});
8
9// Prepare Actor input
10const input = {};
11
12// Run the Actor and wait for it to finish
13const run = await client.actor("plenteous_humidifier/ai-crawler-log-analyzer").call(input);
14
15// Fetch and print Actor results from the run's dataset (if any)
16console.log('Results from dataset');
17console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
18const { items } = await client.dataset(run.defaultDatasetId).listItems();
19items.forEach((item) => {
20    console.dir(item);
21});
22
23// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

AI Crawler Log Analyzer for GPTBot & AI Search API in JavaScript

The Apify API client for JavaScript is the official library that allows you to use AI Crawler Log Analyzer — GPTBot, Claude & Perplexity API in JavaScript or TypeScript, providing convenience functions and automatic retries on errors.

Install the apify-client

$npm install apify-client

Other API clients include:

AI Crawler Log Analyzer — GPTBot, Claude & Perplexity API in Python

AI Crawler Log Analyzer — GPTBot, Claude & Perplexity API through CLI

AI Crawler Log Analyzer — GPTBot, Claude & Perplexity OpenAPI definition

AI Crawler Log Analyzer — GPTBot, Claude & Perplexity API

AI Agent Traffic Detector

seeb/ai-agent-traffic-detector

Detect AI crawler and bot traffic from access-log lines or public bot policy pages.

Techionik

Robots Txt Audit

apage/robots-txt-audit

Audit robots.txt files for AI crawler access. Get an AI Readiness Score (0-100), analyze 61+ AI crawlers (ChatGPT, Claude, Perplexity, Gemini), detect syntax errors, security concerns, and get actionable recommendations. Batch audit multiple domains at once with optional subdomain scanning.

Andy Page

AI Readability Checker — Can ChatGPT & Claude Read Your Site?

eliai/ai-readability-checker

Audit a site the way AI crawlers see it: robots.txt rules per bot (GPTBot, ClaudeBot, PerplexityBot), real per-bot fetches, and how much text exists before JavaScript runs. Weighted from crawler logs, not blog advice. $0.05 per audit.

Anthony Snider

Perplexity AI Search Scraper

zhorex/perplexity-ai-scraper

Search Perplexity AI programmatically. Get AI-generated answers with sources and citations. No API key needed. The only Perplexity scraper on Apify that works without your own API key. Built for AEO and brand monitoring. Fast, clean JSON output.

Sami

103

AI Crawler Access Checker

q_services/ai-crawler-access-checker

Audit robots.txt policies for GPTBot, ClaudeBot, Google-Extended, CCBot, PerplexityBot and other AI crawlers.

Q Services

AI Crawler Robots.txt Auditor

phoenix2810/ai-crawler-robots-auditor

Fetch a public website's robots.txt and report which AI crawlers (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot, Bytespider, and more) are blocked, allowed, or defaulted. Returns an AI-visibility readiness score with recommendations for GEO and AI-SEO strategy.

Sanskar Jaiswal

Crawler Traffic Log Report

checksmithcats/crawler-traffic-log-report

Summarizes crawler-claim traffic, current robots-policy mismatch signals, status codes, and redacted HTTP error patterns from submitted access logs.

Checksmith Cats

AI Brand Monitor Track in ChatGPT, Claude, Perplexity & Gemini

alizarin_refrigerator-owner/ai-brand-monitor

Monitor how major AI assistants talk about your brand. Input a business name and get an AI Visibility Score (0-100), sentiment analysis across platforms, competitor mentions, and actionable recommendations to improve your AI presence.