Pricing

from $0.01 / 1,000 results

LLMs.txt Checker - AI Readiness Scanner

Batch check websites for llms.txt files & identify AI optimization opportunities. A emerging standard that websites use to communicate with AI systems. It helps LLMs understand, What the company does, How to present information about the business, Contact information & key pages Brand guidelines

Pricing

from $0.01 / 1,000 results

Rating

0.0

(0)

Developer

The Howlers

Actor stats

Bookmarked

Total users

Monthly active users

8 days ago

Last modified

LLMs.txt Checker - AI Readiness Scanner with Recommendations

Check websites for llms.txt and llms-full.txt files and assess AI/LLM readiness. Batch-check hundreds of URLs, analyze file structure and sections, get AI-generated recommendations for improving existing files, and identify prospects missing AI optimization.

Quick Start

Test with Demo Mode (free, no API key needed)

{
  "demoMode": true,
  "urls": [
    "https://example.com"
  ]
}

Run with real data

{
  "demoMode": false,
  "urls": [
    "https://example.com"
  ],
  "includeRecommendations": true,
  "identifySalesOpportunities": true
}

Input Parameters

Parameter	Type	Default	Required	Description
`urls`	array	-	No	List of website URLs to check for llms.txt files. Will check both /llms.txt and /llms-full.txt
`includeRecommendations`	boolean	`true`	No	Generate AI-powered recommendations for improving llms.txt content
`identifySalesOpportunities`	boolean	`true`	No	Flag sites missing llms.txt or with poor implementations as sales opportunities
`webhookUrl`	string	-	No	Optional webhook URL to receive results when processing completes
`demoMode`	boolean	`true`	No	Run with sample data for testing (no URLs required)

Pricing

This actor uses pay-per-event billing:

Event	Description	Price
URL Checked	Each URL checked for llms.txt files	$0.01

Demo mode is free -- no charges for sample data.

Troubleshooting

"API error 429" or "Rate limit"

Too many requests. Wait a minute and try again, or reduce the number of items per run.

No results or empty dataset

Check the run log for error messages. Common causes:

Invalid input format (check the examples above)
The target data doesn't exist or is too small to track

How do I test without an API key?

Enable Demo Mode in the input. This returns realistic sample data so you can verify the output format works for your workflow.

Built by John Rippy | Actor Arsenal

/llms.txt Generator

jakub.kopecky/llmstxt-generator

The /llms.txt Generator 🕸️📄 extracts website content to create an llms.txt file for AI apps 🤖✨ like LLM fine-tuning and indexing. Output is available 📥 in the Key-Value Store for easy download and integration into workflows. 🚀

Jakub Kopecký

761

3.1

Website Content to Markdown for LLM Training

easyapi/website-content-to-markdown-for-llm-training

🚀 Transform web content into clean, LLM-ready Markdown! 📘 Scrape multiple pages, extract main content, and convert to Markdown format. Perfect for AI researchers, data scientists, and LLM developers. Fast, efficient, and customizable. Supercharge your AI training data today! 🌐📝🧠

EasyApi

283

5.0

Gemini AI MCP SERVER

bhansalisoft/gemini-ai-mcp-server

Gemini AI MCP SERVER unique tool for Gamini AI functionality integration with apify and other AI tool.

bhansalisoft

Website Content Crawler Pro

datascoutapi/website-content-crawler-pro

Crawl websites and extract clean, structured content in Markdown, JSON, or plain text for AI models, LLMs, vector DBs, or RAG pipelines. Fast, reliable, and stealthy, with bulk processing, advanced metadata extraction, and seamless integration with LangChain, LlamaIndex, and AI workflows.

halam

534

2.1

AI Website Content Markdown Scraper

quaking_pail/ai-website-content-markdown-scraper

This Apify Actor, "Website Content Crawler with Markdown Extraction," is designed to perform a comprehensive crawl of specified websites, extract their text content, convert it into Markdown format, and store it in a structured dataset. The extracted content is suitable for feeding LLMs.

AI_Builder

925

2.3

Webpage to Markdown

extremescrapes/webpage-to-markdown

This actor cost-effectively converts websites into structured markdown optimized for AI processing. It extracts webpage content, formats it into clean markdown, and ensures compatibility with AI models.

Extreme Scrapes

199

5.0

Quick Website Content Scraper ( Extract Text for RAG & LLMs )

automateitplease/ai-web-content-scraper-extract-text-for-rag-llms

Extract clean text from any website for AI/LLM applications. Supports both static and JavaScript-rendered sites (React, Vue, Angular). Perfect for RAG systems, chatbot training, and content analysis.

AutomateItPlease Workflow And Automaton Ops

✨ Google Search Results Scraper

damilo/google-search-apify

⚡️ Lightning-fast Google Search Scraper. Grab pinpoint-accurate, geo-targeted SERP data with query, lang, country, date, and pagination controls. Ideal for SEO, market research & analytics. Delivers clean JSON for instant integration.

Imad

602

Webpage Text Extractor

automation-lab/webpage-text-extractor

This actor fetches web pages and extracts their clean text content by stripping all HTML tags, scripts, and styles. It identifies the main content area (article, main, etc.), extracts headings structure, page links, and metadata like author, publish date, and language. Use it for LLM input...

Stas Persiianenko

AI Travel Agent

harvestlabs/ai-travel-agent

Helps users plan their trips - simply provide your travel request, and the AI will generate structured recommendations, including flights and accommodations, to suit your needs