Pricing

Pay per usage

Go to Store

/llms.txt Generator

Try for free

Developed by

Jakub Kopecký

The /llms.txt Generator 🕸️📄 extracts website content to create an llms.txt file for AI apps 🤖✨ like LLM fine-tuning and indexing. Output is available 📥 in the Key-Value Store for easy download and integration into workflows. 🚀

2.7 (2)

Pricing

Pay per usage

Total users

191

Monthly users

Runs succeeded

90%

Issue response

1.8 days

Last modified

2 months ago

Automation

Open source

You can access the /llms.txt Generator programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

$echo '{
<  "startUrl": "https://docs.apify.com/cli/docs",
<  "crawlerType": "playwright:adaptive"
<}' |
<apify call jakub.kopecky/llmstxt-generator --silent --output-dataset

/llms.txt Generator API through CLI

The Apify CLI is the official tool that allows you to use /llms.txt Generator locally, providing convenience functions and automatic retries on errors.

Install the Apify CLI

$npm i -g apify-cli
$apify login

Other API clients include:

/llms.txt Generator API in Python

/llms.txt Generator API in JavaScript

/llms.txt Generator OpenAPI definition

/llms.txt Generator API

Real-time knowledge for LLMs

ai-developer/real-time-knowledge-for-llms

Unlock the Full Potential of Your LLM with Real-Time Web Knowledge! Say goodbye to outdated responses, misinformation, and hallucinations. Now you can ground your Language Model with the freshest information from the web. NO API NEEDED!

aideveloper

Extract-any-webpage-content-for-llm

ai-developer/extract-any-webpage-content-for-llm

Fast and easy way to extract data from any webpage and are LLM friendly. The tool lets you easily extract content from any website. Ideal for researchers, marketers, and developers.

aideveloper

439

Nba Teams Info

pintostudio/nba-teams-info

This Actor Retrieves NBA Information on any team.

Pinto Studio

Playwright MCP

jiri.spilka/playwright-mcp

A Model Context Protocol (MCP) server that provides browser automation capabilities using Playwright

Jiří Spilka

Airbnb Properties Actor

getdataforme/airbnb-properties-actor

Airbnb Properties Actor is the web scraper that helps to scraper all kinds of properties and rental listed on Airbnb. We have created a filters to help easily filter and get the result based on types of properties and rentals listed on the Airbnb platform. Easy use, tested and verified with support

GetDataForMe

Browserbase MCP

jakub.kopecky/browserbase-mcp

A Model Context Protocol (MCP) server that provides browser automation capabilities using Browserbase.

Jakub Kopecký

Website to MarkDown (AI-Ready)

mintii/website-to-markdown-ai-ready

Use this to scrape webpages and use for AI Tools and LLMs.

Martin from Mintii

Pinecone Integration

apify/pinecone-integration

This integration transfers data from Apify Actors to a Pinecone and is a good starting point for a question-answering, search, or RAG use case.

Apify

359

ESPN NBA Scraper (Current Season Stats)

scraped/espn-nba-scraper-current-season-stats

This actor provides NBA player statistics sourced from ESPN, including performance data such as points, rebounds, assists, and more.

scraped

Google Condensed SERP Scraper

quaking_pail/google-condensed-serp-scraper

It condenses search engine results into a single, well-formatted text summary. Ideal for feeding into a language model (LLM), making it perfect for lead generation, research, or quickly finding specific details from searches. This process is quite resource intensive hence a renting model.