AI Tools Directory Scraper

Extract structured data from the world's largest AI tools directories — Futurepedia and Toolify.ai. Get tool names, descriptions, pricing models, categories, ratings, feature lists, pros/cons, and social links in a clean, spreadsheet-ready format for competitive intelligence and market research.

What does the AI Tools Directory Scraper do?

The AI Tools Directory Scraper crawls Futurepedia.io and Toolify.ai — two of the largest directories of AI-powered tools — and extracts structured data from every listing.

For each AI tool you can extract:

• 📝 Name, description, and full long-form description
• 🏷️ Category and all associated tags
• 💰 Pricing model (free, freemium, paid)
• ⭐ User ratings and review counts
• 🌐 Tool website URL
• 🔗 Directory listing URL
• 📋 Key features list, pros, and cons (with detail mode)
• 🔗 Social media links (Twitter, LinkedIn, GitHub, YouTube, Discord)
• 📅 Launch/added date

Browse by category (e.g., chatbots, image generators, code assistants), run a keyword search, or filter by pricing model. No API key or login required.

Who is the AI Tools Directory Scraper for?

🏢 Product managers and startup founders Conducting competitive landscape analysis. Monitor which AI tools compete in your space, track their pricing changes, and identify whitespace opportunities by analyzing feature gaps across hundreds of competitors at once.

📊 VC analysts and investors Building deal flow databases and market maps. Quickly export the full landscape of AI tools in any vertical — chatbots, image generation, sales automation — to spot trends and emerging players before they appear in press coverage.

🤖 AI newsletter writers and curators Publishing roundups of the latest AI tools. Automate your "Top 10 AI tools for X" content by scraping fresh tool listings weekly, filtered by category and sorted by rating.

📈 Marketers and growth teams Tracking competitor launches, pricing movements, and feature rollouts. Set up scheduled runs to get weekly snapshots of a category and detect changes over time.

🔬 Researchers and analysts Building datasets for academic or commercial research on AI tool adoption, pricing trends, and market saturation across categories.

Why use this scraper?

• ✅ No API key needed — Futurepedia and Toolify don't have public APIs; this scraper does the work
• ✅ Cloudflare-bypassing Playwright — both sites use Cloudflare protection; our actor handles it reliably
• ✅ Structured, flat output — every field is a primitive value, ready for Google Sheets or Excel export
• ✅ Category + search + pricing filters — target exactly the tools you care about
• ✅ Detail enrichment mode — optionally visit each tool page for features, pros/cons, and social links
• ✅ Pagination handled automatically — scrapes beyond the first page to reach your target count
• ✅ Deduplicated results — no duplicate tools even when running across multiple categories

What data can you extract from AI tools directories?

Field	Description	Example
name	Tool display name	"ChatGPT"
source	Directory source	"futurepedia"
description	Short description	"Research, create, and automate tasks..."
fullDescription	Long description (detail mode)	Full paragraph about the tool
category	Primary category	"Ai Chatbots"
categories	All tags/categories	["Ai Chatbots", "Research", "Ai Agents"]
pricingModel	Pricing type	"freemium" / "free" / "paid" / "unknown"
pricingDetails	Raw pricing text	"Freemium" / "Contact for Pricing"
url	Tool website URL	"https://chatgpt.com/"
directoryUrl	Listing page URL	"https://www.futurepedia.io/tool/chatgpt"
rating	Average user rating (0-5)	4.3
reviewCount	Number of ratings	9
monthlyVisitors	Monthly visits (Toolify)	6303
features	Key features list (detail mode)	["GPT-5 Integration: ...", "Agent Mode: ..."]
pros	Pros list (detail mode)	["Enhanced intelligence: ...", ...]
cons	Cons list (detail mode)	["Occasional inaccuracies", ...]
logoUrl	Tool logo image URL	CDN URL
launchDate	Date added to directory	"2024-01-15"
socialLinks	Social media URLs (detail mode)	{twitter, linkedin, github, ...}
isVerified	Verified badge	true / false
isFeatured	Featured/Editor's Pick status	true / false
scrapedAt	Extraction timestamp	"2026-04-05T10:30:00Z"

How much does it cost to scrape AI tool directories?

This actor uses Pay-Per-Event (PPE) pricing — you pay per tool scraped, not per compute minute.

Volume	Cost
1–100 tools	$0.003/tool (max $0.30)
100–1,000 tools	$0.0027/tool
1,000–5,000 tools	$0.0024/tool
5,000+ tools (Enterprise)	$0.0012/tool

Plus a one-time $0.03 start fee per run.

Real-world cost examples:

• Scrape 100 chatbot tools (listing mode): ~$0.33
• Scrape 100 chatbot tools (detail mode, with features/pros/cons): ~$0.33 + extra Playwright time
• Export all image generation tools (~200): ~$0.63
• Full Futurepedia chatbots category (~200 tools): ~$0.63

Free plan estimate: Apify's free plan includes $5 in monthly credits. At $0.003/tool, you can scrape ~1,600 AI tools per month on the free plan (listing mode).

How to scrape AI tools from Futurepedia

1. Go to the AI Tools Directory Scraper page on Apify Store
2. Click Try for free
3. In Source directory, select "Futurepedia.io" (default)
4. Optionally set a Category filter (e.g., "chatbots", "image-generators")
5. Set Max tools to scrape (start with 20 for a test run)
6. Toggle Scrape detail pages on to get features, pros/cons, and social links (slower but richer)
7. Click Save & Run
8. When the run completes, download results as JSON, CSV, or Excel

Example inputs for different use cases:

// Scrape chatbot category (fast listing mode)
{
  "source": "futurepedia",
  "category": "chatbots",
  "maxTools": 100,
  "includeDetails": false
}

// Keyword search with detail enrichment
{
  "source": "futurepedia",
  "searchQuery": "customer support automation",
  "maxTools": 50,
  "includeDetails": true
}

// Free tools only from image generation
{
  "source": "futurepedia",
  "category": "image-generators",
  "maxTools": 200,
  "pricingFilter": "free",
  "includeDetails": false
}

Input parameters

Parameter	Type	Default	Description
source	Select	"futurepedia"	Source directory to scrape. Options: "futurepedia", "toolify", "all"
category	String	—	Filter by category slug (e.g., "chatbots", "image-generators", "code-assistant")
searchQuery	String	—	Keyword search for tools by name or description
maxTools	Integer	100	Maximum number of tools to extract
includeDetails	Boolean	true	Visit each tool's detail page for features, pros/cons, and social links
pricingFilter	Select	"all"	Filter by pricing model: "all", "free", "freemium", "paid"
proxyConfiguration	Object	Residential US	Proxy configuration. Residential proxies recommended for Cloudflare bypass

Futurepedia category slugs (use in category field):

• chatbots — AI chatbots and virtual assistants
• image-generators — AI image creation tools
• text-generators — AI writing assistants
• code-assistant — AI coding tools
• video — AI video generation and editing
• productivity — Productivity and workflow tools
• research-assistant — Research and analysis tools
• ai-agents — Autonomous AI agent platforms

Output examples

Listing mode output (fast, no detail pages):

{
  "name": "ChatGPT",
  "source": "futurepedia",
  "description": "Research, create, and automate tasks with the leader in AI.",
  "category": "Ai Chatbots",
  "categories": ["Ai Chatbots", "Research", "Ai Agents"],
  "pricingModel": "freemium",
  "pricingDetails": "Freemium",
  "url": "https://chatgpt.com/",
  "directoryUrl": "https://www.futurepedia.io/tool/chatgpt",
  "rating": 4.3,
  "reviewCount": 9,
  "logoUrl": "https://cdn2.futurepedia.io/...",
  "launchDate": null,
  "isVerified": false,
  "isFeatured": true,
  "scrapedAt": "2026-04-05T10:30:00.000Z"
}

Detail mode output (includes features, pros/cons, social links):

{
  "name": "ChatGPT",
  "source": "futurepedia",
  "description": "Research, create, and automate tasks with the leader in AI.",
  "fullDescription": "ChatGPT is a conversational AI platform that combines advanced language capabilities...",
  "category": "Ai Chatbots",
  "categories": ["Ai Chatbots", "Research", "Ai Agents"],
  "pricingModel": "freemium",
  "url": "https://chatgpt.com/",
  "directoryUrl": "https://www.futurepedia.io/tool/chatgpt",
  "rating": 4.3,
  "reviewCount": 9,
  "features": [
    "GPT-5 Integration: Deeper reasoning capabilities and improved performance...",
    "Agent Mode: Executes multi-step tasks in a virtual environment...",
    "Deep Research: Conducts autonomous online investigations..."
  ],
  "pros": [
    "Enhanced intelligence: GPT-5 improves reasoning and adaptability.",
    "Extensive functionality: Combines conversational AI with research and automation."
  ],
  "cons": [
    "Occasional inaccuracies: Output may still require fact-checking.",
    "Usage limits: Higher-tier tools are capped by subscription plan."
  ],
  "socialLinks": {
    "twitter": "https://x.com/OpenAI",
    "linkedin": "https://www.linkedin.com/company/openai",
    "github": "https://github.com/openai",
    "youtube": "https://www.youtube.com/OpenAI",
    "discord": "https://discord.gg/openai"
  },
  "isVerified": false,
  "isFeatured": true,
  "scrapedAt": "2026-04-05T10:30:00.000Z"
}

Tips for best results

• 🚀 Start small: Run with maxTools: 20 first to verify the output format before scaling up
• ⚡ Use listing mode for large exports: includeDetails: false is 3–5x faster and sufficient for competitive overviews
• 💰 Use detail mode for depth: Enable includeDetails: true when you need features, pros/cons, and social links for a smaller set of tools
• 🏷️ Filter by pricing: Set pricingFilter: "freemium" to find tools with free tiers you can test, or "paid" for premium-only tools
• 📅 Schedule weekly runs: Set up a scheduled run to track new tools added to a category each week
• 🔍 Combine category + pricing: Use both filters for precise targeting (e.g., free image generators)
• 📊 Export to Google Sheets: Use Apify's built-in Google Sheets integration for live-updating spreadsheets

Integrations

🗂️ AI Tools Directory Scraper → Google Sheets Run weekly on a category to maintain a live competitive landscape spreadsheet. New tools automatically appear in Sheets via the Apify + Google Sheets integration.

📬 AI Tools Directory Scraper → Slack/Discord Use an Apify webhook + Zapier/Make to post notifications when new high-rated tools (rating > 4.5) appear in your target category. Stay ahead of competitors.

📊 AI Tools Directory Scraper → Airtable Build a CRM for AI tools you're evaluating. Export structured tool data to Airtable, then track your testing notes and integration status alongside the original data.

🔄 Scheduled monitoring Run daily/weekly with pricingFilter: "freemium" to detect when tools shift from free to paid — a common signal that they're scaling revenue.

🤖 AI Tools Directory Scraper → Claude/GPT analysis Export a category to CSV, then feed it to an AI assistant with a prompt like "Analyze this dataset of AI writing tools and identify the top 3 underserved niches based on feature gaps." Great for strategic planning.

Using the Apify API

You can trigger this actor programmatically using the Apify API.

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });

const run = await client.actor('automation-lab/ai-tools-directory-scraper').call({
  source: 'futurepedia',
  category: 'chatbots',
  maxTools: 100,
  includeDetails: false,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Scraped ${items.length} AI tools`);

Python

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("automation-lab/ai-tools-directory-scraper").call(run_input={
    "source": "futurepedia",
    "category": "image-generators",
    "maxTools": 200,
    "includeDetails": False,
    "pricingFilter": "freemium",
})

dataset_items = client.dataset(run["defaultDatasetId"]).list_items().items
print(f"Found {len(dataset_items)} AI tools")

cURL

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~ai-tools-directory-scraper/runs" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_APIFY_TOKEN" \
  -d '{
    "source": "futurepedia",
    "category": "code-assistant",
    "maxTools": 50,
    "includeDetails": true
  }'

Use with AI agents via MCP

AI Tools Directory Scraper is available as a tool for AI assistants that support the Model Context Protocol (MCP).

Add the Apify MCP server to your AI client — this gives you access to all Apify actors, including this one:

Setup for Claude Code

$claude mcp add --transport http apify "https://mcp.apify.com"

Setup for Claude Desktop, Cursor, or VS Code

Add this to your MCP config file:

{
    "mcpServers": {
        "apify": {
            "url": "https://mcp.apify.com"
        }
    }
}

Your AI assistant will use OAuth to authenticate with your Apify account on first use.

Example prompts

Once connected, try asking your AI assistant:

• "Use automation-lab/ai-tools-directory-scraper to get all freemium chatbot tools from Futurepedia with ratings above 4 stars"
• "Scrape the top 50 AI image generation tools from Futurepedia with their features and pros/cons, then summarize the most common pricing models"
• "Get 200 AI productivity tools and export as CSV — I want to find tools with free tiers that compete with Notion AI"

Learn more in the Apify MCP documentation.

Is it legal to scrape AI tools directories?

This actor collects publicly available information from Futurepedia.io and Toolify.ai — the same data any visitor can see in their browser. No authentication is required or bypassed. Futurepedia and Toolify display this data publicly to drive tool discovery.

Web scraping of publicly available data is generally considered legal in most jurisdictions, as confirmed by the hiQ Labs v. LinkedIn (9th Circuit) ruling and similar EU case law. Apify operates under ethical scraping principles.

Best practices we recommend:

• Use scraped data for research, analysis, and product development — not for republishing without attribution
• Respect each tool's own website terms when visiting their URLs
• Don't use this data to spam or cold-contact tool developers

FAQ

How fast does the scraper work? In listing mode (no detail pages), the actor scrapes approximately 12–24 tools per page visit. With Playwright and residential proxies, each page takes 4–8 seconds. You can expect ~100 tools in under 2 minutes in listing mode.

With includeDetails: true, each tool requires an additional page visit (~5–10 seconds), so 100 tools takes roughly 15–25 minutes. For large runs, use listing mode first to identify tools of interest, then re-scrape specific tools with detail mode.

How much does it cost to scrape 1,000 AI tools? At $0.003/tool for the first 100 and $0.0027/tool for the next 900, plus a $0.03 start fee: approximately $2.73 for 1,000 tools in listing mode.

Why do some tools show pricingModel: "unknown"? Some tools list custom pricing like "Contact for Pricing" or "Free Trial". These don't fit the free/freemium/paid model cleanly, so they're classified as "unknown". The raw pricingDetails field always contains the original text from the directory.

Why are results empty or fewer than expected?

• Check that your category slug matches a real Futurepedia category (e.g., chatbots, not Chatbots)
• Some category pages load slowly — try increasing the actor timeout
• If using pricingFilter, the category may have fewer tools matching that filter than maxTools

Does it work for Toolify.ai? Yes, Toolify is supported with source "toolify" or "all". Toolify uses Cloudflare protection similar to Futurepedia. Toolify provides monthly visitor data (monthlyVisitors) that Futurepedia doesn't offer. Note: Toolify's DOM structure is more dynamic and may occasionally require proxy rotation.

Can I scrape both directories at once? Yes — set source: "all" to scrape both Futurepedia and Toolify in a single run. The source field in each output record identifies which directory it came from. Set maxTools to the combined total you want.

Related scrapers

Looking for more data extraction tools? Check out these other automation-lab actors:

• 🔍 Bing Search Scraper — Scrape Bing search results for SERP analysis
• 📋 Product Hunt Scraper — Extract product launches and upvotes from Product Hunt
• 🛍️ AppSumo Scraper — Scrape lifetime software deals and reviews
• 📊 Apify Store Analyzer — Analyze actors and pricing in the Apify Store
• 🏪 Amazon Product Scraper — Extract Amazon product listings and reviews