# AI Agent Marketplace Scraper (`bornoo/ai-agent-marketplace-scraper`) Actor Discover and scrape AI agent marketplace listings in seconds. Extract agent names, descriptions, categories, pricing, ratings, developers, tags, websites, demo links, GitHub URLs, APIs, images, and more. Export clean structured data to JSON, CSV, Excel, or via API. - **URL**: https://apify.com/bornoo/ai-agent-marketplace-scraper.md - **Developed by:** [Biddut Hossain](https://apify.com/bornoo) (community) - **Categories:** AI, Agents, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.01 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## AI Agent Marketplace Scraper Extract structured data from AI agent marketplaces with ease. This actor crawls marketplace listings and collects detailed information about AI agents, making it ideal for market research, lead generation, competitive analysis, and AI directory creation. ### Features - Scrape AI agent listings - Extract agent names - Collect descriptions - Capture categories - Get developer/company names - Extract pricing information - Collect ratings and reviews - Extract tags - Capture website URLs - Extract demo links - Collect GitHub repositories - Extract documentation URLs - Detect API availability - Detect open-source projects - Capture marketplace URLs - Extract agent images - Export JSON - Export CSV - Access data via API --- ### Input Example: ```json { "startUrls": [ { "url": "https://example.com/marketplace" } ], "maxItems": 100 } ```` #### Input Fields | Field | Type | Description | |--------|------|-------------| | startUrls | Array | Marketplace URLs to scrape | | maxItems | Integer | Maximum number of agents to collect | *** ### Output Example: ```json { "agentName": "AI Research Assistant", "description": "AI-powered research and summarization agent.", "category": "Productivity", "developer": "Example AI", "pricing": "Free", "rating": 4.8, "reviews": 245, "tags": [ "AI", "Research", "Automation" ], "website": "https://example.com", "demoUrl": "https://example.com/demo", "github": "https://github.com/example/project", "documentation": "https://docs.example.com", "apiAvailable": true, "openSource": true, "image": "https://example.com/image.png", "marketplaceUrl": "https://example.com/agent" } ``` *** ### Typical Use Cases - AI market research - Competitor analysis - AI directory creation - Product comparison - Startup research - AI trend analysis - Lead generation - Dataset creation - Business intelligence - Investment research *** ### Export Formats - JSON - CSV - Excel - API *** ### Performance - Fast crawling - Structured output - Duplicate reduction - Reliable extraction - Scalable for large datasets *** ### Requirements - Valid marketplace URLs - Internet access during actor execution *** ### Limitations Some marketplaces may: - Require authentication - Limit requests - Block automated traffic - Load data dynamically Additional customization may be required for certain websites. *** ### Best Practices - Start with a small `maxItems` value for testing. - Verify extracted data before large-scale runs. - Respect website Terms of Service and robots.txt. - Avoid excessive request rates. *** ### Support If you encounter issues or have feature requests, please open an issue or contact the actor author. *** ### License MIT License # Actor input Schema ## `startUrls` (type: `array`): One or more AI agent marketplace URLs to scrape. ## `maxItems` (type: `integer`): Maximum number of AI agents to scrape. ## Actor input object example ```json { "startUrls": [ { "url": "https://example.com" } ], "maxItems": 100 } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://example.com" } ] }; // Run the Actor and wait for it to finish const run = await client.actor("bornoo/ai-agent-marketplace-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://example.com" }] } # Run the Actor and wait for it to finish run = client.actor("bornoo/ai-agent-marketplace-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://example.com" } ] }' | apify call bornoo/ai-agent-marketplace-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=bornoo/ai-agent-marketplace-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "AI Agent Marketplace Scraper", "description": "Discover and scrape AI agent marketplace listings in seconds. Extract agent names, descriptions, categories, pricing, ratings, developers, tags, websites, demo links, GitHub URLs, APIs, images, and more. Export clean structured data to JSON, CSV, Excel, or via API.", "version": "0.0", "x-build-id": "7MlFDTfEQ9mfHfj6i" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/bornoo~ai-agent-marketplace-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-bornoo-ai-agent-marketplace-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/bornoo~ai-agent-marketplace-scraper/runs": { "post": { "operationId": "runs-sync-bornoo-ai-agent-marketplace-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/bornoo~ai-agent-marketplace-scraper/run-sync": { "post": { "operationId": "run-sync-bornoo-ai-agent-marketplace-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "startUrls" ], "properties": { "startUrls": { "title": "Marketplace URLs", "type": "array", "description": "One or more AI agent marketplace URLs to scrape.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "maxItems": { "title": "Maximum Items", "minimum": 1, "type": "integer", "description": "Maximum number of AI agents to scrape.", "default": 100 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```