# xAI Grok Bot Marketplace Profiles Scraper (`rainminer/grok-bot-marketplace-scraper`) Actor Extract public Grok Bot templates from the xAI marketplace. Each row includes the name, creator, handle, categories, summary, install count, avatar, skills, routines, and profile link. Built for teams comparing bots before they install one. No login required. Export JSON or CSV for a spreadsheet. - **URL**: https://apify.com/rainminer/grok-bot-marketplace-scraper.md - **Developed by:** [rainminer](https://apify.com/rainminer) (community) - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.99 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## xAI Grok Bot Marketplace Scraper Build a public directory of **Grok Bot** templates from the [xAI bot marketplace](https://x.ai/bot/marketplace). Product, marketing, and research teams use it to see which bots exist, who published them, which category they sit in, and what skills or routines the public profile lists. Paste the marketplace URL, a category URL such as engineering or sales, or a single bot page. The Actor returns one row per bot. No xAI login is required. ### What can this Grok Bot marketplace scraper do? - Read the public marketplace and keep bots in the order the page lists them. - Limit the export with `maxItems` per start URL, so a test stays small. - Open a category URL and return only bots in that category. - Open one bot URL when you already know the profile you want. - Save the creator name, handle, summary, install count, avatar, and profile link. - Save public skills, routines, memories, and integrations from the bot page. - Mark bots that the marketplace features on the home rail. - Run several start URLs in one Actor run. - Schedule a refresh and export JSON, CSV, or Excel through Apify. The marketplace is one public catalog, not a long search index. A default run keeps the first few bots so you can check the columns. Raise the cap when you want a category or the full list. ### What data can you extract? | Field | Meaning | | ------------- | ------------------------------------------------ | | name | Bot name | | creatorName | Public creator name | | handle | Creator handle when the page shows one | | summary | Public bot summary | | categories | Marketplace categories | | installCount | Public install count | | imageUrl | Bot avatar | | url | Bot page on the marketplace | | featured | Whether the bot is on the featured rail | | skills | Public skill name and description | | routines | Public routine name and summary | | memories | Public memory name and description | | integrations | Public integration name and description | | updatedAt | When the public profile was updated | | color / shape | Avatar color and shape | | startUrl | The marketplace or category URL you started from | ### How to scrape the xAI Grok Bot marketplace 1. Open the Actor and keep `https://x.ai/bot/marketplace`, or paste a category or bot URL. 2. Set **Maximum items**. Use 5 for a first look. Raise it to cover the category or the full marketplace. 3. Leave the proxy off unless x.ai blocks the run. 4. Start the run and download the dataset. Category pages use the public path, for example `https://x.ai/bot/marketplace/engineering` or `https://x.ai/bot/marketplace/sales`. A bot can appear in more than one category. `maxItems` applies to each start URL on its own, so the same bot can show up twice if you combine the full marketplace with a category. ### Use cases - Compare public Grok Bot templates before installing one. - Build a creator list with names and handles. - See which bots publish skills, routines, and integrations. - Track install counts by scheduling the same URL. - Export one category, such as engineering or recruiting, into a sheet. - Collect avatars for an internal catalog. - Share a CSV with a team choosing a bot. - Diff summaries after the marketplace updates. - Pull one bot from its marketplace URL. - Review how creators describe memories and routines. - Prepare a workshop handout from the public catalog. - Check that a category page still lists the bots you expect. ### How much does it cost to scrape the Grok Bot marketplace? The public catalog is small enough to export in one run. You pay for Apify platform usage. The default maximum of 5 bots keeps a first run short. Raise `maxItems` only for the slice you need. When the Actor is billed per result, each exported bot is one result. ### Input **Start URLs** accepts the marketplace, a category page, or a bot page on `x.ai`. **Maximum items** caps rows per start URL. **Proxy configuration** stays off unless a run is blocked. ### Output You can download the dataset extracted by this Grok Bot marketplace scraper in JSON, CSV, Excel, or HTML. Nested skills, routines, memories, and integrations stay on the full row. The overview table shows the avatar, name, creator, categories, install count, and link. ```json { "imageUrl": "https://grok-bot-marketplace-public-assets.s3.amazonaws.com/img/creators/8172289b-4cc4-4d64-bed4-783080233ba3.png", "id": "dr-eggbot-v2", "name": "dr eggbot", "creatorName": "Lauren Tan", "handle": "poteto", "summary": "Designs high-quality Grok Bots.", "categories": ["From Grok Bot Team"], "installCount": 0, "featured": true, "url": "https://x.ai/bot/marketplace/bots/dr-eggbot-v2", "skills": [ { "name": "Design a Grok Bot", "description": "Use this when designing or creating a new Grok Bot." } ] } ``` ### FAQ **Does this need an xAI or X login?** No. The public marketplace and bot pages are enough. **Why is the install count zero?** The page publishes a number for every bot. If the marketplace still shows zero, the row keeps that number. **Can I start from one bot?** Yes. Paste `https://x.ai/bot/marketplace/bots/` plus the bot id. **Are private bot settings included?** No. Only the public marketplace profile is saved. ### Is it legal to scrape the xAI bot marketplace? This Actor collects public marketplace listings: names, summaries, and the profile fields the page shows without a login. It does not collect private account data. Personal data is protected by the GDPR in the European Union and by other regulations. Do not scrape personal data unless you have a legitimate reason. If you are unsure, consult your lawyers. ### Image Credit No site screenshot is used. # Actor input Schema ## `startUrls` (type: `array`): Public marketplace, category, or bot URLs on https://x.ai/bot/marketplace. ## `maxItems` (type: `integer`): Maximum number of bots to return per start URL. ## `proxyConfiguration` (type: `object`): Proxy settings. Leave disabled unless x.ai blocks your IP. ## Actor input object example ```json { "startUrls": [ { "url": "https://x.ai/bot/marketplace" } ], "maxItems": 5, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://x.ai/bot/marketplace" } ], "proxyConfiguration": { "useApifyProxy": false } }; // Run the Actor and wait for it to finish const run = await client.actor("rainminer/grok-bot-marketplace-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://x.ai/bot/marketplace" }], "proxyConfiguration": { "useApifyProxy": False }, } # Run the Actor and wait for it to finish run = client.actor("rainminer/grok-bot-marketplace-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://x.ai/bot/marketplace" } ], "proxyConfiguration": { "useApifyProxy": false } }' | apify call rainminer/grok-bot-marketplace-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,rainminer/grok-bot-marketplace-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/BNy7NAS0fLmeOasER/builds/HIHpaZyoHGczFn6Je/openapi.json