# Pipedream Apps Directory Scraper (`automation-lab/pipedream-apps-directory-scraper`) Actor Extract Pipedream app directory metadata, actions, triggers, categories, website links, API docs, and component names for integration research. - **URL**: https://apify.com/automation-lab/pipedream-apps-directory-scraper.md - **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community) - **Categories:** Business - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Pipedream Apps Directory Scraper Scrape Pipedream's public app directory into clean Apify datasets. Use it to collect app names, slugs, descriptions, categories, website URLs, API docs URLs, public app URLs, actions, triggers, and component names from `pipedream.com/apps`. ### What does Pipedream Apps Directory Scraper do? Pipedream Apps Directory Scraper extracts public integration metadata from the Pipedream app directory. It turns app directory pages and public app detail data into structured rows. Each row represents one Pipedream app. The actor is useful when you need a current map of Pipedream's integration ecosystem. It can scrape broad directory samples or a focused search query. It can also target specific Pipedream app URLs. ### Who is it for? Automation consultants use it to compare platform coverage. RevOps teams use it to audit which SaaS tools are supported. SaaS partnership teams use it to find integration gaps. Integration platform vendors use it for competitive intelligence. Growth teams use it to find co-marketing and ecosystem opportunities. Developers use it to inspect available Pipedream actions and triggers before building workflows. ### Why use this actor? ✅ Structured output instead of manual directory browsing. ✅ Public app detail enrichment. ✅ Search-query runs for focused app categories. ✅ Specific app URL runs for repeatable monitoring. ✅ API-friendly datasets for BI, enrichment, and lead workflows. ✅ No Pipedream account required for the public data this actor collects. ### What data can you extract? | Field | Description | | --- | --- | | `appName` | Pipedream app name | | `slug` | URL slug | | `appId` | Pipedream internal app ID | | `appUrl` | Public Pipedream app URL | | `description` | App description | | `status` | Pipedream app status | | `premium` | Whether the app is marked premium | | `category` | Public category when available | | `websiteUrl` | App website URL | | `apiDocsUrl` | API documentation URL | | `logoUrl` | Public logo URL | | `numActions` | Number of actions when exposed | | `triggers` | Trigger names | | `actions` | Action names | | `componentNames` | Combined component names | | `components` | Component detail objects | | `scrapedAt` | ISO scrape timestamp | ### How much does it cost to scrape Pipedream apps? This actor uses pay-per-event pricing. There is a small run-start event of **$0.005 per run** and a per-result event charged for each Pipedream app row saved. The measured BRONZE per-result price is **about $0.000052 per app**. Other Apify plan tiers use automatic discounts: FREE about $0.000060, SILVER about $0.000041, GOLD about $0.000031, PLATINUM about $0.000021, and DIAMOND about $0.000015 per app. You control cost with `maxItems`. A small first run with `maxItems: 25` costs roughly **$0.0063 on BRONZE**: $0.005 start fee plus about $0.0013 for results. A realistic directory intelligence run with `maxItems: 100` costs roughly **$0.0102 on BRONZE**: $0.005 start fee plus about $0.0052 for results. A larger 1,000-app export costs roughly **$0.057 on BRONZE** before Apify platform usage costs: $0.005 start fee plus about $0.052 in result events. The final price users see is calculated by Apify from the live pay-per-event pricing for their account tier. ### How to scrape Pipedream apps 1. Open the actor on Apify. 2. Enter an optional `searchQuery`, such as `crm`, `sales`, or `database`. 3. Set `maxItems` to the number of apps you need. 4. Keep `includeDetails` enabled when you need actions, triggers, categories, docs URLs, and website URLs. 5. Run the actor. 6. Export the dataset as JSON, CSV, Excel, or via API. ### Input options `searchQuery` filters the public Pipedream app directory. `startUrls` accepts app detail URLs such as `https://pipedream.com/apps/notion`. `maxItems` limits the number of app rows saved. `includeDetails` controls detail GraphQL enrichment. `maxComponentsPerApp` limits the number of action/trigger/component names stored per app. ### Example input ```json { "searchQuery": "crm", "maxItems": 100, "includeDetails": true, "maxComponentsPerApp": 50 } ```` ### Specific app URL example ```json { "startUrls": [ { "url": "https://pipedream.com/apps/notion" }, { "url": "https://pipedream.com/apps/slack" } ], "maxItems": 20, "includeDetails": true } ``` ### Output example ```json { "appName": "Notion", "slug": "notion", "appId": "app_X7Lhxr", "appUrl": "https://pipedream.com/apps/notion", "description": "Notion is a new tool that blends your everyday work apps into one.", "category": "Productivity", "websiteUrl": "https://notion.so", "apiDocsUrl": "https://developers.notion.com/reference/intro", "actions": ["Append Block to Parent", "Create Comment"], "triggers": [], "source": "pipedream-graphql", "scrapedAt": "2026-07-07T00:00:00.000Z" } ``` ### Tips for best results Start with a focused query when exploring a niche. Use an empty search query when you need a broad directory snapshot. Keep detail extraction on for competitive intelligence. Lower `maxComponentsPerApp` when you only need high-level coverage. Use specific app URLs for monitoring key integrations. ### Common use cases Build a Pipedream integration coverage database. Compare Pipedream app coverage with Zapier, Make, and n8n. Find apps with rich action coverage. Find categories where your SaaS should build integrations. Create lead lists for partnership outreach. Monitor new or changed app support over time. ### Integration workflows Send output to Google Sheets for RevOps review. Load output into BigQuery or Snowflake for ecosystem analytics. Join app names with CRM account lists. Compare app slugs against internal integration roadmaps. Feed component names into product documentation audits. ### API usage with Node.js ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('automation-lab/pipedream-apps-directory-scraper').call({ searchQuery: 'crm', maxItems: 100, includeDetails: true }); console.log(run.defaultDatasetId); ``` ### API usage with Python ```python from apify_client import ApifyClient client = ApifyClient('MY-APIFY-TOKEN') run = client.actor('automation-lab/pipedream-apps-directory-scraper').call(run_input={ 'searchQuery': 'crm', 'maxItems': 100, 'includeDetails': True, }) print(run['defaultDatasetId']) ``` ### API usage with cURL ```bash curl -X POST 'https://api.apify.com/v2/acts/automation-lab~pipedream-apps-directory-scraper/runs?token=MY-APIFY-TOKEN' \ -H 'Content-Type: application/json' \ -d '{"searchQuery":"crm","maxItems":100,"includeDetails":true}' ``` ### MCP integration Use this actor from Claude Desktop or Claude Code through Apify MCP. MCP URL: `https://mcp.apify.com/?tools=automation-lab/pipedream-apps-directory-scraper` Claude Code quick add command: ```bash claude mcp add apify-pipedream-apps --transport http \ https://mcp.apify.com/?tools=automation-lab/pipedream-apps-directory-scraper ``` Claude Desktop JSON config: ```json { "mcpServers": { "apify-pipedream-apps": { "url": "https://mcp.apify.com/?tools=automation-lab/pipedream-apps-directory-scraper" } } } ``` Example prompt: "Scrape 100 Pipedream CRM apps and summarize which apps have the most available actions." Another prompt: "Compare these SaaS vendors against Pipedream app coverage and flag missing integrations." ### Data quality notes The actor uses Pipedream's public GraphQL responses. Fields may be null when Pipedream does not expose that metadata. Component lists are capped by `maxComponentsPerApp` to keep rows manageable. The source can change without notice because it is a public website and API used by that website. ### Performance notes Directory-only runs are fastest. Detail runs perform one extra public GraphQL request per app. For large exports, increase `maxItems` gradually. The actor is HTTP/API based and does not need a browser. ### Troubleshooting If you get fewer rows than expected, remove the search query and increase `maxItems`. If specific URLs return no rows, check that the app slug exists in the public directory. If component arrays are short, increase `maxComponentsPerApp`. If a website/API docs URL is null, Pipedream did not expose it for that app. ### Legality: is scraping Pipedream legal? This actor collects public directory metadata. You should use the data responsibly. Respect Pipedream's terms, privacy rules, and applicable laws. Do not use output for spam or abusive automation. ### Related scrapers Explore related automation and directory actors from Automation Lab: - https://apify.com/automation-lab/zapier-app-directory-scraper - https://apify.com/automation-lab/n8n-nodes-directory-scraper - https://apify.com/automation-lab/website-contact-finder ### FAQ #### Does this require a Pipedream login? No. The actor targets public directory and public app detail metadata. #### Can it scrape all Pipedream apps? Yes, set a high `maxItems` and leave `searchQuery` empty. #### Can it monitor changes? Yes. Schedule recurring runs and compare datasets by `appId`, `slug`, and component arrays. #### Does it scrape private workflow data? No. It only extracts public app directory metadata. #### Can I filter by category? Use `searchQuery` for category-like terms. Category-specific filtering can be added later if Pipedream exposes a stable public category parameter. ### Changelog Initial version extracts public Pipedream app directory rows with detail enrichment. # Actor input Schema ## `searchQuery` (type: `string`): Optional Pipedream directory search term. Leave empty to scrape the full public app directory. ## `startUrls` (type: `array`): Optional Pipedream app detail URLs. When provided, only matching app slugs are saved after directory lookup. ## `maxItems` (type: `integer`): Maximum number of Pipedream apps to save. ## `includeDetails` (type: `boolean`): Fetch each public app detail record to include categories, website/API docs URLs, actions, triggers, and components. ## `maxComponentsPerApp` (type: `integer`): Maximum trigger/action/component names to store per app record. ## Actor input object example ```json { "searchQuery": "crm", "startUrls": [ { "url": "https://pipedream.com/apps/notion" } ], "maxItems": 20, "includeDetails": true, "maxComponentsPerApp": 20 } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "searchQuery": "crm", "startUrls": [ { "url": "https://pipedream.com/apps/notion" } ], "maxItems": 20, "includeDetails": true, "maxComponentsPerApp": 20 }; // Run the Actor and wait for it to finish const run = await client.actor("automation-lab/pipedream-apps-directory-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "searchQuery": "crm", "startUrls": [{ "url": "https://pipedream.com/apps/notion" }], "maxItems": 20, "includeDetails": True, "maxComponentsPerApp": 20, } # Run the Actor and wait for it to finish run = client.actor("automation-lab/pipedream-apps-directory-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "searchQuery": "crm", "startUrls": [ { "url": "https://pipedream.com/apps/notion" } ], "maxItems": 20, "includeDetails": true, "maxComponentsPerApp": 20 }' | apify call automation-lab/pipedream-apps-directory-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=automation-lab/pipedream-apps-directory-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Pipedream Apps Directory Scraper", "description": "Extract Pipedream app directory metadata, actions, triggers, categories, website links, API docs, and component names for integration research.", "version": "0.1", "x-build-id": "jakovoog0jpDUdsLn" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/automation-lab~pipedream-apps-directory-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-automation-lab-pipedream-apps-directory-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/automation-lab~pipedream-apps-directory-scraper/runs": { "post": { "operationId": "runs-sync-automation-lab-pipedream-apps-directory-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/automation-lab~pipedream-apps-directory-scraper/run-sync": { "post": { "operationId": "run-sync-automation-lab-pipedream-apps-directory-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "searchQuery": { "title": "Search query", "type": "string", "description": "Optional Pipedream directory search term. Leave empty to scrape the full public app directory." }, "startUrls": { "title": "Specific app URLs", "type": "array", "description": "Optional Pipedream app detail URLs. When provided, only matching app slugs are saved after directory lookup.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL", "description": "Pipedream app detail URL, for example https://pipedream.com/apps/notion." } } } }, "maxItems": { "title": "Maximum apps", "minimum": 1, "maximum": 10000, "type": "integer", "description": "Maximum number of Pipedream apps to save.", "default": 20 }, "includeDetails": { "title": "Fetch app details", "type": "boolean", "description": "Fetch each public app detail record to include categories, website/API docs URLs, actions, triggers, and components.", "default": true }, "maxComponentsPerApp": { "title": "Maximum components per app", "minimum": 0, "maximum": 500, "type": "integer", "description": "Maximum trigger/action/component names to store per app record.", "default": 20 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```