# Actor Pipeline Runner (Chain Actors in One Run) (`nerolabs/actor-pipeline-runner`) Actor

Runs a chain of Apify Actors in one call, handing each step the dataset the previous step produced. Returns every step's run ID, status, dataset and row count, plus the final dataset. Dry run validates the chain first. Charged per pipeline and per started step. Agent-ready.

- **URL**: https://apify.com/nerolabs/actor-pipeline-runner.md
- **Developed by:** [Adam Pearce](https://apify.com/nerolabs) (community)
- **Categories:** Developer tools, Automation, Agents
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.035 / pipeline run

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Actor Pipeline Runner (Chain Actors in One Run)

Scrape, then clean, then filter, then push to your database. Four Actors, four separate runs, and four places to wire the output of one into the input of the next. Every time.

This Actor does the wiring. Give it a list of Actors in order, and it runs them one after another, handing each step the dataset the step before it produced. One call, one run, one set of results telling you what every step did.

### What it does

- **Runs a chain of Actors in order**, up to 25 steps, each with its own input.
- **Passes the data along automatically.** Step 2 gets step 1's output dataset, step 3 gets step 2's, and so on. You do not paste a dataset ID anywhere.
- **Knows which field to put it in.** Most Actors call it `datasetId`, but not all of them do, so the common ones are built in and any Actor at all can be handled with a one-line `datasetField` override.
- **Stops when something breaks.** By default a step that does not succeed halts the chain, so a broken step never feeds bad data into the next one. Turn that off to push on regardless.
- **Reports every step**: which Actor ran, its status, its run ID and a link to it, the dataset it wrote, and how many rows that dataset held.
- **Dry run first.** Validate the whole chain and see the exact input each step would be started with, before anything runs or costs anything.

### Example

```json
{
  "steps": [
    {
      "actor": "apify/website-content-crawler",
      "label": "Crawl the docs site",
      "input": { "startUrls": [{ "url": "https://example.com/docs" }] }
    },
    {
      "actor": "nerolabs/dataset-filter-transform",
      "label": "Keep pages with real content",
      "input": { "filters": [{ "field": "text", "operator": "lengthGreaterThan", "value": 500 }] }
    },
    {
      "actor": "nerolabs/dataset-to-database",
      "label": "Push into Postgres",
      "input": { "connectionString": "postgres://...", "tableName": "docs", "writeMode": "upsert", "keyFields": ["url"] }
    }
  ]
}
```

The crawler runs, its dataset goes into the filter as `datasetId`, the filter's output dataset goes into the database push, and you get back:

| # | Step | Status | Rows out | Dataset |
|---|---|---|---|---|
| 0 | Crawl the docs site | SUCCEEDED | 214 | `aBc...` |
| 1 | Keep pages with real content | SUCCEEDED | 186 | `dEf...` |
| 2 | Push into Postgres | SUCCEEDED | 1 | `gHi...` |

Schedule that once and the whole chain runs itself.

### What it is good for

- A nightly scrape-clean-load pipeline, on one schedule instead of four chained integrations.
- Giving an AI agent a single call that performs a multi-step data job, rather than a loop it has to write, wait on, and get right.
- Reusing the same cleaning and filtering steps behind several different scrapers.
- Trying a chain out with a dry run before committing to it.

### Pricing, and the part people get wrong

Pay per event, no subscription.

| Event | Price |
|---|---|
| Pipeline run | $0.05, once per pipeline |
| Pipeline step | $0.01 per step that actually started |
| Actor start | $0.00005 per run |

Store discounts apply automatically: Bronze 10% off, Silver 20% off, Gold 30% off every event.

**The important part: each step's own Actor charges you its own usual price, separately.** This Actor charges for the orchestration only. A five-step pipeline costs $0.10 here, plus whatever those five Actors would have cost you if you had run them yourself, which is exactly what you would have paid anyway. Running them by hand is not cheaper, it is just more work.

**What is not charged.** A dry run costs nothing but the run start. A step that could not be started at all, for example because the Actor name is wrong, costs nothing. Steps that never started because an earlier one failed cost nothing.

A step that started and then failed **is** charged, because it really ran and its outcome, including the error, is a real answer.

### How the data gets from one step to the next

Every step is started with its own input, plus one extra field holding the previous step's dataset ID. Which field that is depends on the Actor:

| Actor | Field it receives the dataset in |
|---|---|
| Most Actors | `datasetId` |
| Dataset Cleaner & Exporter | `sourceDatasetId` |
| Dataset Diff & Change Detector | `newDatasetId` |
| Dataset Join & Merge | `leftDatasetId` |
| Anything else | set `datasetField` on that step |

If a step should run purely on its own input and ignore what came before it, set `skipDatasetInjection` to true on that step.

To start the chain from data you already have, set **Starting dataset** and it is handed to the first step the same way.

### Notes worth knowing

- **Start with Dry run on.** It checks every step, shows the exact input each would receive, and costs nothing. It is the default.
- **Set the run timeout above the sum of the steps.** This Actor waits for each step in turn, so its own run has to outlast all of them added together. The default is two hours.
- **A step that writes no dataset breaks the chain quietly.** If a step succeeds but produces no rows, the next step is told so in the warnings and runs on its own input.
- **This Actor uses limited permissions**, which means it can run other limited-permission Actors and read their results, and nothing else on your account. Most Store Actors qualify. An Actor that requires full permissions cannot be run as a step.

### FAQ

**Can I use it with Actors that are not Nero Labs Actors?**
Yes. Any Actor on the Store, and your own private ones. The only built-in knowledge is which input field receives the dataset, and `datasetField` covers everything not on the list above.

**What if a step needs the output of two earlier steps?**
Give that step the second dataset explicitly in its own input, and let the pipeline inject the other one. For example, a Join step receives the previous step's dataset as `leftDatasetId` automatically while you set `rightDatasetId` yourself.

**Does it run steps in parallel?**
No, and deliberately. Each step's input depends on the step before it finishing, so the chain is sequential by design.

**What happens if a step takes too long?**
Each step has a maximum wait, one hour by default, and a step can set its own. If the wait runs out, that step is reported as not having succeeded, and the usual stop-on-failure rule applies.

**Can I see what a step actually produced?**
Yes. Every step row carries its dataset ID and a link to its own run, so you can open any intermediate result, not just the final one.

If this saved you wiring a chain of Actors together by hand, a review on the Store page genuinely helps.

### The toolkit this was built for

- [Dataset Cleaner & Exporter](https://apify.com/nerolabs/dataset-cleaner-exporter): dedupe (exact, normalized or fuzzy), flatten nested JSON, clean emails, phones and URLs, then export CSV or Excel.
- [Dataset Filter & Transform](https://apify.com/nerolabs/dataset-filter-transform): keep the rows you want and reshape the fields (dates, replace, split, hash, 25 ops), sort, dedupe, limit.
- [Dataset Join & Merge](https://apify.com/nerolabs/dataset-join-merge): VLOOKUP-style joins and unions across two datasets, files or Google Sheets on a key field.
- [Dataset Aggregate, Group By & Pivot](https://apify.com/nerolabs/dataset-aggregate-pivot): counts, sums, averages and pivot tables per group.
- [Dataset Diff & Change Detector](https://apify.com/nerolabs/dataset-diff-detector): what was added, removed or changed since last time.
- [Dataset AI Enrich](https://apify.com/nerolabs/dataset-ai-enrich): add LLM-generated columns (classify, extract, summarise) to every row, no API key needed.
- [Dataset Charts & Report](https://apify.com/nerolabs/dataset-charts-report): chart images (PNG, SVG) and a PDF or HTML report from any data.
- [Dataset to Postgres, Supabase & MySQL](https://apify.com/nerolabs/dataset-to-database): write the rows straight into a database table, creating it if needed.
- [Dataset to REST API](https://apify.com/nerolabs/dataset-to-rest-api): send every row to any API as its own request, with templating and auth presets.
- **Actor Pipeline Runner** (this one): chain several of these together in one run, each step fed the previous step's dataset.

A common pipeline: a scraper, then Cleaner, then Filter & Transform, then Join to enrich from a sheet, then Aggregate for the weekly summary, with Diff watching what changed and Charts & Report turning the numbers into the Monday PDF. Pipeline Runner runs that whole chain in one call.

### For AI agents

Pay per event, agent-payable through x402 and MCP, limited permissions, no standby. Input: `steps`, an ordered array of `{actor, input, datasetField?, skipDatasetInjection?, label?, waitSecs?, memoryMbytes?, build?}`; optional `initialDatasetId` to seed the first step; `stopOnFailure` (default true); `dryRun` to validate the chain and get each step's resolved input back without running anything. Each step is started with the previous step's default dataset ID injected into its dataset input field. Returns one dataset item per step with `status`, `runId`, `runUrl`, `datasetId`, `itemCount` and `error`, plus a `PIPELINE_SUMMARY` key-value record carrying `finalDatasetId`. Charged $0.05 per pipeline plus $0.01 per started step; each step's own Actor bills its own events to the caller separately.

# Actor input Schema

## `steps` (type: `array`):

One entry per step, in order. Each entry is {"actor": "username/actor-name", "input": { ... }}. Optional per step: "datasetField" (which input field receives the previous dataset, if it is not the usual one), "skipDatasetInjection" (run this step purely on its own input), "label", "waitSecs", "memoryMbytes" and "build". Up to 25 steps.

## `initialDatasetId` (type: `string`):

A dataset to hand to the FIRST step, as if it were the output of a step before it. Use this to run a pipeline over a scraper's existing results. Leave empty when the first step carries its own input, as in the example above.

## `stopOnFailure` (type: `boolean`):

On: if a step does not succeed, the run stops there and the remaining steps are reported as skipped, so a broken step never feeds bad data forward. Off: every step is attempted, and a step whose predecessor produced nothing runs on its own input.

## `defaultWaitSecs` (type: `integer`):

How long to wait for each step before giving up on it and moving on. A step can override this with its own 'waitSecs'. Make sure this Actor's own run timeout is longer than all the steps added together.

## `dryRun` (type: `boolean`):

On: nothing is started and no step is charged. The output shows each step in order with the exact input it would be started with, including which field receives the previous step's dataset. Turn off to run the pipeline.

## Actor input object example

```json
{
  "steps": [
    {
      "actor": "nerolabs/dataset-filter-transform",
      "label": "Keep the active Pro accounts",
      "input": {
        "data": [
          {
            "email": "ana@example.com",
            "plan": "Pro",
            "mrr": 49,
            "active": true
          },
          {
            "email": "ben@example.com",
            "plan": "Starter",
            "mrr": 19,
            "active": true
          },
          {
            "email": "cara@example.com",
            "plan": "Pro",
            "mrr": 99,
            "active": false
          },
          {
            "email": "dan@example.com",
            "plan": "Pro",
            "mrr": 149,
            "active": true
          }
        ],
        "filters": [
          {
            "field": "plan",
            "operator": "equals",
            "value": "Pro"
          },
          {
            "field": "active",
            "operator": "isTrue"
          }
        ],
        "filterMode": "AND"
      }
    },
    {
      "actor": "nerolabs/dataset-aggregate-pivot",
      "label": "Total the revenue by plan",
      "input": {
        "groupByFields": [
          "plan"
        ],
        "aggregations": [
          {
            "field": "mrr",
            "function": "sum",
            "as": "totalMrr"
          }
        ]
      }
    }
  ],
  "stopOnFailure": true,
  "defaultWaitSecs": 3600,
  "dryRun": true
}
```

# Actor output Schema

## `steps` (type: `string`):

One row per step: which Actor ran, its status, run ID, the dataset it produced and how many rows it held.

## `pipelineSummary` (type: `string`):

Steps run, succeeded and failed, the final dataset ID, and any warnings.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "steps": [
        {
            "actor": "nerolabs/dataset-filter-transform",
            "label": "Keep the active Pro accounts",
            "input": {
                "data": [
                    {
                        "email": "ana@example.com",
                        "plan": "Pro",
                        "mrr": 49,
                        "active": true
                    },
                    {
                        "email": "ben@example.com",
                        "plan": "Starter",
                        "mrr": 19,
                        "active": true
                    },
                    {
                        "email": "cara@example.com",
                        "plan": "Pro",
                        "mrr": 99,
                        "active": false
                    },
                    {
                        "email": "dan@example.com",
                        "plan": "Pro",
                        "mrr": 149,
                        "active": true
                    }
                ],
                "filters": [
                    {
                        "field": "plan",
                        "operator": "equals",
                        "value": "Pro"
                    },
                    {
                        "field": "active",
                        "operator": "isTrue"
                    }
                ],
                "filterMode": "AND"
            }
        },
        {
            "actor": "nerolabs/dataset-aggregate-pivot",
            "label": "Total the revenue by plan",
            "input": {
                "groupByFields": [
                    "plan"
                ],
                "aggregations": [
                    {
                        "field": "mrr",
                        "function": "sum",
                        "as": "totalMrr"
                    }
                ]
            }
        }
    ],
    "dryRun": true
};

// Run the Actor and wait for it to finish
const run = await client.actor("nerolabs/actor-pipeline-runner").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "steps": [
        {
            "actor": "nerolabs/dataset-filter-transform",
            "label": "Keep the active Pro accounts",
            "input": {
                "data": [
                    {
                        "email": "ana@example.com",
                        "plan": "Pro",
                        "mrr": 49,
                        "active": True,
                    },
                    {
                        "email": "ben@example.com",
                        "plan": "Starter",
                        "mrr": 19,
                        "active": True,
                    },
                    {
                        "email": "cara@example.com",
                        "plan": "Pro",
                        "mrr": 99,
                        "active": False,
                    },
                    {
                        "email": "dan@example.com",
                        "plan": "Pro",
                        "mrr": 149,
                        "active": True,
                    },
                ],
                "filters": [
                    {
                        "field": "plan",
                        "operator": "equals",
                        "value": "Pro",
                    },
                    {
                        "field": "active",
                        "operator": "isTrue",
                    },
                ],
                "filterMode": "AND",
            },
        },
        {
            "actor": "nerolabs/dataset-aggregate-pivot",
            "label": "Total the revenue by plan",
            "input": {
                "groupByFields": ["plan"],
                "aggregations": [{
                        "field": "mrr",
                        "function": "sum",
                        "as": "totalMrr",
                    }],
            },
        },
    ],
    "dryRun": True,
}

# Run the Actor and wait for it to finish
run = client.actor("nerolabs/actor-pipeline-runner").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "steps": [
    {
      "actor": "nerolabs/dataset-filter-transform",
      "label": "Keep the active Pro accounts",
      "input": {
        "data": [
          {
            "email": "ana@example.com",
            "plan": "Pro",
            "mrr": 49,
            "active": true
          },
          {
            "email": "ben@example.com",
            "plan": "Starter",
            "mrr": 19,
            "active": true
          },
          {
            "email": "cara@example.com",
            "plan": "Pro",
            "mrr": 99,
            "active": false
          },
          {
            "email": "dan@example.com",
            "plan": "Pro",
            "mrr": 149,
            "active": true
          }
        ],
        "filters": [
          {
            "field": "plan",
            "operator": "equals",
            "value": "Pro"
          },
          {
            "field": "active",
            "operator": "isTrue"
          }
        ],
        "filterMode": "AND"
      }
    },
    {
      "actor": "nerolabs/dataset-aggregate-pivot",
      "label": "Total the revenue by plan",
      "input": {
        "groupByFields": [
          "plan"
        ],
        "aggregations": [
          {
            "field": "mrr",
            "function": "sum",
            "as": "totalMrr"
          }
        ]
      }
    }
  ],
  "dryRun": true
}' |
apify call nerolabs/actor-pipeline-runner --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,nerolabs/actor-pipeline-runner"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/cxiqCuZh7FsQv1itH/builds/KJmaEZ0BwfiFMN2OG/openapi.json