Actor picture

Actor Build Starter

fjvs0283/actor-batch-builder

Run builds for multiple actors in your account simultaneously. This can be useful when many actors in a given project have been updated. For example, the documentation might have been updated in 50 actors. This tool will help you trigger actor builds for all 50 at once.

No credit card required

Author's avatarFrancisco Villarreal
  • Modified
  • Users2
  • Runs235
Actor picture
Actor Build Starter

.editorconfig

root = true

[*]
indent_style = space
indent_size = 4
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true
end_of_line = lf

.eslintrc

{
    "extends": "@apify"
}

.gitignore

# This file tells Git which files shouldn't be added to source control

.idea
node_modules

apify_storage

Dockerfile

# First, specify the base Docker image. You can read more about
# the available images at https://sdk.apify.com/docs/guides/docker-images
# You can also use any other image from Docker Hub.
FROM apify/actor-node:16

# Second, copy just package.json and package-lock.json since it should be
# the only file that affects "npm install" in the next step, to speed up the build
COPY package*.json ./

# Install NPM packages, skip optional and development dependencies to
# keep the image small. Avoid logging too much and print the dependency
# tree for debugging
RUN npm --quiet set progress=false \
 && npm install --only=prod --no-optional \
 && echo "Installed NPM packages:" \
 && (npm list --all || true) \
 && echo "Node.js version:" \
 && node --version \
 && echo "NPM version:" \
 && npm --version

# Next, copy the remaining files and directories with the source code.
# Since we do this after NPM install, quick build will be really fast
# for most source file changes.
COPY . ./

# Optionally, specify how to launch the source code of your actor.
# By default, Apify's base Docker images define the CMD instruction
# that runs the Node.js source code using the command specified
# in the "scripts.start" section of the package.json file.
# In short, the instruction looks something like this:
#
# CMD npm start

INPUT_SCHEMA.json

{
    "title": "PuppeteerCrawler Template",
    "type": "object",
    "schemaVersion": 1,
    "properties": {
        "batchType": {
            "title": "Batch Mode",
            "description": "Choose how to identify the target actors to batch process. You can target actors that contain specific text as part of the actor name, target actors by their id, or target all actors in your account.",
            "type": "string",
            "editor": "select",
            "enum": [
                "actorNameContains",
                "actorIds",
                "all"
            ],
            "enumTitles": [
                "Actor name contains",
                "Actor IDs",
                "All actors"
            ],
            "default": "actorNameContains"
        },
        "actorNameContains": {
            "title": "Actor Name Contains",
            "type": "string",
            "description": "Text included in names of target actors.",
            "editor": "textfield",
            "prefill": "my-awesome"
        },
        "actorIds": {
            "title": "Actor IDs",
            "type": "array",
            "description": "Enter the ids of the target actors.",
            "prefill": [
                "abc123",
                "abc123"
            ],
            "editor": "json"
        }
    }
}

README.md

# Actor Batch Builder

<!-- toc start -->

## Table of contents

- [Introduction](#introduction)
- [Cost of usage](#cost-of-usage)
- [Input](#input)
- [Output](#output)

<!-- toc end -->

## Introduction

This actor provides the capability to run actor builds for multiple actors in your Apify account. This can be useful in cases where several actors have been updated and they need to be re-built to reflect the latest changes. An example would be when updating the documentation for multiple actors in a given project.

The actor calls the actor build process for each target actor via API calls. It then periodically checks for the finished builds to fetch the status and other related data.

## Cost of usage

The actor is very cost-effective to run. It will consume approximately 1-2 compute units for every 15,000 actor builds.

## Input

There are three modes for fetching the target actors to be built. The first is by providing a string for matching all actors that contain the string in their name:

```json
{
    "batchType": "actorNameContains",
    "actorNameContains": "my-actor"
}
```

The second is providing an array of actor IDs:

```json
{
    "batchType": "actorIds",
    "actorIds": [
        "abc",
        "abc",
        "abc"
    ]
}
```

The third option is for building all actors availale in the account:

```json
{
    "batchType": "all"
}
```

Note that in all three scenarios any actors in the account that do not have the required access permissions will be skipped.

## Output

The actor stores the build data along with the actor name and id for each target actor in the default dataset:

```json
[
  {
    "id": "clvyb0YFfI62KRV5F",
    "status": "SUCCEEDED",
    "startedAt": "2021-11-06T22:16:17.531Z",
    "finishedAt": "2021-11-06T22:16:29.816Z",
    "meta": {
      "origin": "API"
    },
    "actorId": "4PSW9rUi7vvhJfbLr",
    "actorName": "my-actor"
  }
]
```

The actor also saves some run statistics in the key-value stores under "STATS":

```json
{
  "total": 16,
  "failed": 0,
  "succeeded": 16,
  "requests": 56
}
```

apify.json

{
	"name": "actor-batch-builder",
	"version": "0.0",
	"buildTag": "latest",
	"env": null,
	"template": "project_empty"
}

main.js

This file is 158 lines long. Only the first 50 are shown. Show all

import Apify from 'apify';

const { utils: { log, sleep } } = Apify;

Apify.main(async () => {

    const client = Apify.newClient();
    const { items: actors } = await client.actors().list();
    const { batchType, actorNameContains, actorIds } = await Apify.getInput();

    let batchTypeDescription = "";
    let targetActors = [];
    let actorCount = 0;
    let totalBuildsFinished = 0;
    let requestsCount = 0;

    let results = {
        "builds": [],
        "stats": {
            "total": 0,
            "failed": 0,
            "succeeded": 0,
            "requests": 0
        }
    };


    async function prepTargetActors() {
        for (const item in actors) {
            if (batchType == "actorNameContains" && actors[item].name.includes(actorNameContains)) {
                targetActors.push(actors[item]);
                batchTypeDescription = "matching part of the actor name.";
            }
            else if (batchType == "actorIds" && actorIds.includes(actors[item].id)) {
                targetActors.push(actors[item]);
                batchTypeDescription = "matching selected actor ids.";
            }
            else if (batchType == "all") {
                targetActors.push(actors[item]);
                batchTypeDescription = "for all actors in the account.";
            }
        }
        log.info(`Starting builds ${batchTypeDescription}`);
        log.info('–––––––––––––––––––––––––––––––––––––––––');
    }


    async function buildActors(targetActors) {
        let actorsInProgress = [];
        for (const item in targetActors) {

package-lock.json

This file is 7435 lines long. Only the first 50 are shown. Show all

{
	"name": "actor-batch-builder",
	"version": "0.0.1",
	"lockfileVersion": 2,
	"requires": true,
	"packages": {
		"": {
			"name": "actor-batch-builder",
			"version": "0.0.1",
			"license": "ISC",
			"dependencies": {
				"apify": "^2.0.7"
			},
			"devDependencies": {
				"@apify/eslint-config": "^0.1.3",
				"eslint": "^7.0.0"
			}
		},
		"node_modules/@apify/consts": {
			"version": "1.4.0",
			"resolved": "https://registry.npmjs.org/@apify/consts/-/consts-1.4.0.tgz",
			"integrity": "sha512-OcysWtfs+NOVlGHdIIDci7iOhr0ZDyCoZcsG/VpfJCBwna+w7AVfxX3TmYdFgrJCuq7KM7y/8BYQ+azJmceALQ=="
		},
		"node_modules/@apify/datastructures": {
			"version": "1.0.1",
			"resolved": "https://registry.npmjs.org/@apify/datastructures/-/datastructures-1.0.1.tgz",
			"integrity": "sha512-AgnrfMjzDph+Te5WGNnIsz3+dJM7v/Sqo82nWwSqca292paRotUhORXr9Ik+d0yurC5LutDAhcvu8VZ8SfANGg=="
		},
		"node_modules/@apify/eslint-config": {
			"version": "0.1.4",
			"resolved": "https://registry.npmjs.org/@apify/eslint-config/-/eslint-config-0.1.4.tgz",
			"integrity": "sha512-sbEpFJk+drdTxRVRoL3Ou0h9pmfu/BAiAxZDH3ANHuF7NoprLV1tQvs3PRu+IsFhxIHihI/6znY19KnPOq1dpA==",
			"dev": true,
			"dependencies": {
				"eslint-config-airbnb": "^18.2.0",
				"eslint-config-airbnb-base": "^14.2.0",
				"eslint-import-resolver-typescript": "^2.2.1",
				"eslint-plugin-import": "^2.22.0",
				"eslint-plugin-jsx-a11y": "^6.2.3",
				"eslint-plugin-promise": "^4.2.1",
				"eslint-plugin-react": "^7.20.0",
				"eslint-plugin-react-hooks": "^4.1.0"
			},
			"peerDependencies": {
				"eslint": "*"
			}
		},
		"node_modules/@apify/eslint-config/node_modules/eslint-import-resolver-typescript": {
			"version": "2.5.0",
			"resolved": "https://registry.npmjs.org/eslint-import-resolver-typescript/-/eslint-import-resolver-typescript-2.5.0.tgz",

package.json

{
	"name": "actor-batch-builder",
	"version": "0.0.1",
	"type": "module",
	"description": "This is a boilerplate of an Apify actor.",
	"dependencies": {
		"apify": "^2.0.7"
	},
	"devDependencies": {
		"@apify/eslint-config": "^0.1.3",
		"eslint": "^7.0.0"
	},
	"scripts": {
		"start": "node main.js",
		"lint": "./node_modules/.bin/eslint ./src --ext .js,.jsx",
		"lint:fix": "./node_modules/.bin/eslint ./src --ext .js,.jsx --fix",
		"test": "echo \"Error: oops, the actor has no tests yet, sad!\" && exit 1"
	},
	"author": "It's not you it's me",
	"license": "ISC"
}