Webhooks

Webhooks provide an easy and reliable way to configure the Apify platform to carry out an action when a certain system event occurs. For example, you can use webhooks to start another actor when an actor run finishes or fails.

To define a webhook one needs to select an event that triggers the webhook, from the available system events and provide an action to be executed after the event occurs. When the event occurs, the system executes the action.

Currently, the only available action is to send an HTTP POST request to a URL specified in the webhook.

Events

An event is represented by a type and related data. The type is a string that shows the event's name and its hierarchy in the tree of available events, e.g.: ACTOR.RUN.SUCCEEDED. The data are an Object with variable properties. See documentation of individual events for examples of data.

Currently, webhooks are only available for actor run events, with new types in the pipeline.

Actor run

Actor run events are triggered when an actor run gets created or transitions into a new state. Webhook can be defined for all runs of an actor at its detail page or for a specific actor task at its detail page. In that case, the webhook is invoked only for runs started for that task.

Event types:

  • ACTOR.RUN.CREATED - New actor run has been created.
  • ACTOR.RUN.SUCCEEDED - Actor run finished with status SUCCEEDED
  • ACTOR.RUN.FAILED - Actor run finished with status FAILED
  • ACTOR.RUN.ABORTED - Actor run finished with status ABORTED
  • ACTOR.RUN.TIMED_OUT - Actor run finished with status TIMED-OUT

Event data:

{
    "actorId": "ID of the triggering actor.",
    "actorTaskId": "If task was used, its ID.",
    "actorRunId": "Id of the triggering actor run.",
}

Actions

Currently, the only available action is to send an HTTP POST request to a URL specified in the webhook. New actions will come later.

HTTP request

This action sends an HTTP POST request to the provided URL with a JSON payload. The payload is defined using a payload template, a JSON-like syntax that extends JSON with the use of variables enclosed in doubly curly braces {{variable}}. This enables the payload to be dynamically injected with data at the time when the webhook is triggered.

The response to the POST request must have an HTTP status code in the 2XX range. Otherwise it is considered an error and the request is periodically retried with an exponential back-off: first retry happens after roughly 1 minute, second after 2 minutes, third after 4 minutes etc. After 11 such retries which take around 32 hours, the system gives up and stops retrying the requests.

For safety reasons, the webhook URL should contain a secret token to ensure only Apify can invoke it. To test your endpoint, you can use the Test button in the user interface. The timeout of the webhook HTTP request is 30 seconds. If your endpoint performs a time-consuming operation, you should respond to the request immediately so that it does not time out before Apify receives the response. To ensure that the time-consuming operation is reliably finished, you can internally use a message queue to retry the operation on internal failure. In rare circumstances, the webhook might be invoked more than once, you should design your code to be idempotent to duplicate calls.

Payload template

The payload template is a JSON-like string, whose syntax is extended with the use of variables. This is useful when a custom payload structure is needed, but at the same time dynamic data, that is only known at the time of the webhook's invocation, need to be injected into the payload. Aside from the variables, the string must be a valid JSON.

The variables need to be enclosed in double curly braces and cannot be chosen arbitrarily. A pre-defined list, that can be found below, shows all the currently available variables. Using any other variable than one of the pre-defined will result in a validation error.

The syntax of a variable therefore is: {{oneOfAvailableVariables}}.

Default payload template:
{
    "userId": {{userId}},
    "createdAt": {{createdAt}},
    "eventType": {{eventType}},
    "eventData": {{eventData}},
    "resource": {{resource}}
}
Default payload example:
{
    "userId": "abf6vtB2nvQZ4nJzo",
    "createdAt": "2019-01-09T15:59:56.408Z",
    "eventType": "ACTOR.RUN.SUCCEEDED",
    "eventData": {
        "actorId": "fW4MyDhgwtMLrB987",
        "actorRunId": "uPBN9qaKd2iLs5naZ"
    },
    "resource": {
        "id": "uPBN9qaKd2iLs5naZ",
        "actId": "fW4MyDhgwtMLrB987",
        "userId": "abf6vtB2nvQZ4nJzo",
        "startedAt": "2019-01-09T15:59:40.750Z",
        "finishedAt": "2019-01-09T15:59:56.408Z",
        "status": "SUCCEEDED",
        ...
    }
}

You may have noticed that the eventData and resource properties contain redundant data. This is for backwards compatibility. Feel free to only use eventData or resource in your templates, depending on your use case.

Available variables

Variable Type Description
userId string ID of the user who owns the webhook.
createdAt string ISO string date of the webhook's trigger event.
eventType string Type of the trigger event, see Events.
eventData Object Data associated with the trigger event, see Events.
resource Object The resource that caused the trigger event, see below.
Resource

The resourcevariable represents the triggering system resource. For example when using the ACTOR.RUN.SUCCEEDED event, the resource is the actor run. The variable will be replaced by an Object that one would receive as response from the relevant API at the moment when the webhook is triggered. So for the actor run resource, it would be the response of the GET actor run API endpoint.

Ad hoc webhooks

An ad hoc webhook is a one-time webhook created for a certain actor run when starting the run using Apify API. It's triggered at most once when the given run transitions into the selected state. Ad hoc webhooks can be defined using a URL parameter webhooks added to the API endpoint that starts an actor or actor task:

https://api.apify.com/v2/acts/[ACTOR_ID]/runs?token=[YOUR_API_TOKEN]&webhooks=[AD_HOC_WEBHOOKS]

where AD_HOC_WEBHOOKS is a base64 encoded stringified JSON array of webhook definitions:

[
    {
        "eventTypes": ["ACTOR.RUN.CREATED"],
        "requestUrl": "https://example.com/run-created"
    },
    {
        "eventTypes": ["ACTOR.RUN.SUCCEEDED"],
        "requestUrl": "https://example.com/run-succeeded",
        "payloadTemplate": "{\"hello\": \"world\", \"resource\":{{resource}}}"
    },
]

Creating an ad hoc webhook dynamically

You can also create a webhook dynamically from a code of your actor using Apify.addWebhook() function:

await Apify.addWebhook({
    eventTypes: ['ACTOR.RUN.CREATED'],
    requestUrl: 'https://example.com/run-created'
});

To learn more see Apify SDK documentation.

To ensure that duplicate ad hoc webhooks won't get created in a case of actor restart you can use idempotencyKey parameter. Idempotency key must be unique across all the webhooks of a user so the only one webhook gets created for a given value. You can use for example actor run ID as idempotency key:

await Apify.addWebhook({
    eventTypes: ['ACTOR.RUN.CREATED'],
    requestUrl: 'https://example.com/run-created'
    idempotencyKey: process.env.APIFY_ACTOR_RUN_ID,
});