# Keboola Uploader (`hlidac-shopu/keboola-uploader`) Actor

Reliable uploader of Apify Datasets to Keboola Connection (aka KBC). Integration-ready. Supports development of non-profit service https://www.hlidacshopu.cz/

- **URL**: https://apify.com/hlidac-shopu/keboola-uploader.md
- **Developed by:** [Hlídač shopů](https://apify.com/hlidac-shopu) (community)
- **Categories:** Developer tools, Integrations
- **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Keboola Uploader

Reliable uploader of Apify Datasets to [Keboola Connection](https://www.keboola.com/).
We are using [Storage API Importer](https://developers.keboola.com/integrate/storage/api/importer/) with optimal
defaults.
This actor is helpful in workflows or for ad-hoc data uploads.

This actor is generalisation of our custom-made uploaders for many of our projects. It uses minimum dependencies and
optimizes for speed and reliability.

- gracefully
  handles [migrations](https://docs.apify.com/academy/expert-scraping-with-apify/migrations-maintaining-state?fpr=rarous)
- implements retry policy for failed uploads
- supports [Actor Integration](https://docs.apify.com/platform/integrations/actors/integration-ready-actors?fpr=rarous)
- allows to fine tune the batch size for you optimal usage of resources

Your Apify Dataset will be split into batches, converted to CSV and uploaded with gzip compression enabled.
You should choose the `batchSize` according to the nature of you data. Primitive properties from your Dataset will
be 1:1 mapped to CSV table. Complex properties (arrays and objects) will be serialized to JSON, so you can
use [Snowflake support for JSON](https://docs.snowflake.com/en/user-guide/querying-semistructured) in your transformations.

### Inputs

#### Apify Dataset ID `datasetId`

ID of [Apify Dataset](https://docs.apify.com/platform/storage/dataset?fpr=rarous) that should be uploaded to Keboola.
When you use this actor in Integrations workflow, this field is optional. Default Dataset of previous actor in the flow
will be used.

#### Keboola stack name `keboolaStack`

Hostname of your Keboola stack import endpoint.
See [Keboola documentations](https://developers.keboola.com/overview/api/#stacks-and-endpoints)
for more details. Default is `import.keboola.com` for AWS US-East region.
You can alternatively set `KEBOOLA_STACK` environment variable instead.

Current multi-tenant stacks are:

| region           | hostname                                |
| ---------------- | --------------------------------------- |
| US Virginia AWS  | `import.keboola.com`                    |
| US Virginia GCP  | `import.us-east4.gcp.keboola.com`       |
| EU Frankfurt AWS | `import.eu-central-1.keboola.com`       |
| EU Ireland Azure | `import.north-europe.azure.keboola.com` |
| EU Frankfurt GCP | `import.europe-west3.gcp.keboola.com`   |

If you are a single tenant user then your hostname is in format `import.CUSTOMER_NAME.keboola.com`.

#### Keboola API Key `keboolaStorageApiKey`

Your API Key to a Keboola project where you want to upload the data.
You should generate a new API key just for this actor with limited rights to **write-only** for destination bucket.
You can alternatively set `KEBOOLA_STORAGE_API_KEY` environment variable instead.

#### Keboola bucket `bucket`

Name of the destination Keboola bucket. eg. `in.c-apify`

#### Keboola table `table`

Name of the destination Keboola table. eg. `scrape_results`

#### Keboola table headers `headers`

Array of header names of destination Keboola table.
You can use this to select subset of properties to result table or to reorder the columns - the order of headers is
preserved in result table.
You can leave it blank if your Dataset items have all properties always specified (without `undefined` values).
In this case properties of the first Dataset item are used.
Our recommendation is to be explicit to prevent unexpected data loss.

#### Batch size `batchSize`

Size of the batch to upload. Dataset will be split into more batches if it has more items that this number.
Batches will be uploaded sequentially. Choose the batch size according to the nature of you data and parallelization of
you process.
Generally speaking, Keboola Importer works best if you send less frequent bigger portions (dozens of MB gzipped) of
data.
On the other side you are constrained by the Actor size. You can easily hit OOM condition when this number is too high.

#### Incremental load `incremental`

When enabled, imported data will be added to the existing table.
When disabled, table will be truncated - all existing data will be deleted from the table.
Default is enabled (`true`).

### SDK

You can also use this actor from yours custom actors via `@hckr_/apify-keboola` npm package.

#### Install

```shell
npm install @hckr_/apify-keboola
```

#### Usage

You can use free (pay per usage) variant:

```javascript
import { uploadToKeboola } from "@hckr_/apify-keboola";
```

Or you can support our work on https://www.hlidacshopu.cz/ with the [paid option](https://apify.com/hlidac-shopu/keboola-uploader?fpr=rarous):

```javascript
import { uploadToKeboola } from "@hckr_/apify-keboola/hlidac-shopu";
```

# Actor input Schema

## `datasetId` (type: `string`):

ID of the source Apify Dataset

## `keboolaStack` (type: `string`):

Hostname of the destination Keboola stack

## `keboolaStorageApiKey` (type: `string`):

API Key to Keboola project where you want to upload the data

## `bucket` (type: `string`):

Name of the destination Keboola bucket

## `table` (type: `string`):

Name of the destination Keboola table

## `headers` (type: `array`):

Comma separated headers of destination Keboola table. You can leave it blank if your Dataset items have always all properties specified.

## `batchSize` (type: `integer`):

Size of the batch to upload. Dataset will be split into batches if it has more items.

## `incremental` (type: `boolean`):

If disabled, the target Keboola table is truncated before each import.

## Actor input object example

```json
{
  "keboolaStack": "import.keboola.com",
  "bucket": "in.c-apify",
  "table": "scrape_results",
  "batchSize": 1000,
  "incremental": true
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "keboolaStack": "import.keboola.com",
    "bucket": "in.c-apify",
    "table": "scrape_results",
    "batchSize": 1000
};

// Run the Actor and wait for it to finish
const run = await client.actor("hlidac-shopu/keboola-uploader").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "keboolaStack": "import.keboola.com",
    "bucket": "in.c-apify",
    "table": "scrape_results",
    "batchSize": 1000,
}

# Run the Actor and wait for it to finish
run = client.actor("hlidac-shopu/keboola-uploader").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "keboolaStack": "import.keboola.com",
  "bucket": "in.c-apify",
  "table": "scrape_results",
  "batchSize": 1000
}' |
apify call hlidac-shopu/keboola-uploader --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,hlidac-shopu/keboola-uploader"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/9xd8dNbWK8yVFI5Uz/builds/QBykCKk2b3mcpNJAD/openapi.json