# BigQuery Parquet Preflight (`l3digital/bigquery-parquet-preflight`) Actor

Experimental bounded metadata-only BigQuery Parquet preparation receipt; no data-page reads or ingestion guarantee.

- **URL**: https://apify.com/l3digital/bigquery-parquet-preflight.md
- **Developed by:** [L3Digital](https://apify.com/l3digital) (community)
- **Categories:** Developer tools, AI
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$0.05 / useful bigquery preparation receipt

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## BigQuery Parquet Preflight

Identify selected schema differences and load-preparation questions before
loading small Parquet files into BigQuery. Supply one to three authorized inline
files and optional flat target-field intent. The receipt localizes name conflicts,
reports documented default types, flags optional-to-required concerns and compares
ordered schemas across files.

The current public release accepts inline files only. This experimental Actor
does not open caller paths, execute SQL, access a warehouse, read data pages,
convert files or certify successful ingestion. Its commercial advantage over
native tools and official documentation is unvalidated.

### Input

Run the uncharged synthetic example with:

```json
{"demo": true}
```

For real input, provide permission and one to three files:

```json
{
  "processingPermission": true,
  "files": [{"name": "export.parquet", "parquetBase64": "<canonical padded standard base64>"}],
  "targetFields": [
    {"name": "id", "type": "INT64"},
    {"name": "title", "type": "STRING", "mode": "NULLABLE"}
  ]
}
```

The placeholder above is illustrative, not runnable file content. No URL,
filesystem path, GCS selection or load setting is accepted. File names are
unique case-sensitive logical identifiers: an ASCII stem of 1–64 characters,
starting alphanumeric and continuing alphanumeric, underscore or hyphen,
followed by `.parquet`. Base64 must be canonical standard RFC4648 with padding;
whitespace, URL-safe alphabet, data URIs and alternate padding are refused.

Optional `targetFields` contains 1–64 objects with only `name`, `type` and
`mode`. Names contain 1–128 ASCII graphic characters and are unique ASCII
case-insensitively. Types are BOOL, INT64, FLOAT64, BYTES, STRING, DATE or
TIMESTAMP. Modes are NULLABLE or REQUIRED; the default is NULLABLE. There are
no type aliases or coercion. Omit `targetFields` when no intent is supplied.

`demo` and `processingPermission` are actual booleans, default false. Demo
requires `files` and `targetFields` to be absent, including empty or null values,
and permission false or absent. Real input requires permission true and demo
false or absent. Unknown keys are refused before inspection.

### What the receipt checks

Catalog `bq-parquet-1` supports only flat, top-level, nonrepeated scalar fields.
Every rule has explicit completed, skipped or not-applicable coverage.

| Rule | Interpretation |
| --- | --- |
| BQ01 | ASCII case-insensitive top-level name collisions require renaming. |
| BQ02 | A literal period in a top-level name requires renaming; the Parquet character-map setting does not repair it. A nested path separator is not a name finding. |
| BQ03 | Compute documented default destination types and advise on differing matching target intent. A difference does not prove every load option rejects it. |
| BQ04 | OPTIONAL source with REQUIRED target warrants value/load-setting review. Metadata does not prove actual nulls or load rejection. |
| BQ05 | Compare ordered supported schema descriptors across files. Physical, logical, name, order or mode differences warrant review without a universal incompatibility verdict. |

Supported default mappings are BOOLEAN/NONE to BOOL; INT32/INT64 without
annotations to INT64; FLOAT/DOUBLE without annotations to FLOAT64;
BYTE_ARRAY/NONE to BYTES; BYTE_ARRAY/STRING or legacy UTF8 to STRING;
INT32/DATE to DATE; and UTC-adjusted INT64 timestamps in milliseconds or
microseconds, including matching legacy annotations, to TIMESTAMP.

Nested/repeated fields, DECIMAL, unsigned integers, INT96, fixed-length binary,
nanosecond or local timestamps and other annotations are outside this finite
profile. BigQuery supports INT96; exclusion here is not a BigQuery rejection
claim. Conflicting annotations are uninspected. Colliding source names make
target matching ambiguous, so intent/mode comparisons for those fields are skipped
while independently known default types remain available. Unmatched source and
target fields are listed for advisory review.

No values, statistics or min/max are inspected. Serialized Arrow schema metadata
does not redefine conclusions. Load settings, actual GCS/wildcard schema
selection, Unicode case equivalence and successful ingestion remain unchecked.

### Retrieve the full report

After successful delivery, the default dataset contains **one full receipt
object**, including semantic refusal outcomes. The default key-value store's `REPORT` JSON is the
same object. Use the Output tab's **Complete REPORT JSON** link or retrieve
`GET /v2/key-value-stores/{defaultKeyValueStoreId}/records/REPORT` with the
appropriate Apify authorization. The **Receipt dataset** link returns dataset
items. Console tables show a summary; retrieve the full object for descriptors,
localized findings, per-rule coverage, unmatched intent and cited next actions.

Statuses are `complete`, `partial`, `refused` and `failed`. A partial receipt can
contain useful completed checks alongside unsupported fields or uninspectable
files. None of these statuses means ready, valid or importable. Outcomes are
`review_findings`, `no_findings_in_completed_checks` or `not_inspected`; absence
of findings says only that completed finite checks found none.

### Recover and retain this run's evidence

1. Use your authorized Apify caller to [start asynchronously](https://docs.apify.com/api/v2/actors-runs-post). Save the returned run `id` and a caller-generated request key in your own records before proceeding; retain `defaultKeyValueStoreId` and `defaultDatasetId` from the run response.
2. Check `GET /v2/actor-runs/RUN_ID` with that saved ID ([run API](https://docs.apify.com/api/v2/actor-run-get)). If a wait or retrieval times out, check the same run again. A failed or timed-out run can still have stored output; inspect it before deciding what to do.
3. Retrieve the authoritative full receipt with `GET /v2/key-value-stores/STORE_ID/records/REPORT`, using that run's `defaultKeyValueStoreId` as `STORE_ID` ([record API](https://docs.apify.com/api/v2/key-value-store-record-get)). Its equal dataset item is available at `GET /v2/datasets/DATASET_ID/items?format=json`, using `defaultDatasetId` ([dataset items API](https://docs.apify.com/api/v2/dataset-items-get)). Check `REPORT` during failure recovery even if dataset delivery failed.
4. Export the full typed JSON, parse it and verify the input binding and decision-required evidence before acting or cleaning up. Keep `catalogVersion` (`bq-parquet-1`), each file's `name`/`sha256`, descriptors, findings and per-file/per-rule `coverage`. A `partial` receipt includes unsupported or uninspectable portions; even `complete` does not mean ready or importable.

Do not silently POST again after a failed retrieval. If the initial start response was lost and no run ID is known, reconcile your authorized run history against your recorded request and input before considering a paid restart. A missing response does not prove no run started; this workflow promises no universal start idempotency.

A table or optional dataset field projection can support a limited notification, but it is not the full report. Retain the run/storage IDs, the identity and version/profile fields above, and relevant coverage; retrieve omitted detail from this same run before making a detailed decision.

#### Selective cleanup

After the run is terminal, no restart is planned and your exported evidence has been parsed and verified, you may remove authorized copies you no longer need. Use your saved exact storage IDs, never a latest-run shortcut or an assumed shared store. See Apify's [record deletion](https://docs.apify.com/api/v2/key-value-store-record-delete) and [dataset deletion](https://docs.apify.com/api/v2/dataset-delete) references for platform operations, which were not exercised in this Actor's QA: deleting `INPUT` leaves `REPORT`; deleting `REPORT` does not delete the dataset. Input records, REPORT records, dataset items and external copies are separate. Removing one does not erase logs, exports, saved task input, source files or customer copies, or guarantee provider-wide erasure.

### Bounds, pricing and failure behavior

| Bound | Maximum |
| --- | --- |
| Files | 3 |
| Decoded bytes per file / aggregate | 1 MiB (1,048,576) / 3 MiB (3,145,728) |
| Encoded characters per file | 1,398,104 |
| Footer / Thrift string / Thrift container allocation | 65,536 bytes / 8,192 / 4,096 |
| Leaf columns / row groups per file | 64 / 128 |
| Raw source field name returned | 128 characters (chosen profile bound; longer names are uninspected) |
| Findings / serialized UTF-8 REPORT | 512 / 1 MiB |
| Candidate memory / platform timeout / adapter deadline | 512 MiB / 60 seconds / 55 seconds |

These conservative bounds are design choices, not measured performance
guarantees or a native-parser sandbox. Encrypted PARE files are refused.
Findings overflow yields explicit partial coverage with unknown omitted count.
Output overflow yields a fixed failed report and no billable event.

The current price is **$0.05 per useful report**. A useful receipt is non-demo, complete
or partial, with at least one completed supported default-type mapping. Useful
partial receipts may charge. Demo, refused, wholly unsupported/unuseful and failed
reports are uncharged. Fully identical REPORT and dataset copies are persisted
before one eligible `report-produced` event. Storage, deadline or charge failures
fail the run; dataset redirects and application replay after ambiguity are
disabled. SDK-managed charge idempotency remains in place.

Authorized input and output may persist in normal Apify storage. No automatic
deletion, original-input redaction or private-data erasure is promised. Reports
include logical filenames and bounded raw field names, but omit arbitrary
key/value metadata, statistics, source bytes and parser exception messages.

This is an experimental finite receipt, without ingestion certification.
Customer demand and savings over native tools remain unvalidated.

# Actor input Schema

## `demo` (type: `boolean`):

Fixed synthetic clean fixture; always uncharged. Omit files and targetFields entirely, including null/empty values, and leave processingPermission false.

## `processingPermission` (type: `boolean`):

Must be true for real files. Confirm authorization and accept normal Apify retention; false for demo.

## `files` (type: `array`):

Each object has only a unique logical name and canonical padded standard base64. Name is a 1–64 character ASCII alphanumeric/underscore/hyphen stem starting alphanumeric plus .parquet; never a path or GCS selection claim.

## `targetFields` (type: `array`):

Names contain 1–128 ASCII graphic characters and are unique ASCII case-insensitively. Only flat target fields; default mode NULLABLE. Differences are advisory, not universal load rejection. Runtime rejects unknown keys and aliases.

## Actor input object example

```json
{
  "demo": true,
  "processingPermission": false
}
```

# Actor output Schema

## `dataset` (type: `string`):

One receipt object per run, including explicit coverage gaps.

## `report` (type: `string`):

The same object as the dataset receipt.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "demo": true
};

// Run the Actor and wait for it to finish
const run = await client.actor("l3digital/bigquery-parquet-preflight").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "demo": True }

# Run the Actor and wait for it to finish
run = client.actor("l3digital/bigquery-parquet-preflight").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "demo": true
}' |
apify call l3digital/bigquery-parquet-preflight --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,l3digital/bigquery-parquet-preflight"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/OjuAjGzbuftsx8vYa/builds/oIiVaNeHJH3VrxYeZ/openapi.json