AskUI Integration

AskUI Integration – UI Automation Orchestrator & Task Execution Engine

A powerful integration actor that connects Apify workflows with AskUI to execute UI automation tasks across desktop, browser, and mobile environments. This is not a scraper — it is a UI automation orchestration layer designed for running, managing, and scaling automation workflows using natural language or structured steps.

🚀 Start Automating UI Workflows

Run automation tasks across platforms with full control and visibility.

👉 Execute UI automation using natural language
👉 Run structured task workflows across systems
👉 Collect screenshots, logs, and action traces
👉 Integrate automation into CI/CD and pipelines

Click “Run” to execute your first automation task.

🧠 What This Actor Actually Solves

UI automation is hard to scale because:

• Tools are platform-specific (desktop, web, mobile)
• Automation scripts require maintenance
• No unified orchestration layer
• Difficult to integrate into pipelines

This actor solves that by acting as a central orchestration engine:

• Accepts tasks from Apify
• Sends them to AskUI environments
• Monitors execution
• Returns structured results

🔄 How It Works

1. Define tasks (natural language or structured steps)
2. Actor validates and prepares execution payload
3. Tasks are dispatched to AskUI-compatible environment
4. Execution is monitored and polled
5. Results are normalized and returned

⚙️ Task Definition Modes

🔹 Natural Language Automation

Describe actions like:

• “Open browser and search for product”
• “Login and submit form”

🔹 Structured Steps

Define precise actions (examples):

• click
• type
• navigate
• extract

🔹 Hybrid Mode

Combine both for flexibility and precision.

⚡ Core Features

• AskUI integration for UI automation
• Cross-platform automation (desktop, web, mobile)
• Natural language task execution
• Structured workflow automation
• Batch task processing
• Artifact collection (screenshots, logs, traces)
• Execution monitoring & status tracking
• Normalized output for pipelines

📊 Execution Artifacts

Each task can return:

• Screenshots (UI state capture)
• Execution logs (step-by-step logs)
• Action traces (detailed workflow steps)
• Timing data (execution duration)
• Session metadata

🎯 Real-World Use Cases

• 🧪 QA Automation & Testing: run UI tests across multiple platforms and environments.
• 🤖 RPA (Robotic Process Automation): automate repetitive UI tasks across applications.
• ⚙️ DevOps & CI/CD: integrate UI-level checks into deployment pipelines.
• 🖥️ Cross-Platform Automation: control desktop apps, browsers, and mobile interfaces.
• 📊 Business Workflow Automation: automate data entry, form submissions, and system interactions.

🛠️ How to Use

• Define tasks in input (natural language or steps)
• Choose run mode (dry_run, execute, validate_only)
• Configure AskUI environment (optional)
• Run the actor
• Review results (logs, screenshots, status)

🛠️ How to Use AskUI Integration on Apify (aligned to this implementation)

At a high level:

1. Add one or more automation tasks in tasks (natural language, structured steps, or hybrid).
2. Choose runMode (dry_run, execute, or validate_only).
3. Choose targetEnvironment:
   • adapter_mock (default) for out-of-the-box testing
   • askui_remote to dispatch to a remote AskUI-compatible API (requires credentials)
4. Run the actor and review results in the default dataset (and optionally a combined export via KV store).

Input example

Full schema: INPUT_SCHEMA.json. Example:

{
  "tasks": [
    {
      "taskId": "demo-001",
      "title": "Open calculator and add two numbers",
      "instruction": "Open the calculator app, type 42 + 18, and press equals.",
      "taskMode": "natural_language",
      "platformHint": "windows",
      "applicationHint": "calculator",
      "expectedOutcome": "Calculator displays 60"
    }
  ],
  "runMode": "dry_run",
  "taskDefinitionStyle": "hybrid",
  "targetEnvironment": "adapter_mock",
  "maxConcurrency": 3,
  "outputMode": "dataset"
}

• tasks (required): Array of task objects. Each task must include at least a title or instruction.
• runMode (optional): dry_run (default), execute, or validate_only.
• targetEnvironment (optional): adapter_mock (default), askui_remote, or askui_local_agent (placeholder).
• askUiBaseUrl (required for askui_remote): Base URL to your AskUI-compatible gateway.
• askUiAccessToken (required for askui_remote): Secret token for the remote gateway.
• maxConcurrency (optional): Parallel task processing limit.
• outputMode (optional): dataset, kv_store, or both (default dataset).
• outputKvStoreKey (optional): KV key name for combined output (default ASKUI_OUTPUT).

Output

Results are stored in the Actor’s default dataset. Each item is a compact JSON object and typically includes:

• Identity: taskId, inputTitle
• Modes: runMode, targetEnvironment
• Status: executionStatus, normalizedResult
• Artifacts (optional): artifacts.screenshots, artifacts.logs, artifacts.actionTrace
• Meta: sessionId, durationSecs, timestamp

Example result row (illustrative):

{
  "type": "task_result",
  "taskId": "demo-001",
  "inputTitle": "Open calculator and add two numbers",
  "runMode": "dry_run",
  "targetEnvironment": "adapter_mock",
  "executionStatus": "validated",
  "normalizedResult": "ready",
  "builtPayloadPreview": {
    "taskId": "demo-001",
    "title": "Open calculator and add two numbers",
    "taskMode": "natural_language",
    "platform": "windows"
  },
  "timestamp": "2026-04-29T12:00:00.000Z"
}

🔌 Integration Options

This actor can be used with:

• Apify API
• Webhooks and schedules
• Zapier / Make
• Internal automation systems

🔐 Execution Modes Explained

• dry_run → Validate tasks and preview execution
• execute → Run tasks on AskUI environment
• validate_only → Check task structure only

FAQ

What is AskUI integration used for?

This tool connects AskUI with the Apify platform to enable UI automation workflows, task orchestration, and cross-platform automation.

Is this a web scraping tool?

No. This is a UI automation and RPA orchestration tool, not a scraper.

Can I automate tasks using natural language?

Yes. You can describe tasks in plain English, and the actor will process and execute them using AskUI.

What platforms does this support?

It supports desktop applications, web browsers, and mobile interfaces through AskUI environments.

Can I use this for QA automation testing?

Yes. It is ideal for UI testing, regression testing, and cross-platform QA automation.

Does this work as an RPA tool?

Yes. It can be used for robotic process automation (RPA), including repetitive UI workflows and business automation tasks.

What is dry_run mode?

Dry run validates tasks and builds execution payloads without actually executing them — useful for testing.

Can I run multiple automation tasks at once?

Yes. The actor supports batch execution with configurable concurrency.

What kind of output do I get?

You get structured results including:

• Execution status
• Logs
• Screenshots
• Action traces

Do I need AskUI credentials?

Only if you want to run real executions. Mock mode works without any external setup.

Can I integrate this into CI/CD pipelines?

Yes. This is one of the primary use cases for automation and DevOps workflows.

What makes this different from traditional automation tools?

This actor provides:

• Central orchestration
• Cross-platform support
• Natural language automation
• Structured output for pipelines

📈 Why Use This Actor?

Instead of managing multiple automation tools, this gives you:

• Unified automation layer
• Scalable task execution
• Structured output
• Easy integration into workflows

🚀 Get Started

Run your first UI automation task and streamline your workflows instantly.