# URL Retrieval Preflight (`vincesoft/url-retrieval-preflight`) Actor

Probe a public HTTPS URL and recommend the lowest-cost likely retrieval strategy (plain HTTP, browser render, PDF/feed/JSON/XML parser, authentication required, blocked, or unsupported) with the evidence that produced the recommendation.

- **URL**: https://apify.com/vincesoft/url-retrieval-preflight.md
- **Developed by:** [VinceSoft](https://apify.com/vincesoft) (community)
- **Categories:** Agents, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$0.01 / url preflight

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

### What does URL Retrieval Preflight do?

**URL Retrieval Preflight recommends the lowest-cost likely way to retrieve a public HTTPS URL** and returns the deterministic evidence that produced the decision. It is a small decision primitive for agents and pipelines that must choose between plain HTTP, a browser render, a PDF/feed/JSON/XML parser, or walking away from an authenticated, blocked, or unsupported target. It does not use an LLM and never returns an opaque score.

The Actor starts with a bounded HEAD probe and, only when the headers cannot settle the decision (method not allowed, missing or ambiguous content type, HTML that might depend on JavaScript, or generic XML that might be a feed), performs a GET bounded to the first 64 KiB after decompression. Redirects and transient retries stay within strict request, time, and byte budgets. You can invoke it from the Apify API, scheduled runs, integrations, or Apify MCP and monitor every run through the Apify Console.

### Why use URL Retrieval Preflight?

- Decide *how* to fetch a URL before spending money on the wrong retrieval tool.
- Replace ad-hoc HEAD/GET probing, redirect handling, content-type sniffing, and JS-shell detection with one stable JSON contract.
- Receive the recommendation plus structured evidence: status, content type, redirect history, and the specific markers that fired.
- Keep routing deterministic and reproducible without runtime AI or a paid data supplier.
- Block localhost, private networks, link-local addresses, metadata services, unsafe redirects, oversized responses, and slow requests by default.

Example agent queries include “How should I fetch this URL?”, “Does this page need a browser or plain HTTP?”, “Is this URL a PDF, a feed, or a JSON API?”, and “Check whether this link requires authentication before I crawl it.”

### How to preflight a URL

1. Open the Actor's **Input** tab.
2. Enter one public HTTPS URL.
3. Start the Actor and inspect the single dataset result.

Always ensure your retrieval and use of the target complies with its terms, robots directives, copyright rules, and applicable law.

### Input

See the Input tab for the full schema. The contract deliberately has a single field:

```json
{
  "url": "https://example.com/article"
}
```

### Output

The Actor writes exactly one versioned JSON envelope. You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

```json
{
  "contractVersion": "1.0",
  "ok": true,
  "request": { "url": "https://example.com/article" },
  "probe": {
    "method": "head+get",
    "requestedUrl": "https://example.com/article",
    "finalUrl": "https://example.com/article",
    "httpStatus": 200,
    "contentType": "text/html; charset=utf-8",
    "sniffBytes": 65536,
    "truncated": true,
    "contentHash": "sha256:0000000000000000000000000000000000000000000000000000000000000000",
    "retrievedAt": "2026-08-11T06:00:00.000Z",
    "durationMs": 184,
    "redirectChain": []
  },
  "recommendation": {
    "strategy": "plain_http",
    "evidence": [
      {
        "type": "http_response",
        "source": "https://example.com/article",
        "retrievedAt": "2026-08-11T06:00:00.000Z",
        "tier": "direct",
        "response": {
          "status": 200,
          "contentType": "text/html; charset=utf-8",
          "finalUrl": "https://example.com/article"
        }
      },
      {
        "type": "http_header",
        "source": "https://example.com/article",
        "retrievedAt": "2026-08-11T06:00:00.000Z",
        "tier": "direct",
        "field": "content-type",
        "value": "text/html; charset=utf-8"
      },
      {
        "type": "content_sniff",
        "source": "https://example.com/article",
        "retrievedAt": "2026-08-11T06:00:00.000Z",
        "tier": "direct",
        "field": "readable_text_characters",
        "value": 4312
      }
    ]
  },
  "warnings": []
}
```

| Strategy                  | Triggering evidence (summary)                                                     |
| ------------------------- | --------------------------------------------------------------------------------- |
| `plain_http`              | Readable text is present in the first 64 KiB of the HTML/text response            |
| `browser_render`          | HTML with little readable text plus JS-dependence markers (listed in `evidence`)  |
| `pdf_parser`              | `application/pdf` content type or `%PDF-` body signature                          |
| `feed_parser`             | RSS/Atom/JSON-Feed content type or an RSS/Atom root element                       |
| `json_api`                | JSON content type (including `+json` suffixes) or a valid JSON body               |
| `xml_parser`              | Generic XML inspected without RSS/Atom feed markers                               |
| `authentication_required` | HTTP 401, or 403 with a `www-authenticate` challenge                              |
| `blocked`                 | 404/410, bare 403, 429, or server responses — the URL cannot be retrieved as-is   |
| `unsupported`             | Binary or otherwise unhandled media types (archives, images, audio, video, fonts) |

| Field            | Type    | Meaning                                                                        |
| ---------------- | ------- | ------------------------------------------------------------------------------ |
| `ok`             | boolean | Whether a preflight determination completed                                    |
| `probe`          | object  | Method used, final URL, status, content type, byte count, timestamp, redirects |
| `recommendation` | object  | Strategy plus the evidence entries that produced it                            |
| `error`          | object  | Stable machine-readable free failure when `ok` is false                        |

### How much does URL Retrieval Preflight cost?

The configured price is one **$0.01** `url-preflight` pay-per-event charge for a completed determination — including `blocked` and `unsupported` outcomes, which are valid answers. Network, timeout, security-policy, resource-budget, invalid-input, and internal failures are not charged. A fixed 20-run private Apify sample measured a p95 platform cost of `$0.0001029`against the`$0.004` release ceiling.

### Accuracy and advanced options

The recommendation is only as current as the probe; sites can vary responses by client, geography, or time. `browser_render` is a routing hint derived from explicit structural markers, not a guarantee that rendering will succeed. Query values are used for retrieval but redacted from output and logs.

### FAQ, responsible use, and support

#### Does `plain_http` guarantee the full page is readable?

No. It means the probed prefix contains substantive readable text. A page can still lazy-load sections or paginate content that the 64 KiB sniff does not cover.

#### Why is 404 reported as `blocked` instead of an error?

Because “this URL is not retrievable” is a valid preflight answer. The envelope stays `ok: true`, the strategy is `blocked`, and the HTTP status is included as evidence. Errors are reserved for failures of the Actor itself (DNS, timeouts, redirect-policy failures, and resource limits). If a required GET probe fails, the Actor emits a free structured failure instead of guessing from insufficient HEAD evidence.

The Actor reads public response headers and content prefixes and does not intentionally collect private user data. Use results only with a lawful purpose and consult qualified counsel when unsure. Use the Actor's Issues tab for bug reports and the API tab for programmatic invocation.

# Actor input Schema

## `url` (type: `string`):

Public HTTPS URL to preflight. The Actor sends a HEAD request and, when needed, a GET bounded to the first 64 KiB. Private, local, credential-bearing, and non-HTTPS targets are rejected.

## Actor input object example

```json
{
  "url": "https://example.com/article"
}
```

# Actor output Schema

## `dataset` (type: `string`):

Dataset containing the single preflight envelope.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "url": "https://example.com/"
};

// Run the Actor and wait for it to finish
const run = await client.actor("vincesoft/url-retrieval-preflight").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "url": "https://example.com/" }

# Run the Actor and wait for it to finish
run = client.actor("vincesoft/url-retrieval-preflight").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "url": "https://example.com/"
}' |
apify call vincesoft/url-retrieval-preflight --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,vincesoft/url-retrieval-preflight"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/EfFcbh9i84E7pgR7U/builds/uqNBqtd5tZPgWZohU/openapi.json