# OAI AdsBot Landing Page Auditor (`eternal_algorithms/oai-adsbot-landing-page-auditor`) Actor

Audit public HTTPS landing-page reachability and declared robots policy for OAI-AdsBot, with structured evidence and explicit limitations. Unofficial independent tool; not affiliated with or endorsed by OpenAI.

- **URL**: https://apify.com/eternal\_algorithms/oai-adsbot-landing-page-auditor.md
- **Developed by:** [Eternal Algorithms](https://apify.com/eternal_algorithms) (community)
- **Categories:** Automation, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 audited landing pages

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## OAI AdsBot Landing Page Auditor

Audit public HTTPS landing-page reachability and declared robots policy for OAI-AdsBot. This independent tool is not affiliated with or endorsed by OpenAI.

### What this Actor helps you do

Check whether public landing pages are reachable and whether their declared robots.txt rules allow OAI-AdsBot before you investigate advertising or crawler-delivery problems. Each URL produces one structured evidence row with an explicit limitation statement.

#### Features

- Audits one to fifty unique public HTTPS landing-page URLs per run.
- Evaluates declared robots policy for OAI-AdsBot and optionally OAI-SearchBot.
- Follows bounded, revalidated public HTTPS redirects and blocks private-network targets.
- Returns typed readiness evidence and machine-readable reason codes in the default dataset.

#### Good fits

- Preflight a campaign landing-page list before a human advertising review.
- Find declared robots exclusions or public reachability failures across a small URL batch.
- Feed bounded technical evidence into monitoring, QA, or incident-response workflows.

#### Not a fit

- It does not approve ads, predict ranking, citations, traffic, revenue, or campaign performance.
- It does not impersonate an OpenAI crawler or prove crawler source addresses and WAF allowlists.
- It is not a legal, privacy, accessibility, security, or regulatory compliance certification.

### Tutorial

1. Open the input form and keep the safe example.com URL for your first test run.
2. Replace or extend the URL list with up to fifty public HTTPS landing pages; optionally enable the OAI-SearchBot companion check.
3. Run the Actor with limited permissions and review the Overview table in the default dataset.
4. Use outcome, reason\_codes, HTTP evidence, and the limitation field together; do not treat a ready row as ad approval or crawler-delivery proof.

### Example input

The Actor input form is prefilled with this safe example. API users must still send an input object explicitly.

```
{
  "include_search_bot": false,
  "max_redirects": 2,
  "urls": [
    "https://example.com/"
  ]
}
```

### Representative output

This illustrative row explains the output shape; live values depend on the requested public page and its declared robots policy.

```
{
  "adsbot_matched_rule": "",
  "adsbot_policy": "default_allow",
  "bytes_read": 1256,
  "content_type": "text/html",
  "evidence_summary": "The page was publicly reachable and no explicit OAI-AdsBot robots exclusion was observed.",
  "final_url": "https://example.com/",
  "input_url": "https://example.com/",
  "limitation": "This checks public reachability and declared robots policy; it does not verify crawler source addresses, WAF allowlisting, ad approval, ranking, citation, revenue, or compliance.",
  "outcome": "ready",
  "page_http_status": 200,
  "page_reachable": true,
  "reason_codes": [
    "robots_missing_default_allow",
    "ready_for_oai_adsbot_public_preflight",
    "crawler_network_identity_not_verified"
  ],
  "redirect_count": 0,
  "robots_http_status": 404,
  "robots_url": "https://example.com/robots.txt",
  "searchbot_checked": false,
  "searchbot_matched_rule": "",
  "searchbot_policy": "not_checked"
}
```

### Pricing

Proposed pay-per-event pricing includes platform usage: USD 0.00005 when a run starts plus USD 0.00200 for each validated dataset row. These are draft prices and do not take effect until the exact release, benchmark, and pricing settings receive signed approval.

- One persisted URL row: approximately USD 0.00205.
- Ten persisted URL rows: approximately USD 0.02005.
- Fifty persisted URL rows: approximately USD 0.10005.
- A failed input that produces no row must not incur a dataset-item event charge.

### Data and privacy

Inputs must contain only public HTTPS URLs and never credentials or private customer data. The Actor connects to each supplied public host and its robots.txt endpoint, does not call a model provider, and writes only the documented rows to the default Apify dataset. Apify workspace storage and retention settings continue to apply to input and results.

### Limitations

- Robots.txt is publisher-declared policy and is not proof that a crawler will visit a page.
- Public HTTP evidence can change immediately after the run completes.
- A reachable page may still be blocked by systems or policies this Actor cannot observe.
- The Actor intentionally rejects private, local, credential-bearing, malformed, and non-HTTPS targets.

### Support and maintenance

Report reproducible product issues through the Actor issue channel with a minimal public URL fixture and relevant reason codes. Do not include credentials, private URLs, billing details, or customer records. Routine reports target a two-business-day response; urgent security or source-visibility reports target one business day.

Releases use semantic versions and content-addressed packages. Health, issue age, cost, and quality are reviewed on a governed schedule. Breaking changes require a new reviewed release; planned deprecation preserves the listing, gives public notice, and identifies a replacement when one exists.

### Input reference

- `urls` (array): One to fifty unique public HTTPS landing-page URLs.
- `include_search_bot` (boolean, optional): Also evaluate declared robots policy for OAI-SearchBot.
- `max_redirects` (integer, optional): Maximum zero to five validated redirects per request.

### Dataset output reference

- `adsbot_matched_rule` (string): Longest matched OAI-AdsBot robots rule when present.
- `adsbot_policy` (string): Declared robots decision for OAI-AdsBot.
- `bytes_read` (integer): Landing-page response bytes read within the fixed ceiling.
- `content_type` (string): Bounded landing-page response content type when available.
- `evidence_summary` (string): Concise explanation of observed public evidence.
- `final_url` (string): Final public HTTPS URL after bounded redirects.
- `input_url` (string): Validated landing-page URL supplied by the customer.
- `limitation` (string): Explicit statement of claims this audit cannot verify.
- `outcome` (string): Bounded technical readiness outcome.
- `page_http_status` (integer): Observed landing-page HTTP status, or zero without HTTP evidence.
- `page_reachable` (boolean): Whether the landing page returned a successful public HTTP status.
- `reason_codes` (array): Machine-readable evidence and limitation reason codes.
- `redirect_count` (integer): Number of validated landing-page redirects followed.
- `robots_http_status` (integer): Observed robots HTTP status, or zero without HTTP evidence.
- `robots_url` (string): Robots policy URL evaluated for the final page origin.
- `searchbot_checked` (boolean): Whether the optional OAI-SearchBot companion check ran.
- `searchbot_matched_rule` (string): Longest matched OAI-SearchBot robots rule when present.
- `searchbot_policy` (string): Declared robots decision for OAI-SearchBot or not\_checked.

### Operational boundary

Generated from `CAP-OAI-ADSBOT-LANDING-PAGE-AUDITOR` version `0.4.0`. The Actor fails closed on invalid input and writes only the typed dataset rows described above. No credential is embedded in this package.

# Actor input Schema

## `include_search_bot` (type: `boolean`):

Also evaluate declared robots policy for OAI-SearchBot

## `max_redirects` (type: `integer`):

Maximum zero to five validated redirects per request

## `urls` (type: `array`):

One to fifty unique public HTTPS landing-page URLs

## Actor input object example

```json
{
  "max_redirects": 2,
  "urls": [
    "https://example.com/"
  ]
}
```

# Actor output Schema

## `results` (type: `string`):

Validated rows in the default dataset.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "include_search_bot": false,
    "max_redirects": 2,
    "urls": [
        "https://example.com/"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("eternal_algorithms/oai-adsbot-landing-page-auditor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "include_search_bot": False,
    "max_redirects": 2,
    "urls": ["https://example.com/"],
}

# Run the Actor and wait for it to finish
run = client.actor("eternal_algorithms/oai-adsbot-landing-page-auditor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "include_search_bot": false,
  "max_redirects": 2,
  "urls": [
    "https://example.com/"
  ]
}' |
apify call eternal_algorithms/oai-adsbot-landing-page-auditor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,eternal_algorithms/oai-adsbot-landing-page-auditor"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/ctmbT2xeCfAR6CQsX/builds/AudUIcEguwXWd6pnm/openapi.json