# AI Exposure Scanner: Public Secrets, Files & AI-Crawler Policy (`fairquery/ai-exposure-scanner`) Actor

Passive, scored scan of your own domains and public GitHub repos: masked leaked secrets, exposed .env, .git and backup files, which AI crawlers may train on your site, security headers and SPF/DMARC, each with a fix. Authorization required. From $0.039 per domain, $0.0039 per repo.

- **URL**: https://apify.com/fairquery/ai-exposure-scanner.md
- **Developed by:** [Fairquery](https://apify.com/fairquery) (community)
- **Categories:** Developer tools, AI, Business
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $39.00 / 1,000 domain scanneds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## AI Exposure Scanner: Public Secrets, Files & AI-Crawler Policy

**See what your own public footprint exposes to AI systems and the open web, in one passive, scored report.** Point it at domains and GitHub orgs you own: it looks for secrets committed to your public GitHub repos (always masked), well-known sensitive files left on your web server, which AI crawlers your robots.txt lets train on your content, contact data visible on your homepage, missing security headers and missing SPF/DMARC records. Every finding comes with a severity and a suggested fix, plus a 0–100 score and a short summary.

Pricing (pay per event, no start fee): **$0.039 per domain scanned, $0.0039 per public repo scanned, $0.0019 per finding reported**. Example: one domain plus a GitHub org with 15 public repos and 10 findings costs about **$0.12**.

> **Scan only what you own or are allowed to test.** The `authorizedToScan` checkbox is required. Without it the run stops before sending a single request to your targets, and nothing is charged.

### When to use this Actor (LLM / tool selection)

Use it when a person, agent or workflow needs to:

- Check whether **API keys, tokens or passwords were committed** to an organization's public GitHub repos, without ever seeing the full values
- Check whether a website serves **`.env`, `.git/config`, backups, phpinfo or API docs** it probably shouldn't
- See **which AI bots (GPTBot, ClaudeBot, Google-Extended, CCBot, PerplexityBot and others) may train on or answer from** a site, based on robots.txt, `ai.txt`, `llms.txt` and `noai` meta tags
- Get a quick **security-header and SPF/DMARC check** for a domain
- Re-run the same snapshot on a schedule to see whether fixes landed

Do **not** use it for: targets you are not authorized to scan, penetration testing, private repositories, gists, logged-in areas, or a compliance audit. It is a passive snapshot, not a certification.

### What it checks

| Check (`checks` value) | What it does | Example finding |
| --- | --- | --- |
| `secrets` | Lists the org's public repos (newest-pushed first), picks up to 40 likely files per repo (config, env, CI, infra, source) and matches rules for AWS, GitHub, Slack, Google/GCP, Stripe, SendGrid, Twilio, OpenAI, Anthropic, npm, JWTs, private keys, database URLs and generic high-entropy assignments | `AWS Access Key ID in public repo acme/api` with evidence `aws-access-key-id: AK********LE (len 20)` |
| `exposed_files` | One GET per path from a short fixed list: `/.env`, `/.env.backup`, `/.git/config`, `/.git/HEAD`, `/config.json`, `/wp-config.php.bak`, `/backup.zip`, `/backup.sql`, `/.DS_Store`, `/phpinfo.php`, `/server-status`, `/openapi.json`, `/.well-known/openapi.json`, `/swagger.json`, `/api-docs`, plus a `.map` check for up to 8 scripts the homepage itself loads. Reported only when a 200 response also matches the file's content signature | `/.git/config is publicly readable` |
| `ai_policy` | robots.txt rules per AI crawler, `ai.txt`, `llms.txt`, TDM/`noai` meta tags; which bots may **train** vs only **search/answer** | `13 AI training crawler(s) are allowed by robots.txt` |
| `data_exposure` | Email addresses and phone numbers on the homepage (masked), presence of `security.txt` | `1 email address(es) exposed on the homepage` with evidence `f***y@proton.me` |
| `headers` | HSTS, CSP, X-Content-Type-Options, X-Frame-Options, Referrer-Policy on the homepage | `4 security header(s) missing` |
| `email_auth` | SPF and DMARC TXT records | `No DMARC record` |

**Secrets are masked at the source.** Only the rule, file, line and a redacted preview (first and last two characters plus the length) are written. Full values never reach the dataset, the log or `OUTPUT`. Each secret finding carries a `confidence` of `likely_valid`, `likely_test` or `unknown` (from the file path, placeholder words and entropy), and identical values repeated in one repo are collapsed into one finding with a count.

### Input

| Field | Default | What it does |
| --- | --- | --- |
| `authorizedToScan` | `false` | **Required to scan.** Confirms you own the targets or have explicit permission |
| `domains` | | Your domains, e.g. `example.com` (up to 20) |
| `githubOrgs` | | Your GitHub organization or user names; only public repos are read |
| `githubToken` | | Optional, stored as a secret. Raises GitHub's API limit from 60 to 5,000 requests/hour. Sent only to api.github.com, never logged |
| `checks` | all | Any of `secrets`, `exposed_files`, `ai_policy`, `data_exposure`, `headers`, `email_auth` |
| `maxRepos` | 50 (≤ 300) | Public repos scanned per org |
| `includeForks` / `includeArchived` | `false` | Also scan forks / archived repos |
| `minSeverity` | `info` | Findings below it are not written or charged (they still count in the score) |
| `requestDelaySecs` | 2 (1–30) | Minimum gap between requests to the same website host |

#### Example input

```json
{
  "authorizedToScan": true,
  "domains": ["example.com"],
  "githubOrgs": ["your-org"],
  "minSeverity": "low"
}
```

### Output

One dataset row per finding (view **Findings**), `error` rows for anything that could not be checked, and a final `summary` row. The same summary goes to the run's `OUTPUT` record.

#### Finding row (from a real run on our own GitHub org, 2026-10-18)

```json
{
  "type": "finding", "category": "secrets", "kind": "aws-access-key-id", "severity": "low",
  "title": "AWS Access Key ID in public repo fairquery/fairquery-agent-trust-mcp",
  "detail": "A string matching AWS Access Key ID is committed at src/server.ts:64. It looks like a test/placeholder value. The value is shown redacted only. This exact value appears 3 times in the repo.",
  "location": "https://github.com/fairquery/fairquery-agent-trust-mcp/blob/main/src/server.ts#L64",
  "evidence": "aws-access-key-id: AK********LE (len 20)",
  "confidence": "likely_test",
  "recommendation": "Revoke/rotate the credential if real, remove it from the code, purge it from git history (e.g. git filter-repo), and load secrets from environment variables or a secrets manager instead."
}
```

(That value is AWS's documented example key in our own test code, so it is correctly flagged low and `likely_test`.)

#### Run summary (`OUTPUT`)

`status`, `overall_score` (0–100), `grade` (A–F), `category_scores`, `findings_total`, `findings_charged`, `findings_by_severity`, per-domain details (reachability, AI-crawler table per bot, SPF/DMARC), per-org repo counts, `requests`, `events`, `event_charges_usd_at_list_price` and `duration_secs`. The score starts at 100 and subtracts a weight per finding; `likely_test` secrets weigh a quarter.

### Pricing (pay per event)

| Event | Price |
| --- | --- |
| Domain scanned (all selected website and DNS checks for one domain) | $0.039 |
| Public repo scanned | $0.0039 |
| Finding reported (at or above `minSeverity`) | $0.0019 |

No Actor-start fee and no monthly plan. Free: runs without the authorization checkbox, error rows, the summary row and findings below `minSeverity`. Set a maximum cost per run in the run options and the Actor stops before exceeding it.

| Example | Cost |
| --- | --- |
| 1 domain, 6 findings | ≈ $0.05 |
| 1 domain + 15 repos, 10 findings | ≈ $0.12 |
| 1 domain + 50 repos, 25 findings | ≈ $0.28 |

### How it behaves (passive by design)

- **No port scans, logins, exploits, fuzzing or brute forcing.** Only GET requests to public URLs, public GitHub API calls and DNS lookups.
- **robots.txt is respected** for every website request, including the well-known-file checks; disallowed paths are skipped.
- **Each well-known path is requested at most once** per domain. The list above is complete.
- **Honest user agent:** `FairqueryExposureScanner/1.0 (+https://getfairquery.com; fairquery@proton.me)`.
- **Polite rate:** one request at a time per host with a minimum gap; a 403 or 429 stops all further requests to that host for the run.
- Private and internal addresses are refused (SSRF protection with DNS pinning; ports 80 and 443 only).

### Memory / runtime

512 MB is plenty. In our tests one domain took about 45 s, and a 13-repo org about 85 s without a token.

### Limitations

- **A clean report is not proof of safety.** It covers only what was public and checked at scan time. It is not a penetration test, not an audit and not a compliance certification.
- **Secret detection is heuristic.** It can miss secrets in formats it has no rule for, in files outside the 40 candidates per repo, or in git history, and it can flag harmless strings. Verify every finding before acting.
- Public repos only: private repos, gists and (by default) forks and archived repos are not read.
- Only the homepage is checked for headers, contact data and `noai` tags; the site is not crawled.
- Without a `githubToken`, GitHub's 60 requests/hour limit can cut large org scans short (reported as an error row).

### FAQ

**Will it show me the leaked secret?** No. You get the type, file, line and a masked preview. Open the file on GitHub to see it, then rotate it.

**Can I scan a competitor or a client?** Only with their explicit permission. The checkbox records your confirmation; you are responsible for having it.

**Why does a test key lower my score?** It is still a credential-shaped string in public code. Test/placeholder findings weigh a quarter of a real one, and you can raise `minSeverity` to hide them.

**Is this a replacement for GitGuardian or a security platform?** No. Those monitor continuously, including private code. This is a cheap, on-demand snapshot of your public footprint.

### Fairquery family

Other Fairquery tools on the Apify Store. Each one returns structured rows with machine-readable error codes, and error rows are never charged:

- [Email Validator for Agents (MX, Disposable, Typo)](https://apify.com/fairquery/email-validator-mx-disposable): syntax, MX, disposable, role and typo checks without sending mail; $0.79 per 1,000 emails
- [Website Contact & Company Enricher](https://apify.com/fairquery/website-contact-company-enricher): domain in, public business emails, phones, socials and company details out; $3 per 1,000 domains
- [Website Tech, SEO & llms.txt Check for Agents](https://apify.com/fairquery/website-tech-seo-ai-readiness): tech stack, SEO score, AI-crawler rules and llms.txt per site; $1.50 per 1,000 sites
- [Sanctions & Watchlist Screener (OFAC, EU, UK, UN)](https://apify.com/fairquery/sanctions-watchlist-screener): fuzzy name screening against official sanctions lists; $3 per 1,000 names + $0.007 per run
- [LLM Data Guard: PII & Secret Redactor](https://apify.com/fairquery/llm-data-guard-pii-secrets): redact PII and secrets before text reaches an LLM; $1.50 per 1,000 documents + $0.20 per 1,000 findings
- [Dependency Vulnerability Check for Agents](https://apify.com/fairquery/osv-dependency-vulnerability-check): packages and lockfiles against OSV.dev; $0.50 per 1,000 packages + $1.50 per 1,000 vulnerabilities
- [EDGAR Company Facts & Filings for Agents](https://apify.com/fairquery/edgar-company-facts-filings): SEC filings and XBRL facts by ticker, CIK or name; from $1.50 per 1,000 companies
- [Company Jobs Feed: Greenhouse, Lever & Ashby](https://apify.com/fairquery/company-jobs-feed): live jobs from employers' own ATS boards with age and repost signals; $0.99 per 1,000 jobs + $0.0005 per board
- [Competitor Change Tracker: Pricing, Terms, Careers & Changelog](https://apify.com/fairquery/competitor-change-tracker): watch competitors' public pricing, terms, careers and changelog pages for real changes with plain-English summaries and webhook alerts; $0.00049 per page checked + $0.0049 per change
- **MCP server** [`@fairquery/agent-trust-mcp`](https://www.npmjs.com/package/@fairquery/agent-trust-mcp): one MCP server for Claude Desktop, Cursor and VS Code (`npx -y @fairquery/agent-trust-mcp`). It has 7 free tools that need no token, plus paid tools that call the PII, email, sanctions and OSV Actors with your own Apify token.

All Fairquery Actors: https://apify.com/fairquery

### Disclaimer

Fairquery is not affiliated with GitHub, OpenAI, Anthropic, Google, GitGuardian or any other company mentioned. Use the Actor only on targets you own or are authorized to scan.

# Actor input Schema

## `authorizedToScan` (type: `boolean`):

Required to scan. Tick this only if you own the domains and GitHub orgs below, or have explicit permission to scan them. Without it the run stops before sending any request to the targets and nothing is charged. The scan is passive (GET requests and DNS lookups only), but you are responsible for having authorization.

## `domains` (type: `array`):

Your own domains, e.g. example.com. The scanner fetches the homepage, robots.txt/ai.txt/llms.txt/security.txt, a short fixed list of well-known sensitive paths (each once), and SPF/DMARC DNS records.

## `githubOrgs` (type: `array`):

Your own GitHub organization or user names. The scanner reads only public repositories via the public GitHub API and scans candidate files for committed secrets.

## `githubToken` (type: `string`):

A GitHub token (classic or fine-grained, read-only/public access is enough) raises the API rate limit from 60 to 5,000 requests/hour. Stored as a secret. Only sent to api.github.com.

## `checks` (type: `array`):

Which categories to run. Default: all.

## `maxRepos` (type: `integer`):

Public repositories scanned per org (newest-pushed first).

## `includeForks` (type: `boolean`):

Also scan forked repositories (off by default; forks usually mirror upstream code).

## `includeArchived` (type: `boolean`):

Also scan archived repositories (off by default).

## `minSeverity` (type: `string`):

Findings below this are not written to the dataset or charged (they still affect the score).

## `requestDelaySecs` (type: `integer`):

Minimum gap between requests to the same host. A larger robots.txt Crawl-delay wins.

## Actor input object example

```json
{
  "authorizedToScan": false,
  "domains": [
    "getfairquery.com"
  ],
  "githubOrgs": [
    "fairquery"
  ],
  "checks": [
    "secrets",
    "exposed_files",
    "ai_policy",
    "data_exposure",
    "headers",
    "email_auth"
  ],
  "maxRepos": 50,
  "includeForks": false,
  "includeArchived": false,
  "minSeverity": "info",
  "requestDelaySecs": 2
}
```

# Actor output Schema

## `findings` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "getfairquery.com"
    ],
    "githubOrgs": [
        "fairquery"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("fairquery/ai-exposure-scanner").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "domains": ["getfairquery.com"],
    "githubOrgs": ["fairquery"],
}

# Run the Actor and wait for it to finish
run = client.actor("fairquery/ai-exposure-scanner").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "getfairquery.com"
  ],
  "githubOrgs": [
    "fairquery"
  ]
}' |
apify call fairquery/ai-exposure-scanner --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,fairquery/ai-exposure-scanner"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/rhe8g1yc9oNdSfBPj/builds/AzbzMeGDc2kYX710k/openapi.json