# Security.txt Checker - Website Security Contacts (`benthepythondev/security-txt-checker`) Actor

Check websites for security.txt disclosure files and extract contacts, policies, encryption keys, expiry dates and raw security metadata.

- **URL**: https://apify.com/benthepythondev/security-txt-checker.md
- **Developed by:** [ben](https://apify.com/benthepythondev) (community)
- **Categories:** Business, Automation, SEO tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.50 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## 🔎 Security.txt Checker - Website Security Contacts

Check websites for security.txt disclosure files and extract contacts, policies, encryption keys, expiry dates and raw security metadata. Export to JSON/CSV/Excel, run on a schedule, call via API, or connect to Make, Zapier or n8n.

### What is the Security.txt Checker?

The Security.txt Checker turns public website standard files into clean structured data. These small files are easy to miss manually, but they are useful for audits, lead qualification, compliance checks, SEO operations, advertising research, and automated monitoring. Instead of opening every domain in a browser, you can provide a list of domains or direct file URLs and receive normalized rows in an Apify dataset.

The actor is intentionally lightweight. It uses direct HTTP, follows redirects, and parses the public text file without launching a browser. That keeps runs fast, inexpensive, and suitable for scheduled checks across many domains.

#### What data does it extract?

- Final source URL and input URL
- Domain and HTTP status code
- Content type
- Parsed fields relevant to this file type
- Raw line or raw text excerpts for auditability
- URLs and email addresses where present
- Structured result rows ready for export
- Source marker for downstream pipelines

### Input

| Field | Type | Description |
| --- | --- | --- |
| `targets` | array | Domains or direct URLs to check. Domains are automatically converted to the standard file path. |
| `maxResults` | integer | Maximum rows to return per target. Useful for large files such as ads.txt or llms.txt. |

#### Example input

```json
{
  "targets": [
    "https://www.google.com/.well-known/security.txt"
  ],
  "maxResults": 25
}
````

### Output

```json
{"contact": ["https://example.com/security", "mailto:security@example.com"], "policy": "https://example.com/vulnerability-disclosure", "expires": "2027-01-01T00:00:00Z", "emails": ["security@example.com"], "source": "security"}
```

### Use cases

- SEO audits: collect machine-readable public metadata from client or competitor domains.
- Lead generation: qualify companies by their public operational, advertising, or contact signals.
- Compliance monitoring: schedule checks and detect missing, changed, or expired files.
- Advertising research: extract authorized seller relationships from publisher domains.
- AI visibility work: monitor llms.txt files and documentation links as AI search practices evolve.
- Data enrichment: add public standards metadata to domain lists, CRM accounts, and prospect databases.

### Practical tips

For domain lists, pass bare domains such as `example.com` and the actor will request the standard path automatically. For custom locations, pass the full URL. Keep `maxResults` small for quick health checks and larger for full exports. When monitoring a portfolio, schedule the actor daily or weekly and compare rows by `domain`, `source_url`, and file-specific identifiers.

The actor returns raw excerpts because public standards files are not always perfectly formatted. Keeping the original line or text makes it easier to debug unusual publishers, malformed entries, or custom fields that are still valuable to a human analyst.

### Data quality notes

Public website-standard files are simple on paper, but real-world files vary a lot. Some domains add comments, blank lines, custom fields, redirects, casing differences, or extra whitespace. This actor keeps both normalized fields and the original source values so you can trust the output in automated workflows while still having enough context for manual audits.

For files that contain multiple records, such as advertising seller lists or documentation indexes, the actor emits one row per useful record. For files that describe one website-level policy or team file, it emits one normalized row per target. That keeps the dataset natural to join with domain lists, CRM exports, spreadsheet audits, and BI tools.

### Workflow ideas

Start with a list of customer domains, competitor domains, publishers, SaaS companies, app developers, or prospect websites. Run the actor once to create a baseline, then schedule it weekly to detect changes. You can send the output to a webhook, compare it with the previous dataset, and alert your team when a file appears, disappears, expires, changes seller relationships, or adds new contact details.

For agencies, this can become a repeatable audit product: upload a domain list, export the results, and include missing or malformed standards files in the client report. For lead-generation teams, the same data can help prioritize companies that show active advertising operations, security maturity, AI documentation readiness, or public site-maintenance signals.

### Reliability and performance

This actor does not use a browser, paid unblocker, or residential proxy. It is designed for public text files that are meant to be fetched by automated systems. If a target returns no file, the actor logs the failure and continues with the next target. That makes it safe for larger domain lists where some websites are expected to be missing the standard file.

Because the parser is conservative, it avoids pretending malformed files are perfect. Rows include HTTP metadata and raw source values so your workflow can decide whether to accept, clean, or review them.

### FAQ

**Does it require an account?** No. It reads public files from the provided websites.

**Can I pass domains instead of full URLs?** Yes. Bare domains are converted to the correct standard path.

**Can I run it on many websites?** Yes. Add multiple targets and use Apify scheduling for recurring checks.

**Does it crawl the whole website?** No. It fetches only the standard file URL for each target.

**Can I export to Excel?** Yes. Apify datasets export to JSON, CSV, Excel, XML, and RSS.

**Can I connect it to automation tools?** Yes. Use Apify API, webhooks, Make, Zapier, n8n, or dataset exports.

**What happens when a file is missing?** The actor logs the target and continues with the next one.

**Is it legal?** The actor reads public website files. You are responsible for respecting website terms, rate limits, privacy rules, and applicable laws.

**How is billing calculated?** Pay-per-event billing charges per result pushed to the dataset.

**Why include raw text?** Raw excerpts make audits easier and preserve fields that may not be part of the common format yet.

### You might also like

- Website Contact Extractor
- Meta Tags Extractor
- HTTP Headers Checker
- Robots Sitemap Analyzer
- Schema Markup Extractor

**Keywords:** security.txt checker, vulnerability disclosure scraper, security contact extractor, bug bounty contact data, website security audit, web standards scraper, domain audit tool, website metadata extractor, Apify scraper, SEO automation, lead generation data, compliance monitoring, website intelligence

# Actor input Schema

## `targets` (type: `array`):

Domains or direct file URLs to check.

## `maxResults` (type: `integer`):

Maximum rows to return per target.

## Actor input object example

```json
{
  "targets": [
    "https://www.google.com/.well-known/security.txt"
  ],
  "maxResults": 10
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "targets": [
        "https://www.google.com/.well-known/security.txt"
    ],
    "maxResults": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("benthepythondev/security-txt-checker").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "targets": ["https://www.google.com/.well-known/security.txt"],
    "maxResults": 10,
}

# Run the Actor and wait for it to finish
run = client.actor("benthepythondev/security-txt-checker").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "targets": [
    "https://www.google.com/.well-known/security.txt"
  ],
  "maxResults": 10
}' |
apify call benthepythondev/security-txt-checker --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=benthepythondev/security-txt-checker",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "Security.txt Checker - Website Security Contacts",
        "description": "Check websites for security.txt disclosure files and extract contacts, policies, encryption keys, expiry dates and raw security metadata.",
        "version": "1.0",
        "x-build-id": "AKW9o5L2ucHq5bIUN"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/benthepythondev~security-txt-checker/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-benthepythondev-security-txt-checker",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/benthepythondev~security-txt-checker/runs": {
            "post": {
                "operationId": "runs-sync-benthepythondev-security-txt-checker",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/benthepythondev~security-txt-checker/run-sync": {
            "post": {
                "operationId": "run-sync-benthepythondev-security-txt-checker",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "properties": {
                    "targets": {
                        "title": "Targets",
                        "type": "array",
                        "description": "Domains or direct file URLs to check.",
                        "default": [
                            "https://www.google.com/.well-known/security.txt"
                        ],
                        "items": {
                            "type": "string"
                        }
                    },
                    "maxResults": {
                        "title": "Max results per target",
                        "minimum": 1,
                        "maximum": 10000,
                        "type": "integer",
                        "description": "Maximum rows to return per target.",
                        "default": 25
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```