# crt.sh Certificate Transparency Scraper (`parseforge/crtsh-certificate-transparency-scraper`) Actor Search the crt.sh certificate transparency logs for any domain you control and surface the hosts behind it. Each record carries the common name, every subject alternative name, the issuing authority, serial number, and validity window. Built for attack surface mapping and asset inventory. - **URL**: https://apify.com/parseforge/crtsh-certificate-transparency-scraper.md - **Developed by:** [ParseForge](https://apify.com/parseforge) (community) - **Categories:** Developer tools, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $2.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ![ParseForge Banner](https://github.com/ParseForge/apify-assets/blob/ad35ccc13ddd068b9d6cba33f323962e39aed5b2/banner.jpg?raw=true) ## ๐Ÿ” crt.sh Certificate Transparency Scraper > ๐Ÿš€ **Map a domain's certificate footprint in one run.** A single search of `github.com` returns 1,060 certificate log entries, each with its common name, subdomains, issuer, and validity window. > ๐Ÿ•’ **Last updated:** 2026-06-04 ยท **๐Ÿ“Š 12 fields** per record ยท public Certificate Transparency data ยท works for any domain you are authorized to assess Certificate Transparency (CT) logs are public, append-only records of every TLS certificate a Certificate Authority issues. Because a certificate has to name the host it protects, those logs quietly reveal subdomains and internal hostnames that never appear in DNS scans or sitemaps. This actor queries [crt.sh](https://crt.sh), the CT search service operated by Sectigo, and turns the raw log into clean, structured records. Point it at one or more domains you own or are authorized to test, flip on the wildcard option, and you get back every certificate ever logged for that domain and its subdomains. It is built for defensive security work: attack-surface mapping, asset inventory, and watching your own certificates as they are issued. | ๐ŸŽฏ Target Audience | ๐Ÿ’ก Primary Use Cases | |---|---| | Security engineers and blue teams | Attack-surface and subdomain mapping | | IT and infrastructure teams | Asset inventory of your own hostnames | | Domain and brand owners | Monitoring certificates issued for your domains | | Compliance and audit teams | Checking issuers and validity on your estate | | Researchers and analysts | Studying public CT data at scale | ### ๐Ÿ“‹ What the crt.sh Certificate Transparency Scraper does The actor searches the crt.sh JSON endpoint for each domain you provide. With the wildcard option enabled it queries `%.yourdomain.com`, so certificates issued to subdomains are included alongside the apex domain. For every certificate in the log it returns one row containing the crt.sh id, the common name, the full list of Subject Alternative Names (SANs) as an array, the issuing CA, the serial number, and the not before, not after, and log entry timestamps. A direct `crt.sh/?id=` link is added so you can open any certificate in one click. crt.sh can be slow and sometimes answers with a 502 while it is under load, so the actor retries each query with a generous timeout and backoff, and falls back from the wildcard form to the plain domain if needed. ### ๐ŸŽฌ Full Demo (_๐Ÿšง Coming soon_) ### โš™๏ธ Input | Field | Type | Required | Description | |---|---|---|---| | `domains` | array | Yes | One or more domains to search, one per line, for example `github.com`. | | `includeSubdomains` | boolean | No | When on, searches `%.domain` so subdomain certificates are included. Default `true`. | | `maxItems` | integer | No | Cap on rows returned. Free plan is limited to 10. Default `10`. | **Example 1 โ€” discover subdomains for one domain** ```json { "domains": ["github.com"], "includeSubdomains": true, "maxItems": 50 } ```` **Example 2 โ€” apex certificates only, several domains** ```json { "domains": ["example.com", "mozilla.org"], "includeSubdomains": false, "maxItems": 100 } ``` > โš ๏ธ **Good to Know:** crt.sh returns every certificate ever logged, including expired and renewed ones, so popular domains produce thousands of rows and the same hostname can appear many times. Only search domains you own or are authorized to assess. This is a defensive, public-data tool. ### ๐Ÿ“Š Output Each record describes a single certificate from the Certificate Transparency log. | Field | Type | Description | |---|---|---| | ๐Ÿ†” `id` | number | crt.sh internal certificate id. | | ๐Ÿ” `commonName` | string | The certificate subject common name. | | ๐ŸŒ `sans` | array | Subject Alternative Names (hostnames the cert covers). | | ๐Ÿข `issuerName` | string | Distinguished name of the issuing CA. | | ๐Ÿท `issuerCaId` | number | crt.sh id of the issuing CA. | | ๐Ÿ”ข `serialNumber` | string | Certificate serial number in hex. | | ๐Ÿ“… `notBefore` | string | Start of the validity window. | | โณ `notAfter` | string | End of the validity window. | | ๐Ÿ•ฐ `entryTimestamp` | string | When the certificate was added to the CT log. | | ๐Ÿ”— `crtShUrl` | string | Direct link to the certificate on crt.sh. | | ๐ŸŽฏ `queryDomain` | string | The domain you searched that produced this row. | | ๐Ÿ•’ `scrapedAt` | string | ISO timestamp when the row was collected. | | โŒ `error` | string | Error message, present only on failed rows. | **Sample record 1** ```json { "id": 26846556512, "commonName": "spotlights-feed.github.com", "sans": ["spotlights-feed.github.com"], "issuerName": "C=US, O=Let's Encrypt, CN=YR1", "issuerCaId": 432476, "serialNumber": "06d3cfe5620a6a3344e42ea4ae25d092a712", "notBefore": "2026-06-03T03:07:28", "notAfter": "2026-09-01T03:07:27", "entryTimestamp": "2026-06-03T04:05:59.463", "crtShUrl": "https://crt.sh/?id=26846556512", "queryDomain": "github.com", "scrapedAt": "2026-06-04T19:25:18.926Z", "error": null } ``` **Sample record 2** ```json { "id": 26743969214, "commonName": "vpn-ca.iad.github.com", "sans": ["vpn-ca.iad.github.com"], "issuerName": "C=US, O=Let's Encrypt, CN=YR2", "issuerCaId": 432477, "serialNumber": "0593ebe3e6b66952c36f1c3b6aa77c959f5e", "notBefore": "2026-05-29T23:29:05", "notAfter": "2026-08-27T23:29:04", "entryTimestamp": "2026-05-30T00:27:36.059", "crtShUrl": "https://crt.sh/?id=26743969214", "queryDomain": "github.com", "scrapedAt": "2026-06-04T19:25:19.053Z", "error": null } ``` **Sample record 3** ```json { "id": 26846547524, "commonName": "spotlights-feed.github.com", "sans": ["spotlights-feed.github.com"], "issuerName": "C=US, O=Let's Encrypt, CN=YR1", "issuerCaId": 432476, "serialNumber": "06d3cfe5620a6a3344e42ea4ae25d092a712", "notBefore": "2026-06-03T03:07:28", "notAfter": "2026-09-01T03:07:27", "entryTimestamp": "2026-06-03T04:05:59.178", "crtShUrl": "https://crt.sh/?id=26846547524", "queryDomain": "github.com", "scrapedAt": "2026-06-04T19:25:19.005Z", "error": null } ``` ### โœจ Why choose this Actor - Reads straight from crt.sh, the Certificate Transparency search run by Sectigo, so the data is the public log itself. - Splits the SAN field into a clean array, which is where most subdomain discovery actually happens. - Adds a direct `crt.sh/?id=` link to every row for one-click inspection. - Built to ride out crt.sh load spikes with retries, a long timeout, and a wildcard to plain-domain fallback. - No account, no API key, and no proxy needed for the source. ### ๐Ÿ“ˆ How it compares to alternatives | Approach | Subdomain coverage | Setup | Structured output | |---|---|---|---| | This actor | Every host in CT logs | Paste a domain | One clean row per cert | | Manual crt.sh browsing | Same source | Copy and paste by hand | None | | DNS brute forcing | Only names you guess | Wordlists and tooling | Varies | | Passive DNS services | Often paid | Accounts and keys | Varies | ### ๐Ÿš€ How to use 1. **Sign up for Apify** using [this link](https://console.apify.com/sign-up?fpr=vmoqkp) if you do not have an account yet. 2. Open the crt.sh Certificate Transparency Scraper and click **Try for free**. 3. Enter one or more domains you own or are authorized to assess, then choose whether to include subdomains. 4. Set `maxItems` to control how many certificates you pull, and click **Start**. 5. When the run finishes, browse the dataset in the console or pull it from the API for your own tooling. ### ๐Ÿ’ผ Business use cases **Attack-surface mapping** | Goal | How | |---|---| | Find forgotten hosts | Search your apex domain with the wildcard on and review every SAN. | | Spot shadow services | Look for staging, vpn, or admin hostnames in the results. | **Asset inventory** | Goal | How | |---|---| | List your own certificates | Pull all certs for each domain you own and group by issuer. | | Track issuers in use | Read `issuerName` to see which CAs sign your hosts. | **Certificate monitoring** | Goal | How | |---|---| | Watch for new issuance | Re-run on a schedule and diff on `id` to catch new certs. | | Catch expiring certs | Sort by `notAfter` to see what lapses soon. | **Compliance and audit** | Goal | How | |---|---| | Evidence of coverage | Show that hosts in scope carry valid, current certificates. | | Validity checks | Verify `notBefore` and `notAfter` windows on your estate. | ### ๐Ÿ”Œ Automating crt.sh Certificate Transparency Scraper Schedule recurring runs in Apify and route the dataset wherever your team works. Connect with Make or Zapier to post new findings, push alerts to Slack, sync rows into a warehouse with Airbyte, trigger a GitHub Action when new certificates appear for a domain, or drop a snapshot into Google Drive for your records. ### ๐ŸŒŸ Beyond business use cases - **Research:** study how Certificate Transparency adoption and issuer mix change over time. - **Personal:** check what certificates exist for a domain you own. - **Non-profit:** help a small organization inventory its own web presence. - **Experimentation:** feed CT data into your own analysis notebooks and dashboards. ### ๐Ÿค– Ask an AI assistant Paste a dataset row into [ChatGPT](https://chat.openai.com), [Claude](https://claude.ai), [Perplexity](https://www.perplexity.ai), or [Microsoft Copilot](https://copilot.microsoft.com) and ask it to group hostnames, flag certificates nearing expiry, or summarize which issuers sign your domains. ### โ“ Frequently Asked Questions **What is Certificate Transparency?** It is a public, append-only logging system that records every TLS certificate a participating CA issues, so anyone can audit what certificates exist for a domain. **Where does the data come from?** From crt.sh, the Certificate Transparency search service operated by Sectigo. The actor reads its public JSON endpoint. **Do I need an API key or account for crt.sh?** No. crt.sh is public and keyless. You only need an Apify account to run the actor. **How do I find subdomains?** Enable the include subdomains option. The actor then searches `%.yourdomain.com` and the SANs in each row list the hostnames the certificate covers. **Why are there duplicate hostnames?** CT logs hold every certificate ever issued, including renewals and pre-certificates, so the same hostname appears once per certificate. Each row still has a unique `id`. **Does it include expired certificates?** Yes. The log keeps historical entries. Use `notAfter` to filter out expired ones if you only want current coverage. **Can I search several domains at once?** Yes. Add each domain on its own line in the domains field. **Why did a run return fewer rows than expected?** You may have hit your `maxItems` cap, or crt.sh was under heavy load. The actor retries, but very busy periods can still limit a single query. **Is this legal to use?** Certificate Transparency data is public. Use this tool only against domains you own or are explicitly authorized to assess. **What format is the output?** A structured dataset with one record per certificate and the fields listed in the Output table, ready to read in the console or pull from the API. ### ๐Ÿ”Œ Integrate with any app Every run writes to an Apify dataset you can read through the API, webhooks, and the official clients, so the output drops straight into your own security tooling, spreadsheets, or pipelines. ### ๐Ÿ”— Recommended Actors - [UK Postcodes Scraper](https://apify.com/parseforge/uk-postcodes-io-scraper) โ€” look up UK postcodes and administrative areas. - [OpenAQ Air Quality Scraper](https://apify.com/parseforge/openaq-air-quality-scraper) โ€” pull public air quality measurements. - [Open Charge Map Scraper](https://apify.com/parseforge/openchargemap-scraper) โ€” collect EV charging station data. - [Libraries.io Scraper](https://apify.com/parseforge/libraries-io-scraper) โ€” gather open source package metadata. > ๐Ÿ’ก **Pro Tip:** browse the complete [ParseForge collection](https://apify.com/parseforge). **๐Ÿ†˜ Need Help?** [Open our contact form](https://tally.so/r/BzdKgA) > **โš ๏ธ Disclaimer:** independent tool, not affiliated with crt.sh or Sectigo. Only publicly available Certificate Transparency data is collected. Use only against domains you own or are authorized to assess. # Actor input Schema ## `domains` (type: `array`): One or more domains to search in the crt.sh certificate transparency logs, one per line, for example 'github.com'. Each domain returns one row per certificate found. ## `includeSubdomains` (type: `boolean`): When enabled, each domain is searched with a wildcard (%.domain) so certificates issued to subdomains are also returned. Useful for subdomain discovery. ## `maxItems` (type: `integer`): Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000 ## Actor input object example ```json { "domains": [ "github.com" ], "includeSubdomains": true, "maxItems": 10 } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "domains": [ "github.com" ], "maxItems": 10 }; // Run the Actor and wait for it to finish const run = await client.actor("parseforge/crtsh-certificate-transparency-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "domains": ["github.com"], "maxItems": 10, } # Run the Actor and wait for it to finish run = client.actor("parseforge/crtsh-certificate-transparency-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "domains": [ "github.com" ], "maxItems": 10 }' | apify call parseforge/crtsh-certificate-transparency-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=parseforge/crtsh-certificate-transparency-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "crt.sh Certificate Transparency Scraper", "description": "Search the crt.sh certificate transparency logs for any domain you control and surface the hosts behind it. Each record carries the common name, every subject alternative name, the issuing authority, serial number, and validity window. Built for attack surface mapping and asset inventory.", "version": "0.1", "x-build-id": "YVGeSFnWII6e7NTQS" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/parseforge~crtsh-certificate-transparency-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-parseforge-crtsh-certificate-transparency-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/parseforge~crtsh-certificate-transparency-scraper/runs": { "post": { "operationId": "runs-sync-parseforge-crtsh-certificate-transparency-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/parseforge~crtsh-certificate-transparency-scraper/run-sync": { "post": { "operationId": "run-sync-parseforge-crtsh-certificate-transparency-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "domains": { "title": "Domains", "type": "array", "description": "One or more domains to search in the crt.sh certificate transparency logs, one per line, for example 'github.com'. Each domain returns one row per certificate found.", "items": { "type": "string" } }, "includeSubdomains": { "title": "Include subdomains (wildcard)", "type": "boolean", "description": "When enabled, each domain is searched with a wildcard (%.domain) so certificates issued to subdomains are also returned. Useful for subdomain discovery.", "default": true }, "maxItems": { "title": "Max Items", "minimum": 1, "maximum": 1000000, "type": "integer", "description": "Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000" } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```