# Broken Structured Data Scanner (`ib4ngz/broken-structured-data-scanner`) Actor Crawl websites and detect malformed, missing, duplicated, and suspicious structured data. - **URL**: https://apify.com/ib4ngz/broken-structured-data-scanner.md - **Developed by:** [Iqbal R](https://apify.com/ib4ngz) (community) - **Categories:** Automation, Developer tools, SEO tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $2.00 / 1,000 audit report generateds This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ### What does Broken Structured Data Scanner do? **Broken Structured Data Scanner crawls websites and finds malformed, missing, duplicated, inconsistent, or suspicious structured data.** It reports JSON-LD parse failures, basic Microdata and RDFa coverage, common Schema.org value problems, page-level validity, and a site-wide Structured Data Health Score. Try it with a public website such as [apify.com](https://apify.com). Run it on the Apify platform to use API access, schedules, integrations, proxy rotation, run monitoring, and downloadable datasets. The scanner uses fast HTTP requests and does not execute page scripts. ### Why use Broken Structured Data Scanner? - Audit structured-data quality across many pages instead of checking pages individually. - Find broken JSON-LD deployments after CMS, theme, or analytics changes. - Identify pages without structured data and see which Schema.org types occur most often. - Feed stable issue codes into QA checks, dashboards, alerts, and Website Intelligence API workflows. - Prioritize fixes with severity counts, type frequencies, top issue codes, and a transparent health score. ### How to use Broken Structured Data Scanner 1. Open the Actor and go to the **Input** tab. 2. Add one or more start URLs. Multiple start URLs may point to different websites. 3. Set the page and depth limits, then choose which structured-data formats to scan. 4. Optionally add URL patterns or a proxy configuration. 5. Click **Start**. 6. Review page records in the Dataset and the site-wide summary in the `OUTPUT` key-value record. ### Input The `startUrls` field is required. There is no Actor-imposed maximum number of seed URLs; `maxPages` controls the total number of pages processed across all seeds. | Field | Default | Description | | ----------------------------------- | ------------------- | ----------------------------------------------------- | | `startUrls` | `https://apify.com` | One or more HTTP(S) seed URLs. | | `maxPages` | `500` | Total page limit across all seeds. | | `maxDepth` | `0` | Link depth. The default scans only supplied URLs. | | `sameDomainOnly` | `true` | Keeps discovered links on the hostname of their seed. | | `includePatterns` | `[]` | Case-insensitive wildcard filters using `*` and `?`. | | `excludePatterns` | `[]` | Wildcard exclusions; exclusions override inclusions. | | `scanJsonLd` | `true` | Scan JSON-LD blocks. | | `scanMicrodata` | `true` | Scan basic Microdata markup. | | `scanRdfa` | `true` | Scan basic RDFa markup. | | `includePagesWithoutStructuredData` | `true` | Include HTML pages where no entities are detected. | | `proxyConfiguration` | proxy disabled | Apify Proxy or custom proxy settings. | Example input: ```json { "startUrls": [{ "url": "https://apify.com" }, { "url": "https://docs.apify.com" }], "maxPages": 500, "maxDepth": 0, "sameDomainOnly": true, "includePatterns": ["https://apify.com/*"], "excludePatterns": ["*/pricing*"], "scanJsonLd": true, "scanMicrodata": true, "scanRdfa": true, "includePagesWithoutStructuredData": true, "proxyConfiguration": { "useApifyProxy": false } } ``` Start URLs are always eligible. Include and exclude patterns apply to links discovered from them. Obvious non-HTML assets such as images, PDFs, archives, stylesheets, scripts, fonts, and media files are not enqueued. ### Output The Actor writes one Dataset item for each included page. It does not store raw HTML or full JSON-LD payloads. ```json { "url": "https://apify.com/store", "finalUrl": "https://apify.com/store", "statusCode": 200, "depth": 1, "title": "Apify Store", "structuredData": { "jsonLdBlocks": 1, "jsonLdEntities": 1, "microdataItems": 0, "rdfaItems": 0, "types": ["WebPage"] }, "issues": [], "issueCounts": { "critical": 0, "error": 0, "warning": 0, "info": 0 }, "valid": true } ``` `valid` is `true` when the page has no critical or error issues. Warnings and informational findings do not invalidate a page. You can download the dataset in various formats such as JSON, HTML, CSV, or Excel. The `OUTPUT` record contains the run summary: ```json { "scannedUrls": 120, "urlsWithStructuredData": 94, "urlsWithoutStructuredData": 26, "urlsWithIssues": 31, "validUrls": 112, "invalidUrls": 8, "issues": { "critical": 0, "error": 9, "warning": 38, "info": 26 }, "schemaTypes": { "Article": 18, "Organization": 1, "WebPage": 75 }, "topIssues": [{ "code": "NO_STRUCTURED_DATA", "count": 26 }], "healthScore": 99 } ``` ### Data table | Field | Meaning | | ------------------------------ | -------------------------------------------------------------------------------- | | `url`, `finalUrl` | Requested and final response URLs. | | `statusCode`, `depth`, `title` | Page response and crawl metadata. | | `structuredData` | Block/item counts and detected type names. | | `issues` | Stable code, severity, category, message, format, property, and source location. | | `issueCounts` | Critical, error, warning, and info totals for the page. | | `valid` | Whether the page has no critical or error issues. | Issue severities are `critical`, `error`, `warning`, and `info`. Categories are `syntax`, `parse`, `schema`, `property`, `value`, `consistency`, `duplicate`, `conflict`, `missing`, and `rendering`. The first rule packs cover `Product`, `Article`, `BlogPosting`, `NewsArticle`, `Organization`, `LocalBusiness`, `Person`, `BreadcrumbList`, `FAQPage`, `WebSite`, `WebPage`, `Event`, `Recipe`, and `VideoObject`. ### How much does it cost to scan structured data? Cost depends mainly on the number and size of pages, response speed, retries, and proxy use. Cheerio-based HTTP crawling is lightweight compared with browser automation. Start with a small `maxPages` value, review the run, then scale up. Apify's free plan can be used for small evaluations within its current platform limits; check the Apify pricing page for up-to-date allowances. ### Tips and advanced options - Use several `startUrls` to audit multiple sites in one run. - Keep `sameDomainOnly` enabled for controlled site audits. - Increase `maxDepth` above `0` only when you want to discover and scan links from the supplied URLs. - Narrow large sites with wildcard include patterns, then remove unwanted paths with exclusions. - Enable a proxy only when the target requires it; proxy use is disabled by default. - Treat the Structured Data Health Score as a QA indicator, not a search ranking signal. The score starts at 100 and subtracts the average per-page penalty: critical 25, error 10, warning 2, and info 0. It is rounded and clamped to 0-100. For local development: ```bash npm install npm start ``` Run with the Apify CLI: ```bash apify run ``` Deploy and run on Apify: ```bash apify login apify push ``` ### FAQ, disclaimers, and support #### Does this replace Google's Rich Results Test? No. It is a deterministic structured-data quality scanner, not a Google eligibility, ranking, or rich-results predictor. #### What are the current limitations? The MVP has no JavaScript rendering, complete Schema.org vocabulary reasoning, SHACL validation, remote context resolution, RDF graph inference, cross-page template anomaly detection, or Google-specific rule pack. It scans HTML returned directly by the server. #### Is website scanning legal? You are responsible for complying with applicable laws, website terms, robots policies, access restrictions, and data-protection requirements. Scan only websites you are authorized to access and use considerate limits. For bugs or feature requests, use the Actor's **Issues** tab. Custom Website Intelligence integrations and validation rule packs can be developed for specialized workflows. # Actor input Schema ## `startUrls` (type: `array`): One or more HTTP(S) website URLs to scan. Each URL can seed a different website. ## `maxPages` (type: `integer`): Maximum number of pages processed across all start URLs. ## `maxDepth` (type: `integer`): Maximum link depth from each start URL. Keep 0 to scan only the URLs supplied in startUrls. ## `sameDomainOnly` (type: `boolean`): Only follow links on the hostname associated with each start URL. ## `includePatterns` (type: `array`): Optional case-insensitive wildcard patterns for discovered URLs. Supports \* and ?. ## `excludePatterns` (type: `array`): Case-insensitive wildcard patterns to exclude. Excludes override includes. ## `scanJsonLd` (type: `boolean`): Extract and validate application/ld+json blocks. ## `scanMicrodata` (type: `boolean`): Discover basic itemscope, itemtype, and itemprop markup. ## `scanRdfa` (type: `boolean`): Discover basic typeof, property, vocab, resource, and about markup. ## `includePagesWithoutStructuredData` (type: `boolean`): Write Dataset records for HTML pages where no structured-data entities are detected. ## `proxyConfiguration` (type: `object`): Optional Apify Proxy or custom proxy settings. ## Actor input object example ```json { "startUrls": [ { "url": "https://apify.com" } ], "maxPages": 500, "maxDepth": 0, "sameDomainOnly": true, "includePatterns": [], "excludePatterns": [], "scanJsonLd": true, "scanMicrodata": true, "scanRdfa": true, "includePagesWithoutStructuredData": true, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `results` (type: `string`): No description ## `summary` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://apify.com" } ] }; // Run the Actor and wait for it to finish const run = await client.actor("ib4ngz/broken-structured-data-scanner").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://apify.com" }] } # Run the Actor and wait for it to finish run = client.actor("ib4ngz/broken-structured-data-scanner").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://apify.com" } ] }' | apify call ib4ngz/broken-structured-data-scanner --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,ib4ngz/broken-structured-data-scanner" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/Z0viMpPRoTI3DcaIn/builds/3Kn09YacDExyTsiTI/openapi.json