# Web Intelligence: WCAG, AI-RAG & E-Commerce Suite (`lightoton/web-intelligence-suite`) Actor

High-speed technical WCAG 2.1 AA DOM auditor, OWASP security header checker, LLM/RAG clean Markdown converter, and universal E-Commerce product/price extractor.

- **URL**: https://apify.com/lightoton/web-intelligence-suite.md
- **Developed by:** [Viktor Bulatov](https://apify.com/lightoton) (community)
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $10.00 / 1,000 page processeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Web Intelligence Suite (v2.2)

> **High-speed technical web auditor and extractor: W3C WCAG 2.1/2.2 Level AA, European Accessibility Act (EAA 2025), German BFSG, US ADA Title III, OWASP Security Headers, AI/RAG Markdown Converter, and Universal E-Commerce Product Intelligence.**

***

### Overview

**Web Intelligence Suite** is a unified, serverless cloud engine designed for digital agencies, compliance officers, data engineers, and AI agent developers. It combines comprehensive technical accessibility auditing, LLM-ready content extraction, security analysis, and e-commerce scraping into a single high-performance pipeline.

Unlike traditional crawlers that spin up resource-heavy headless browsers (Puppeteer/Playwright) consuming 1–2 GB of RAM per run, Web Intelligence Suite uses an optimized DOM analysis pipeline powered by Cheerio and Axios. It processes pages in **200–500 milliseconds** using only **128 MB of memory**, reducing execution costs and latency by over 80%.

***

### Key Capabilities

#### 1. Web Accessibility Audit (WCAG 2.1/2.2 AA, EAA 2025, BFSG, ADA)

Automated technical validation aligned with W3C WCAG 2.1/2.2 Level A/AA criteria:

- **Form Controls & Labels (WCAG 3.3.2 / 4.1.2)**: Identifies inputs, selects, and textareas missing programmatic `<label>`, `aria-label`, or `aria-labelledby`.
- **Text Alternatives (WCAG 1.1.1)**: Detects images missing `alt` attributes or with invalid empty text.
- **Language Declaration (WCAG 3.1.1)**: Validates presence of valid `lang` attribute on the root `<html>` element.
- **Heading Hierarchy (WCAG 1.3.1)**: Verifies presence of main `<h1>` landmarks and logical heading progressions.
- **Viewport Zoom Restrictions (WCAG 1.4.4)**: Detects mobile `user-scalable=no` and `maximum-scale=1.0` viewport locks.
- **CSS Selectors & Code Snippets**: Every finding includes exact DOM CSS selectors and code snippets for instant developer remediation.

#### 2. Third-Party Resource & Privacy Hygiene (GDPR / DSGVO)

- **External CDN Fonts**: Identifies unhosted external font dependencies (e.g. `fonts.googleapis.com`, Adobe Typekit) that pose GDPR compliance liabilities in the European Union.
- **Third-Party Trackers & Pixels**: Audits presence of external analytics and advertising scripts (Google Analytics, Meta Pixel, TikTok, Hotjar).
- **Consent Management Platform (CMP)**: Automatically detects installed CMP banners (Usercentrics, Cookiebot, OneTrust, Borlabs Cookie, Complianz, Axeptio).

#### 3. OWASP HTTP Security Headers & Metadata

- **Security Headers**: Audits essential OWASP headers: HTTP Strict Transport Security (`HSTS`), Content Security Policy (`CSP`), `X-Frame-Options`, `X-Content-Type-Options`, and `Referrer-Policy`.
- **SEO & Social Graph**: Validates `<title>`, meta descriptions, canonical URLs, and OpenGraph/Twitter Card metadata.
- **Schema.org Structured Data**: Extracts top-level JSON-LD schema types (`Organization`, `LocalBusiness`, `FAQPage`, `Product`, `Article`).

#### 4. AI / RAG Clean Markdown Extractor

- Strips navigation bars, footers, scripts, styles, and promotional noise.
- Converts main content, article bodies, headings, lists, and HTML tables into clean, structured Markdown.
- Calculates exact estimated token counts for context window optimization in ChatGPT, Claude, and LangChain agents.

#### 5. Universal E-Commerce Product Intelligence

- Automatically extracts structured product data across Shopify, WooCommerce, Shopware, Magento, and Schema.org Product markup.
- Normalizes product name, SKU, GTIN/EAN, price, currency, availability, and customer review metrics.

***

### Executive Visual HTML Reports

When auditing websites, the Actor automatically compiles an interactive, executive **Visual HTML Audit Report** and stores it in the run's default Key-Value Store:

- **Single Page Scan**: Detailed breakdown with compliance score (0–100), letter grade (A–F), metric cards, and remediation code cards.
- **Multi-Page Site Crawl**: Aggregated domain health score, page-by-page audit table, and prioritized critical violation inventory.
- **Print to PDF**: Built-in CSS print styling allows digital agencies to export white-label PDF audit reports for clients directly from their browser.
- **Access**: Available under the **Key-Value Store** tab as `OUTPUT_REPORT.html`.

***

### Operation Modes

| Mode | Identifier | Description | Use Case |
| :--- | :--- | :--- | :--- |
| **Full Suite** | `full` | Complete WCAG audit, security check, Markdown extraction, and e-commerce scraping. | Default comprehensive domain analysis. |
| **Accessibility Only** | `accessibility` | High-speed accessibility and compliance audit only (WCAG, EAA, BFSG, ADA). | Fast client compliance audits & reporting. |
| **AI / RAG Markdown** | `markdown` | Clean Markdown extraction and token estimation without audit scoring. | Feeding live web content into LLMs & RAG pipelines. |
| **E-Commerce Only** | `ecommerce` | Product price, stock, SKU, and schema extraction. | Price monitoring and catalog synchronization. |

***

### Input Configuration

```json
{
  "startUrls": [
    { "url": "https://example.com" }
  ],
  "mode": "full",
  "crawlInternalLinks": true,
  "maxPages": 20,
  "generateVisualReport": true
}
```

#### Parameter Reference

- **`startUrls`** (*array, required*): One or more URLs to process. You can provide a homepage, multiple links, or a sitemap URL.
- **`mode`** (*string, optional, default: `"full"`*): Pipeline mode (`"full"`, `"accessibility"`, `"markdown"`, or `"ecommerce"`).
- **`crawlInternalLinks`** (*boolean, optional, default: `true`*): When enabled, discovers and audits internal same-domain links automatically.
- **`maxPages`** (*integer, optional, default: `20`*): Total ceiling on audited pages per run to protect budget limits.
- **`generateVisualReport`** (*boolean, optional, default: `true`*): Generates `OUTPUT_REPORT.html` in Key-Value Store for visual inspection.

***

### Output Data Structure

#### Dataset Overview Columns (Table View)

| Field | Type | Description |
| :--- | :--- | :--- |
| `finalUrl` | String | Audited page URL after redirects. |
| `domain` | String | Extracted target domain. |
| `score` | Integer | Overall compliance and quality score (0–100). |
| `grade` | String | Letter rating (`A`, `B`, `C`, `F`). |
| `criticalIssues` | Integer | Number of critical accessibility or security violations. |
| `tokens` | Integer | Estimated token count of clean Markdown content. |
| `durationMs` | Integer | HTTP response and parsing latency in milliseconds. |
| `statusCode` | Integer | HTTP response status code (e.g. 200). |

***

### Programmatic Integration Examples

#### Python (`apify-client`)

```python
from apify_client import ApifyClient

## Initialize the client with your Apify API token
client = ApifyClient("YOUR_APIFY_TOKEN")

## Run the Actor
run_input = {
    "startUrls": [{"url": "https://example.com"}],
    "mode": "accessibility",
    "crawlInternalLinks": True,
    "maxPages": 10
}

run = client.actor("lightoton/web-intelligence-suite").call(run_input=run_input)

## Fetch results from the output dataset
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"URL: {item.get('finalUrl')} | Score: {item.get('score')} | Grade: {item.get('grade')}")
```

#### JavaScript / Node.js (`apify-client`)

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });

const run = await client.actor('lightoton/web-intelligence-suite').call({
  startUrls: [{ url: 'https://example.com' }],
  mode: 'markdown',
  maxPages: 5,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const item of items) {
  console.log(`Title: ${item.title} | Tokens: ${item.tokens}`);
}
```

#### LangChain / AI Agent Tool

```python
from langchain.tools import Tool
from apify_client import ApifyClient

apify_client = ApifyClient("YOUR_APIFY_TOKEN")

def extract_web_markdown(url: str) -> str:
    run = apify_client.actor("lightoton/web-intelligence-suite").call(
        run_input={"startUrls": [{"url": url}], "mode": "markdown", "crawlInternalLinks": False, "maxPages": 1}
    )
    items = list(apify_client.dataset(run["defaultDatasetId"]).iterate_items())
    return items[0].get("markdown", "") if items else "Failed to extract content."

web_extractor_tool = Tool(
    name="WebToMarkdownExtractor",
    func=extract_web_markdown,
    description="Extracts clean, noise-free Markdown with tables and headers from any target URL."
)
```

***

### Pricing

Monetized via Apify **Pay-Per-Event**:

- **$10.00 / 1,000 pages processed** ($0.01 per audited URL).
- No flat monthly fees. Pay only for pages successfully processed.

# Actor input Schema

## `startUrls` (type: `array`):

URLs to audit. Enter a domain/homepage (e.g. https://example.com), paste multiple links, or provide a sitemap.xml URL.

## `mode` (type: `string`):

Select the audit pipeline: Full Suite, Accessibility Audit Only (WCAG 2.1 AA / EAA / BFSG / ADA), AI/LLM Clean Markdown, or E-Commerce Products.

## `crawlInternalLinks` (type: `boolean`):

Automatically discover and audit internal subpages found on the same website/domain.

## `maxPages` (type: `integer`):

Maximum total number of pages to audit (protects your budget limit).

## `generateVisualReport` (type: `boolean`):

Generates an executive visual HTML report with interactive cards, printable to PDF, saved in Key-Value Store as OUTPUT_REPORT.html.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://example.com"
    }
  ],
  "mode": "full",
  "crawlInternalLinks": true,
  "maxPages": 20,
  "generateVisualReport": true
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset containing the processed URLs and their extracted intelligence.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://example.com"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("lightoton/web-intelligence-suite").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://example.com" }] }

# Run the Actor and wait for it to finish
run = client.actor("lightoton/web-intelligence-suite").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://example.com"
    }
  ]
}' |
apify call lightoton/web-intelligence-suite --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,lightoton/web-intelligence-suite"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/VtawlgsUfqcorKCPN/builds/wk7NDjfPNiJeSTdB1/openapi.json