# Web Data To Excel Scraper (`automation-lab/schema-guided-web-data-to-excel`) Actor

Extract user-defined fields from repeated records on public static webpages. Normalize text, numbers, dates, booleans, and URLs into validated dataset rows and a filterable XLSX workbook.

- **URL**: https://apify.com/automation-lab/schema-guided-web-data-to-excel.md
- **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community)
- **Categories:** Developer tools, Business
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event + usage

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Web Data To Excel Scraper

Turn repeated records on public static webpages into a consistent Excel workbook and typed dataset rows.

**Web Data To Excel Scraper** fetches anonymous HTTP(S) pages, finds each record with a CSS selector, and applies your field schema to text, HTML, or attributes. It normalizes strings, numbers, integers, booleans, URLs, and dates, then produces:

- source-attributed dataset rows;
- field-level validation warnings;
- a filterable `.xlsx` workbook;
- a JSON extraction report.

This Actor is designed for repeatable **web data to Excel** jobs where you know the page structure and need explicit, auditable column mappings. It does not use AI to guess fields and does not render JavaScript-only pages.

### Who is it for?

- **Research analysts** consolidating repeated cards, listings, or table-like page records.
- **Ecommerce teams** collecting public product names, prices, availability, and links.
- **Operations teams** replacing manual copy-and-paste work with scheduled Excel exports.
- **Data engineers** feeding a stable dataset shape into Sheets, databases, or ETL tools.
- **Auditors** who need source URLs, row indexes, and visible validation warnings.

Use a source-specific scraper when a website needs login, browser interaction, pagination discovery, or anti-bot handling. Use this Actor when the target is public, server-rendered HTML and CSS selectors describe the records reliably.

### What can it extract?

Each field definition becomes an Excel column inside `data` in the Apify dataset.

| Type | Example input | Normalized output |
| --- | --- | --- |
| `string` | `In stock` | `"In stock"` |
| `number` | `£51.77` | `51.77` |
| `integer` | `1.` | `1` with pattern `(\d+)` |
| `boolean` | `yes` | `true` |
| `url` | `catalogue/item.html` | absolute HTTP(S) URL |
| `date` | `January 15, 2025` | ISO 8601 timestamp |

A field can read:

- visible text with `source: "text"`;
- inner HTML with `source: "html"`;
- an attribute such as `href`, `src`, or `title` with `source: "attribute"`;
- all matching elements with `multiple: true` and a custom delimiter;
- the first regex match or capture group with `pattern`;
- a fallback value when no content is found.

### Why use an explicit field schema?

Explicit selectors make extraction behavior inspectable and repeatable.

You control:

1. what counts as one record;
2. which element maps to each column;
3. how values are converted;
4. which fields are required;
5. whether partially valid rows remain in the output.

The Actor never silently invents missing values. A missing required value, failed regex, or failed type conversion produces a warning and `valid: false`.

### Getting started

1. Open the Actor input page.
2. Add one or more anonymous public webpage URLs.
3. Enter a CSS selector that matches each repeated record.
4. Define the fields relative to each record.
5. Set a small `maxItems` value for the first run.
6. Run the Actor and inspect the **Normalized rows** dataset view.
7. Download `OUTPUT.xlsx` from the run output.
8. Review `REPORT` for matched, saved, valid, and invalid row counts.
9. Increase the limit or schedule the task after confirming the selectors.

Browser developer tools can help find selectors: inspect a repeated card, identify its shared class, and test the selector in the browser console with `document.querySelectorAll('your-selector')`.

### Input parameters

| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `startUrls` | array | Yes | Up to 50 public static HTTP(S) pages with the same record structure. |
| `recordSelector` | string | Yes | CSS selector matching each repeated record. |
| `fields` | array | Yes | Between 1 and 50 field definitions. |
| `maxItems` | integer | No | Maximum saved rows across all pages; default 100, maximum 10,000. |
| `includeInvalidRows` | boolean | No | Keep rows with warnings; default `true`. |
| `requestTimeoutSecs` | integer | No | Timeout per page from 5 to 60 seconds; default 30. |
| `sheetName` | string | No | Excel worksheet name; default `Web data`. |

Field names must begin with a letter and contain only letters, numbers, or underscores. They are used directly as workbook column names.

### Example: extract website product data to Excel

```json
{
  "startUrls": [{ "url": "https://books.toscrape.com/" }],
  "recordSelector": "article.product_pod",
  "fields": [
    {
      "name": "title",
      "selector": "h3 a",
      "source": "attribute",
      "attribute": "title",
      "type": "string",
      "required": true
    },
    {
      "name": "price",
      "selector": ".price_color",
      "type": "number",
      "pattern": "([0-9.]+)",
      "required": true
    },
    {
      "name": "productUrl",
      "selector": "h3 a",
      "source": "attribute",
      "attribute": "href",
      "type": "url",
      "required": true
    }
  ],
  "maxItems": 8,
  "sheetName": "Book prices"
}
```

Relative links are resolved against the final page URL. Numeric patterns remove currency symbols without putting them into the number cell.

### Dataset output

A real row from the product example looks like this:

```json
{
  "sourceUrl": "https://books.toscrape.com/",
  "recordIndex": 1,
  "data": {
    "title": "A Light in the Attic",
    "price": 51.77,
    "productUrl": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"
  },
  "valid": true,
  "validationWarnings": [],
  "scrapedAt": "2025-01-15T12:00:00.000Z"
}
```

| Output field | Meaning |
| --- | --- |
| `sourceUrl` | Final URL used after safe redirects. |
| `recordIndex` | One-based record position on that page. |
| `data` | User-defined fields and normalized values. |
| `valid` | `true` when no field generated a warning. |
| `validationWarnings` | Field, message, and raw value for every validation problem. |
| `scrapedAt` | ISO time when the row was created. |

The default dataset remains the automation-friendly result. The Excel workbook mirrors the configured field columns and adds source, validity, warning, and timestamp columns.

### Excel workbook and report

Every successful run stores two named outputs in the default key-value store:

- `OUTPUT.xlsx` — workbook with frozen headers and filters;
- `REPORT` — JSON summary with source-level matched and saved counts.

Even a legitimate no-result run creates an empty workbook with the configured headers. This makes scheduled pipelines predictable.

The report distinguishes records matched by the CSS selector from rows actually saved. When `includeInvalidRows` is `false`, invalid rows can be matched but omitted.

### Validation warnings

Warnings are attached to their row rather than hidden in logs.

Common messages include:

- `Required value is missing`;
- `Value did not match the configured pattern`;
- `Value is not a valid number`;
- `Value is not an integer`;
- `Value is not a recognized boolean`;
- `Value is not a valid date`;
- `Value is not a valid HTTP(S) URL`.

Start with `includeInvalidRows: true` while developing selectors. Filter the Excel `valid` column, then tighten the schema or disable invalid rows after reviewing edge cases.

### How much does it cost to extract website data to Excel?

The Actor uses pay-per-event pricing:

- one `start` event per run;
- one `item` event for each dataset row produced;
- no separate charge for the XLSX workbook or JSON report.

At the current BRONZE rates, a run starts at **$0.0005** plus **$0.004 per saved row**. For example:

| Saved rows | Example total |
| ---: | ---: |
| 10 | $0.0405 |
| 100 | $0.4005 |
| 1,000 | $4.0005 |

Higher subscription tiers receive lower item prices. The Apify Console shows the applicable price before you start a run. Rows excluded by `includeInvalidRows: false` do not produce an item event.

### Scheduling recurring exports

Create an Apify Task with tested selectors, then add a schedule.

A practical recurring workflow is:

1. run daily or weekly;
2. read the default dataset through the API;
3. compare keys meaningful to your source;
4. send new or changed rows downstream;
5. retain the workbook as a human-review artifact.

The Actor itself does not compare run history, send alerts, or discover new pages. Connect it to Apify integrations, Make, Zapier, a webhook, or your own data pipeline for those steps.

### Integration patterns

#### Google Sheets or Microsoft Excel

Download `OUTPUT.xlsx` manually, or fetch the named key-value-store record after each run and upload it to your document storage.

#### Database and ETL

Read the default dataset as JSON. The stable envelope fields support lineage, while `data` contains your configured schema.

#### Data quality checks

Filter rows where `valid` is false. Route `validationWarnings` to an exception table instead of accepting silent blanks.

#### Multi-page consolidation

Supply pages that share the same HTML structure. The Actor applies one record selector and field schema to every URL, then creates one workbook.

### Run with the Apify API

Replace `<APIFY_TOKEN>` with your token.

#### cURL

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~schema-guided-web-data-to-excel/runs?token=<APIFY_TOKEN>&waitForFinish=120" \
  -H "Content-Type: application/json" \
  -d @input.json
```

#### JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/schema-guided-web-data-to-excel').call({
  startUrls: [{ url: 'https://books.toscrape.com/' }],
  recordSelector: 'article.product_pod',
  fields: [
    { name: 'title', selector: 'h3 a', source: 'attribute', attribute: 'title', type: 'string', required: true },
    { name: 'price', selector: '.price_color', type: 'number', pattern: '([0-9.]+)', required: true }
  ],
  maxItems: 20
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

#### Python

```python
from apify_client import ApifyClient

client = ApifyClient("<APIFY_TOKEN>")
run = client.actor("automation-lab/schema-guided-web-data-to-excel").call(run_input={
    "startUrls": [{"url": "https://books.toscrape.com/"}],
    "recordSelector": "article.product_pod",
    "fields": [
        {"name": "title", "selector": "h3 a", "source": "attribute", "attribute": "title", "type": "string", "required": True},
        {"name": "price", "selector": ".price_color", "type": "number", "pattern": "([0-9.]+)", "required": True}
    ],
    "maxItems": 20
})
items = client.dataset(run["defaultDatasetId"]).list_items().items
print(items)
```

### Use with MCP and AI assistants

Add the Actor to Claude Code:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/schema-guided-web-data-to-excel"
```

#### Claude Desktop, Cursor, and VS Code setup

Use this MCP configuration in Claude Desktop, Cursor, or VS Code:

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/schema-guided-web-data-to-excel"
    }
  }
}
```

Example prompts:

- “Run the web data to Excel Actor on this public catalog with `.product-card` records and extract title, numeric price, and absolute product URL.”
- “Combine these two static quote pages, join all `.tag` values, and give me the workbook link.”
- “Review rows with validation warnings and summarize which required selector failed most often.”

Always review selectors and target authorization before asking an assistant to run extraction.

### Limits and failure behavior

- Only anonymous, public HTTP(S) URLs are accepted.
- Localhost, private-network, reserved, and credential-bearing URLs are rejected.
- Pages must return HTML and stay within the 5 MB response limit.
- Up to five safe redirects are followed.
- Transient request failures receive bounded retries.
- JavaScript rendering, clicks, login, CAPTCHA solving, proxy fallback, and pagination discovery are not included.
- All supplied pages must work with the same record selector and field schema.
- The Actor stops at `maxItems` saved rows across all pages.
- CSS selectors reflect the current website structure and may need updating after a redesign.

A blocked or non-HTML source fails the run instead of returning misleading empty rows. A valid page with zero selector matches succeeds and reports zero matched records.

### Legality and responsible use

Only extract pages and data you are authorized to access.

Review the target website’s terms, robots guidance, copyright rules, privacy obligations, and applicable laws. Avoid collecting sensitive personal data, bypassing access controls, or sending excessive traffic. Keep schedules and page counts proportionate to the source.

This Actor does not provide legal advice. You remain responsible for your input URLs, selectors, storage, and downstream use.

### Troubleshooting

#### Why did the run return zero rows?

Open the page without JavaScript and confirm that the repeated records exist in the original HTML. Test `recordSelector` in browser developer tools. A selector that matches rendered DOM added by JavaScript will not work here.

#### Why is a numeric field null?

Use a pattern that captures only the numeric portion, such as `([0-9.]+)`. Check `validationWarnings.rawValue` to see what the selector returned.

#### Why is a relative link different in output?

URL fields are resolved against the final page URL. This intentionally makes links usable outside the source page.

#### Why did the URL fail before downloading?

The Actor rejects non-public addresses and URLs containing embedded credentials. Host the page on an anonymously reachable public HTTP(S) endpoint or choose a different Actor designed for authenticated sources.

#### Where is the Excel file?

Open the run’s **Output** tab and choose **Excel workbook**, or download the `OUTPUT.xlsx` record from the default key-value store.

### Related Automation Lab Actors

- [CSV & Excel Data Quality Cleaner](https://apify.com/automation-lab/csv-excel-data-quality-cleaner) — normalize, validate, and deduplicate an existing CSV or XLSX table after extraction.
- [Flexible HTTP Request Runner](https://apify.com/automation-lab/flexible-http-request-runner) — execute public HTTP/API requests when you need response bodies and metadata rather than repeated HTML record extraction.

These Actors cover different stages: fetch structured responses, extract webpage records, then clean an existing tabular file.

### FAQ

#### Can it scrape dynamic webpages?

No. It processes server-rendered static HTML. Choose a browser-based source-specific Actor for pages whose records appear only after JavaScript runs.

#### Can every URL use a different schema?

Not in one run. Group URLs by shared structure and create a separate Task for each selector/schema combination.

#### Does it infer fields from free-form prose?

No. The schema is explicit by design. Regex patterns can isolate values from text, but the Actor does not make semantic guesses.

#### Are invalid rows charged?

A row included in the dataset produces an item event. Set `includeInvalidRows` to `false` to omit warning-bearing rows while preserving matched/saved counts in the report.

#### Can I export JSON instead of Excel?

Yes. Every saved row is available in the default Apify dataset as JSON, CSV, XML, Excel, and other platform export formats. The named workbook is an additional ready-to-download artifact.

#### Does it crawl links or paginate automatically?

No. Supply each public page URL explicitly. This keeps scope and cost predictable.

#### How many pages and rows can I process?

A run accepts up to 50 URLs and 10,000 saved rows. Start small to verify selectors and respect the target website.

# Actor input Schema

## `startUrls` (type: `array`):

Static, anonymously reachable HTTP(S) pages that contain the same kind of repeated records. Up to 50 URLs.

## `recordSelector` (type: `string`):

CSS selector for each repeated record, evaluated against every page.

## `fields` (type: `array`):

Columns to extract relative to each matched record. Field names become Excel column names.

## `maxItems` (type: `integer`):

Maximum dataset and Excel rows across all pages.

## `includeInvalidRows` (type: `boolean`):

Keep partially extracted records and expose validation warnings instead of dropping them.

## `requestTimeoutSecs` (type: `integer`):

Per-page HTTP timeout in seconds.

## `sheetName` (type: `string`):

Worksheet name, up to 31 characters.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://books.toscrape.com/"
    }
  ],
  "recordSelector": "article.product_pod",
  "fields": [
    {
      "name": "title",
      "selector": "h3 a",
      "source": "attribute",
      "attribute": "title",
      "type": "string",
      "required": true
    },
    {
      "name": "price",
      "selector": ".price_color",
      "type": "number",
      "pattern": "([0-9.]+)",
      "required": true
    },
    {
      "name": "productUrl",
      "selector": "h3 a",
      "source": "attribute",
      "attribute": "href",
      "type": "url",
      "required": true
    }
  ],
  "maxItems": 20,
  "includeInvalidRows": true,
  "requestTimeoutSecs": 30,
  "sheetName": "Web data"
}
```

# Actor output Schema

## `dataset` (type: `string`):

Dataset containing source-attributed normalized records.

## `workbook` (type: `string`):

Download the generated XLSX workbook.

## `report` (type: `string`):

Open the JSON extraction summary and validation counts.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://books.toscrape.com/"
        }
    ],
    "recordSelector": "article.product_pod",
    "fields": [
        {
            "name": "title",
            "selector": "h3 a",
            "source": "attribute",
            "attribute": "title",
            "type": "string",
            "required": true
        },
        {
            "name": "price",
            "selector": ".price_color",
            "type": "number",
            "pattern": "([0-9.]+)",
            "required": true
        },
        {
            "name": "productUrl",
            "selector": "h3 a",
            "source": "attribute",
            "attribute": "href",
            "type": "url",
            "required": true
        }
    ],
    "maxItems": 20
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/schema-guided-web-data-to-excel").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [{ "url": "https://books.toscrape.com/" }],
    "recordSelector": "article.product_pod",
    "fields": [
        {
            "name": "title",
            "selector": "h3 a",
            "source": "attribute",
            "attribute": "title",
            "type": "string",
            "required": True,
        },
        {
            "name": "price",
            "selector": ".price_color",
            "type": "number",
            "pattern": "([0-9.]+)",
            "required": True,
        },
        {
            "name": "productUrl",
            "selector": "h3 a",
            "source": "attribute",
            "attribute": "href",
            "type": "url",
            "required": True,
        },
    ],
    "maxItems": 20,
}

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/schema-guided-web-data-to-excel").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://books.toscrape.com/"
    }
  ],
  "recordSelector": "article.product_pod",
  "fields": [
    {
      "name": "title",
      "selector": "h3 a",
      "source": "attribute",
      "attribute": "title",
      "type": "string",
      "required": true
    },
    {
      "name": "price",
      "selector": ".price_color",
      "type": "number",
      "pattern": "([0-9.]+)",
      "required": true
    },
    {
      "name": "productUrl",
      "selector": "h3 a",
      "source": "attribute",
      "attribute": "href",
      "type": "url",
      "required": true
    }
  ],
  "maxItems": 20
}' |
apify call automation-lab/schema-guided-web-data-to-excel --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,automation-lab/schema-guided-web-data-to-excel"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/V6I18lGcLpNPVVbzM/builds/KLJP8Qe6klwIwJ1hY/openapi.json