Web Data To Excel Scraper

Turn repeated records on public static webpages into a consistent Excel workbook and typed dataset rows.

Web Data To Excel Scraper fetches anonymous HTTP(S) pages, finds each record with a CSS selector, and applies your field schema to text, HTML, or attributes. It normalizes strings, numbers, integers, booleans, URLs, and dates, then produces:

• source-attributed dataset rows;
• field-level validation warnings;
• a filterable .xlsx workbook;
• a JSON extraction report.

This Actor is designed for repeatable web data to Excel jobs where you know the page structure and need explicit, auditable column mappings. It does not use AI to guess fields and does not render JavaScript-only pages.

Who is it for?

• Research analysts consolidating repeated cards, listings, or table-like page records.
• Ecommerce teams collecting public product names, prices, availability, and links.
• Operations teams replacing manual copy-and-paste work with scheduled Excel exports.
• Data engineers feeding a stable dataset shape into Sheets, databases, or ETL tools.
• Auditors who need source URLs, row indexes, and visible validation warnings.

Use a source-specific scraper when a website needs login, browser interaction, pagination discovery, or anti-bot handling. Use this Actor when the target is public, server-rendered HTML and CSS selectors describe the records reliably.

What can it extract?

Each field definition becomes an Excel column inside data in the Apify dataset.

Type	Example input	Normalized output
string	In stock	"In stock"
number	£51.77	51.77
integer	1.	1 with pattern (\d+)
boolean	yes	true
url	catalogue/item.html	absolute HTTP(S) URL
date	January 15, 2025	ISO 8601 timestamp

A field can read:

• visible text with source: "text";
• inner HTML with source: "html";
• an attribute such as href, src, or title with source: "attribute";
• all matching elements with multiple: true and a custom delimiter;
• the first regex match or capture group with pattern;
• a fallback value when no content is found.

Why use an explicit field schema?

Explicit selectors make extraction behavior inspectable and repeatable.

You control:

1. what counts as one record;
2. which element maps to each column;
3. how values are converted;
4. which fields are required;
5. whether partially valid rows remain in the output.

The Actor never silently invents missing values. A missing required value, failed regex, or failed type conversion produces a warning and valid: false.

Getting started

1. Open the Actor input page.
2. Add one or more anonymous public webpage URLs.
3. Enter a CSS selector that matches each repeated record.
4. Define the fields relative to each record.
5. Set a small maxItems value for the first run.
6. Run the Actor and inspect the Normalized rows dataset view.
7. Download OUTPUT.xlsx from the run output.
8. Review REPORT for matched, saved, valid, and invalid row counts.
9. Increase the limit or schedule the task after confirming the selectors.

Browser developer tools can help find selectors: inspect a repeated card, identify its shared class, and test the selector in the browser console with document.querySelectorAll('your-selector').

Input parameters

Parameter	Type	Required	Description
startUrls	array	Yes	Up to 50 public static HTTP(S) pages with the same record structure.
recordSelector	string	Yes	CSS selector matching each repeated record.
fields	array	Yes	Between 1 and 50 field definitions.
maxItems	integer	No	Maximum saved rows across all pages; default 100, maximum 10,000.
includeInvalidRows	boolean	No	Keep rows with warnings; default true.
requestTimeoutSecs	integer	No	Timeout per page from 5 to 60 seconds; default 30.
sheetName	string	No	Excel worksheet name; default Web data.

Field names must begin with a letter and contain only letters, numbers, or underscores. They are used directly as workbook column names.

Example: extract website product data to Excel

{
  "startUrls": [{ "url": "https://books.toscrape.com/" }],
  "recordSelector": "article.product_pod",
  "fields": [
    {
      "name": "title",
      "selector": "h3 a",
      "source": "attribute",
      "attribute": "title",
      "type": "string",
      "required": true
    },
    {
      "name": "price",
      "selector": ".price_color",
      "type": "number",
      "pattern": "([0-9.]+)",
      "required": true
    },
    {
      "name": "productUrl",
      "selector": "h3 a",
      "source": "attribute",
      "attribute": "href",
      "type": "url",
      "required": true
    }
  ],
  "maxItems": 8,
  "sheetName": "Book prices"
}

Relative links are resolved against the final page URL. Numeric patterns remove currency symbols without putting them into the number cell.

Dataset output

A real row from the product example looks like this:

{
  "sourceUrl": "https://books.toscrape.com/",
  "recordIndex": 1,
  "data": {
    "title": "A Light in the Attic",
    "price": 51.77,
    "productUrl": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"
  },
  "valid": true,
  "validationWarnings": [],
  "scrapedAt": "2025-01-15T12:00:00.000Z"
}

Output field	Meaning
sourceUrl	Final URL used after safe redirects.
recordIndex	One-based record position on that page.
data	User-defined fields and normalized values.
valid	true when no field generated a warning.
validationWarnings	Field, message, and raw value for every validation problem.
scrapedAt	ISO time when the row was created.

The default dataset remains the automation-friendly result. The Excel workbook mirrors the configured field columns and adds source, validity, warning, and timestamp columns.

Excel workbook and report

Every successful run stores two named outputs in the default key-value store:

• OUTPUT.xlsx — workbook with frozen headers and filters;
• REPORT — JSON summary with source-level matched and saved counts.

Even a legitimate no-result run creates an empty workbook with the configured headers. This makes scheduled pipelines predictable.

The report distinguishes records matched by the CSS selector from rows actually saved. When includeInvalidRows is false, invalid rows can be matched but omitted.

Validation warnings

Warnings are attached to their row rather than hidden in logs.

Common messages include:

• Required value is missing;
• Value did not match the configured pattern;
• Value is not a valid number;
• Value is not an integer;
• Value is not a recognized boolean;
• Value is not a valid date;
• Value is not a valid HTTP(S) URL.

Start with includeInvalidRows: true while developing selectors. Filter the Excel valid column, then tighten the schema or disable invalid rows after reviewing edge cases.

How much does it cost to extract website data to Excel?

The Actor uses pay-per-event pricing:

• one start event per run;
• one item event for each dataset row produced;
• no separate charge for the XLSX workbook or JSON report.

At the current BRONZE rates, a run starts at $0.0005 plus $0.004 per saved row. For example:

Saved rows	Example total
10	$0.0405
100	$0.4005
1,000	$4.0005

Higher subscription tiers receive lower item prices. The Apify Console shows the applicable price before you start a run. Rows excluded by includeInvalidRows: false do not produce an item event.

Scheduling recurring exports

Create an Apify Task with tested selectors, then add a schedule.

A practical recurring workflow is:

1. run daily or weekly;
2. read the default dataset through the API;
3. compare keys meaningful to your source;
4. send new or changed rows downstream;
5. retain the workbook as a human-review artifact.

The Actor itself does not compare run history, send alerts, or discover new pages. Connect it to Apify integrations, Make, Zapier, a webhook, or your own data pipeline for those steps.

Integration patterns

Google Sheets or Microsoft Excel

Download OUTPUT.xlsx manually, or fetch the named key-value-store record after each run and upload it to your document storage.

Database and ETL

Read the default dataset as JSON. The stable envelope fields support lineage, while data contains your configured schema.

Data quality checks

Filter rows where valid is false. Route validationWarnings to an exception table instead of accepting silent blanks.

Multi-page consolidation

Supply pages that share the same HTML structure. The Actor applies one record selector and field schema to every URL, then creates one workbook.

Run with the Apify API

Replace <APIFY_TOKEN> with your token.

cURL

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~schema-guided-web-data-to-excel/runs?token=<APIFY_TOKEN>&waitForFinish=120" \
  -H "Content-Type: application/json" \
  -d @input.json

JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/schema-guided-web-data-to-excel').call({
  startUrls: [{ url: 'https://books.toscrape.com/' }],
  recordSelector: 'article.product_pod',
  fields: [
    { name: 'title', selector: 'h3 a', source: 'attribute', attribute: 'title', type: 'string', required: true },
    { name: 'price', selector: '.price_color', type: 'number', pattern: '([0-9.]+)', required: true }
  ],
  maxItems: 20
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python

from apify_client import ApifyClient

client = ApifyClient("<APIFY_TOKEN>")
run = client.actor("automation-lab/schema-guided-web-data-to-excel").call(run_input={
    "startUrls": [{"url": "https://books.toscrape.com/"}],
    "recordSelector": "article.product_pod",
    "fields": [
        {"name": "title", "selector": "h3 a", "source": "attribute", "attribute": "title", "type": "string", "required": True},
        {"name": "price", "selector": ".price_color", "type": "number", "pattern": "([0-9.]+)", "required": True}
    ],
    "maxItems": 20
})
items = client.dataset(run["defaultDatasetId"]).list_items().items
print(items)

Use with MCP and AI assistants

Add the Actor to Claude Code:

claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/schema-guided-web-data-to-excel"

Claude Desktop, Cursor, and VS Code setup

Use this MCP configuration in Claude Desktop, Cursor, or VS Code:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/schema-guided-web-data-to-excel"
    }
  }
}

Example prompts:

• “Run the web data to Excel Actor on this public catalog with .product-card records and extract title, numeric price, and absolute product URL.”
• “Combine these two static quote pages, join all .tag values, and give me the workbook link.”
• “Review rows with validation warnings and summarize which required selector failed most often.”

Always review selectors and target authorization before asking an assistant to run extraction.

Limits and failure behavior

• Only anonymous, public HTTP(S) URLs are accepted.
• Localhost, private-network, reserved, and credential-bearing URLs are rejected.
• Pages must return HTML and stay within the 5 MB response limit.
• Up to five safe redirects are followed.
• Transient request failures receive bounded retries.
• JavaScript rendering, clicks, login, CAPTCHA solving, proxy fallback, and pagination discovery are not included.
• All supplied pages must work with the same record selector and field schema.
• The Actor stops at maxItems saved rows across all pages.
• CSS selectors reflect the current website structure and may need updating after a redesign.

A blocked or non-HTML source fails the run instead of returning misleading empty rows. A valid page with zero selector matches succeeds and reports zero matched records.

Legality and responsible use

Only extract pages and data you are authorized to access.

Review the target website’s terms, robots guidance, copyright rules, privacy obligations, and applicable laws. Avoid collecting sensitive personal data, bypassing access controls, or sending excessive traffic. Keep schedules and page counts proportionate to the source.

This Actor does not provide legal advice. You remain responsible for your input URLs, selectors, storage, and downstream use.

Troubleshooting

Why did the run return zero rows?

Open the page without JavaScript and confirm that the repeated records exist in the original HTML. Test recordSelector in browser developer tools. A selector that matches rendered DOM added by JavaScript will not work here.

Why is a numeric field null?

Use a pattern that captures only the numeric portion, such as ([0-9.]+). Check validationWarnings.rawValue to see what the selector returned.

Why is a relative link different in output?

URL fields are resolved against the final page URL. This intentionally makes links usable outside the source page.

Why did the URL fail before downloading?

The Actor rejects non-public addresses and URLs containing embedded credentials. Host the page on an anonymously reachable public HTTP(S) endpoint or choose a different Actor designed for authenticated sources.

Where is the Excel file?

Open the run’s Output tab and choose Excel workbook, or download the OUTPUT.xlsx record from the default key-value store.

Related Automation Lab Actors

• CSV & Excel Data Quality Cleaner — normalize, validate, and deduplicate an existing CSV or XLSX table after extraction.
• Flexible HTTP Request Runner — execute public HTTP/API requests when you need response bodies and metadata rather than repeated HTML record extraction.

These Actors cover different stages: fetch structured responses, extract webpage records, then clean an existing tabular file.

FAQ

Can it scrape dynamic webpages?

No. It processes server-rendered static HTML. Choose a browser-based source-specific Actor for pages whose records appear only after JavaScript runs.

Can every URL use a different schema?

Not in one run. Group URLs by shared structure and create a separate Task for each selector/schema combination.

Does it infer fields from free-form prose?

No. The schema is explicit by design. Regex patterns can isolate values from text, but the Actor does not make semantic guesses.

Are invalid rows charged?

A row included in the dataset produces an item event. Set includeInvalidRows to false to omit warning-bearing rows while preserving matched/saved counts in the report.

Can I export JSON instead of Excel?

Yes. Every saved row is available in the default Apify dataset as JSON, CSV, XML, Excel, and other platform export formats. The named workbook is an additional ready-to-download artifact.

Does it crawl links or paginate automatically?

No. Supply each public page URL explicitly. This keeps scope and cost predictable.

How many pages and rows can I process?

A run accepts up to 50 URLs and 10,000 saved rows. Start small to verify selectors and respect the target website.