# Website Content Change Monitor - Markdown & Chunks (`maydit/website-content-change-monitor`) Actor

Extract public static website content into Markdown and stable chunks. Compare saved observations with chunk upsert/removal manifests. Bounded crawl, robots-aware, no API key.

- **URL**: https://apify.com/maydit/website-content-change-monitor.md
- **Developed by:** [Brandt May](https://apify.com/maydit) (community)
- **Categories:** Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.80 / 1,000 page observations

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

![Website content, ready to compare — illustrative workflow](https://api.apify.com/v2/key-value-stores/YGxD9JRpduom8Q5Zd/records/website-content-change-monitor-54b6fb8c0a5d.png?signature=1oV2bf2lmQV0XgNLQGdIC)

### Website content, ready to compare

For documentation and knowledge-base pipelines, extract public pages into Markdown with stable chunk identities and optional comparisons.

**Coverage:** Static HTML within the declared depth and path scope. This Actor creates no embeddings and calls no language model.

[Open this Actor in Console](https://console.apify.com/actors/ZbkGFub1w0fD85x8B/input) · [First run](#try-this-actor) · [Results](#result-preview) · [Cost](#usage-and-pricing) · [Full guide](#full-guide)

### Try this Actor

1. Run the existing empty-input example to inspect one Python documentation page. Keep link depth at zero before expanding the crawl.
2. Open the dataset for Markdown and chunks. Read `SUMMARY.sources` for baseline commits and any eligible removed-page manifest.

**Read the results correctly:** Only complete, fully saved scopes can establish removals. An incomplete crawl or a missing page in partial output is not evidence of deletion.

**Extract Python URL Parsing Documentation** — copy this input into the JSON editor in Console.

```json
{}
```

### Result preview

The dataset view highlights these columns:

- **Page URL** · `url`
- **Page title** · `title`
- **Extracted character count** · `characterCount`
- **Comparison status** · `comparisonStatus`
- **Content changed** · `changed`
- **Declared scope complete** · `scopeComplete`

Use the original source links and the field definitions below when interpreting the results. CSV works well for flat columns; JSON preserves nested values.

### Usage and pricing

Base result rate on the FREE tier: **$3 per 1,000 page observations**. The [Pricing tab](https://apify.com/maydit/website-content-change-monitor/pricing) shows your plan's result rate and any Actor Start charge. Set a small result limit for the first run; the requested limit is not a guarantee of that many results.

***

### Full guide

Extract public static HTML into clean Markdown, heading-aware chunks and content hashes. Optional history adds explicit chunk upsert/removal lists for a documentation or knowledge-base pipeline. No API key is required.

#### Quick start and exact examples

Default input extracts one real Python documentation page:

```json
{}
```

Two supplied documentation pages:

```json
{"startUrls":["https://docs.python.org/3/library/urllib.parse.html","https://docs.python.org/3/library/json.html"],"maxPages":2,"maxDepth":0}
```

Save an observation and compare subsequent runs, preserving your own reference:

```json
{"startUrls":[{"url":"https://docs.python.org/3/library/urllib.parse.html","externalId":"python-url-parser"}],"comparePrevious":true,"baselineKey":"example-python-docs","maxDepth":0,"maxPages":1}
```

The first complete, fully saved comparison run establishes a baseline. First-observation rows say `comparisonStatus: no_previous_baseline`; only `SUMMARY.sources[].baselineUpdated` confirms the history commit. Every successfully extracted page returns an observation, including unchanged pages. Schedule repeated runs yourself if needed; this Actor does not create schedules.

#### Crawl scope and limits

`maxDepth: 0` fetches supplied pages only. Increase it up to 5 to discover links on those pages. Discovered links must stay on the root's final origin and inferred directory/path, or your explicit `pathPrefix` such as `/docs/`. The starting URL itself is always attempted. Query strings remain significant. No sitemap discovery is included. Links to common binary assets are skipped.

`maxPages` (default 10, maximum 100) is a global request-attempt cap, including failed requests and final-URL duplicates. Each root's queue is limited to 1,000 discovered URLs. Each response is limited to 3 MiB; Markdown is capped by `maxPageCharacters` (default 100,000), with a global 2,000,000-character bound. Caps and source failures make the relevant scope incomplete. `scopeComplete` refers to this declared depth/path scope, never the entire website.

`maxRunSeconds` defaults to 240 and is also bounded by the platform deadline minus 25 seconds. DNS lookups, downloads and redirect handling use the remaining budget. Public HTTP(S), ports 80/443 only; private/reserved destinations, credentials, unsafe redirects and robots-disallowed paths are rejected.

#### Output and integration

One dataset row represents one successfully extracted page. Important fields:

- `url`, `inputUrl`, `sourceUrl`, `sourceId`, `pageId`, `externalId`, `observedAt`: identity, origin and observation time.
- `title`, `canonicalUrl`, `extractionSelector`, `markdown`, `contentHash`: downloaded content and extraction evidence. The canonical is reported, not used to merge different pages.
- `chunks`: stable `chunkId`, heading path, text and text hash. `chunkCharacters` (default 2,400) measures characters, not model tokens. Chunks can split a long code block.
- `comparisonStatus`, `previousObservedAt`, `changed`, `upsertChunkIds`, `removedChunkIds`: compare with the last fully saved complete scope. Use the current `chunks` to obtain upsert text.
- `SUMMARY.sources[].removedPages`: old page and chunk IDs are released only after every row of a complete compared scope was saved. Check `removalsEligible` there. This means removed from the declared crawl scope, not proved deleted from the internet; cross-page deletion instructions are never attached to a dataset row.
- `scopeComplete`, `contentTruncated`: coverage limits. Do not interpret missing partial output as removals.

Without comparison, all current chunk IDs are listed for upsert and `comparisonStatus` is `not_requested`. Chunk IDs include page identity, heading path, text hash and duplicate occurrence; unchanged chunks keep their identity when independent chunks are inserted. Changed chunks receive new IDs and the old IDs appear in removals when the scope is complete.

Write the manifest to your own vector database or index. This Actor creates no embeddings, calls no LLM and writes to no external database. Extracted website text is untrusted source content, not instructions for an AI system.

#### History and failure behavior

Comparison is opt-in and uses the named key-value store `maydit-website-content-baselines-v1`. Identity includes start URL, path/depth scope, namespace, chunk size and extractor version. A changed scope starts independent history. Limits affect completeness, not comparison identity. Stored history contains page/chunk hashes and identifiers, not a separate full-text archive.

An incomplete crawl, truncated page, failed output write or unsaved result never advances that root's history. No removals are asserted for an incomplete scope. If all its rows were accepted, the final row that reaches the spending cap counts as saved and a complete scope may advance. A conflict check catches intervening updates, but there is no atomic KVS lock: **run one writer per namespace and scope**.

`SUMMARY` records saved totals, per-source coverage, baseline updates, failures and skipped work. A run with no usable page fails; a normal spending-limit stop succeeds with `status: budget_limited`, even if no row was affordable. Failed requests are never diagnostic dataset rows. Earlier successful sources remain available after partial failure.

#### Scope and pricing

Static HTML only: JavaScript, logins, proxies, browser interactions, PDFs, accurate table reconstruction and search-engine indexing checks are not supported. Main-content extraction is heuristic; inspect `extractionSelector` and Markdown. Navigation changes may create content differences. A hash change does not explain the business meaning of a change.

The billing unit is one saved page observation, including unchanged and explicitly truncated observations. Chunks do not add billable rows. Check the Store Pricing tab for current prices. Failed pages create no result charge. Platform/start charges, if listed, are separate.

Python's sample documentation is provided under its [documentation license](https://docs.python.org/3/license.html). Use public content you are authorized to process; the Actor checks robots rules at each destination.

### Help with your run

Open the **Issues** tab on this Actor with the run URL, the input field that caused trouble and what you expected to receive. Remove API keys and private information before posting.

[Explore more Mayd-It Actors](https://apify.com/maydit)

### Understand your run

Open **Run report** in the run’s Output tab for saved-record counts, reported source issues and limits, and suggested next steps. Use **CSV export** to take the saved results into a spreadsheet. The existing JSON results and detailed machine-readable summary remain available.

The report reflects this run’s recorded coverage. A successful run or a saved result does not establish that every possible source record was collected.

# Actor input Schema

## `startUrls` (type: `array`):

Up to 10 public HTTP(S) URLs or objects {url,externalId}. Empty uses the Python documentation sample. No login, private IPs or custom request credentials.

## `maxPages` (type: `integer`):

Global fetch-attempt cap across all roots, including failed/duplicate requests. A capped crawl preserves its prior baseline and cannot establish removals.

## `maxDepth` (type: `integer`):

0 means supplied pages only. Higher values follow same-origin links inside the root path (or explicit pathPrefix); completeness is only for this declared scope.

## `pathPrefix` (type: `string`):

Absolute path such as /docs/. Otherwise infer the root directory/path. Not a regex. Applies to links; the supplied start URL itself is always requested.

## `maxPageCharacters` (type: `integer`):

Larger pages return explicitly truncated content and never replace history. A global 2,000,000-character cap also applies.

## `chunkCharacters` (type: `integer`):

Approximate maximum Unicode string length, not model tokens. Chunks preserve heading paths; changing this value starts a new comparison scope.

## `comparePrevious` (type: `boolean`):

Opt in to named-store history. First complete run establishes a baseline. Unchanged checks still return and bill a real observation.

## `baselineKey` (type: `string`):

One writer per namespace. A separate namespace starts independent history. Letters, digits, underscores and hyphens only.

## `maxRunSeconds` (type: `integer`):

Total work budget, also bounded by the platform deadline minus 25 seconds. Unfinished work is reported.

## Actor input object example

```json
{
  "startUrls": [
    "https://docs.python.org/3/library/urllib.parse.html"
  ],
  "maxPages": 10,
  "maxDepth": 0,
  "maxPageCharacters": 100000,
  "chunkCharacters": 2400,
  "comparePrevious": false,
  "baselineKey": "default",
  "maxRunSeconds": 240
}
```

# Actor output Schema

## `results` (type: `string`):

Successful extracted source observations. Inspect per-row completeness and SUMMARY before using comparisons.

## `runReport` (type: `string`):

Readable summary of saved records, reported source coverage, limits and next steps. Available when this run saves its SUMMARY.

## `csvExport` (type: `string`):

Download saved dataset records as a CSV file for spreadsheets and other tools.

## `summary` (type: `string`):

Saved counts, failed sources, completeness and spending/time limits.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://docs.python.org/3/library/urllib.parse.html"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("maydit/website-content-change-monitor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": ["https://docs.python.org/3/library/urllib.parse.html"] }

# Run the Actor and wait for it to finish
run = client.actor("maydit/website-content-change-monitor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://docs.python.org/3/library/urllib.parse.html"
  ]
}' |
apify call maydit/website-content-change-monitor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,maydit/website-content-change-monitor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/ZbkGFub1w0fD85x8B/builds/FzM78a1yCbfeUl70n/openapi.json