# Documentation Change Monitor — Markdown & RAG Updates (`apterlabs/documentation-change-monitor`) Actor

Monitor public documentation URLs for content changes. Get clean Markdown, exact diffs, stable document IDs and confirmed removal records for RAG updates. Persistent history; no external API key.

- **URL**: https://apify.com/apterlabs/documentation-change-monitor.md
- **Developed by:** [Apter Enterprises](https://apify.com/apterlabs) (community)
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$5.00 / 1,000 successful page checks

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

Keep a documentation index current without processing every unchanged page again. Give this Actor a list of public documentation URLs. The first run produces Markdown snapshots; subsequent runs produce updated content, exact line diffs and confirmed removal records.

Useful for support-bot knowledge bases, developer documentation watchlists and RAG ingestion workflows. No embedding provider, proxy subscription or external API key is needed. This Actor checks explicit URLs; it does not crawl a whole website or execute JavaScript.

### Quick start

```json
{
  "urls": [
    "https://docs.apify.com/platform/actors/running/actors-in-store",
    "https://docs.python.org/3/library/asyncio.html"
  ],
  "monitorName": "docs-watchlist",
  "includeUnchanged": false
}
```

Run once to save a baseline. Reuse `monitorName` on later runs to compare with that history. You can save a task and schedule it daily in Apify. Use a separate monitor name for each independent downstream index. The Actor itself does not create a schedule.

The default dataset contains page records plus one `record_type: "summary"` row. If nothing changed, the summary is still returned. The `CHANGES` key-value record contains only `operation: "upsert"` and `operation: "delete"` records; `SUMMARY` contains every page's status, errors and billing count.

### Output and integration

Each document has a stable `document_id` derived from its requested URL, a `content_hash` of its Markdown and an `event_id` for deduplicating a replay. An upsert includes the complete current Markdown. A content edit also includes a unified `diff` against the last successful snapshot. Diff text is capped at 50,000 characters and `diff_truncated` explicitly reports that limit; the current Markdown remains complete within the documented page limit.

| Change type | Operation | Meaning |
| --- | --- | --- |
| `baseline` | `upsert` | First observation in this monitor; not a claim that the page was newly published. |
| `changed` | `upsert` | Extracted Markdown changed. |
| `metadata_changed` | `upsert` | Title or final URL changed while the content hash stayed the same. |
| `reappeared` | `upsert` | A previously confirmed missing page is readable again. |
| `missing_pending` | `none` | HTTP 404/410 observed, awaiting confirmation. Retain your indexed content. |
| `removed` | `delete` | At least two HTTP 404/410 checks spanning six hours confirmed unavailability. |
| `unchanged` / `still_removed` | `none` | No update is required. Included as page rows only when `includeUnchanged` is true. |

Illustrative excerpt using **synthetic documentation**, not a real vendor change:

```json
{
  "record_type": "page",
  "change_type": "changed",
  "operation": "upsert",
  "url": "https://docs.example.org/quickstart",
  "title": "Quickstart",
  "markdown": "# Quickstart\n\nThe API now accepts a page_size parameter.",
  "diff": "--- previous.md\n+++ current.md\n@@ -1 +1 @@\n-The API accepts a limit parameter.\n+The API now accepts a page_size parameter.",
  "diff_truncated": false
}
```

For a RAG pipeline, process only rows whose `record_type` is `page`. On `upsert`, replace that document's prior chunks and store its new content hash; re-embed only when content changed. On `delete`, remove chunks belonging to that document ID. Ignore `operation: "none"`. Chunking, embeddings, vector-database writes and semantic assessments happen in your own pipeline. A textual change is not necessarily a breaking API change.

Consumers should check the run status and `SUMMARY`, process available successful results, and retry failed pages deliberately. The Actor never sends emails or writes to your vector database.

### Extraction

For HTML, the Actor looks for a unique `article`, `main`, `[role=main]`, `.document`, then `body`. You can choose an exact `contentSelector` and remove volatile content with `excludeSelectors`. Scripts, navigation, footers, sidebars, forms and buttons are excluded. Markdown conversion retains headings, links, lists, tables and code blocks; source layout can affect the result.

Markdown and UTF-8 plain-text responses are supported directly. CSS selectors cannot be applied to them. Sites that return a JavaScript shell, login page or access challenge fail the check instead of becoming a new baseline. Very short content is rejected. A page with at least 500 prior characters that shrinks by more than 80% is held for inspection; use a new monitor name to rebaseline if that shrinkage was intentional.

HTTP conditional requests use ETag or Last-Modified where available. A 304 counts as a successful unchanged check and reuses the retained snapshot. Content is still fetched and compared if the source does not support validators. Extraction is deterministic text comparison, not an AI judgment of meaning.

### Persistent history and removals

State is kept in a named store in your own Apify account. Runs using the same monitor are serialized with a queue lock. `stateStoreId` is optional and normally left blank. Changing content selectors, exclusions or the minimum content length requires a fresh monitor name so parsing changes cannot manufacture edits.

Removing a URL from your input does **not** delete its history or emit a deletion. A missing page must return 404 or 410 on at least two checks spanning six hours. Timeouts, 429s, server failures and rejected extractions do not advance that page's baseline. This reduces false removals but cannot prove that an upstream server's response is correct or permanent.

Up to 500 distinct URLs may be retained per monitor, subject to the state-size limit. There is no automatic expiration of page history. Deleting the named state store resets the baseline. State storage is retained in your account and follows Apify's storage/retention rules.

### Billing

One `page-check` event is charged per successfully processed URL at the price shown on the Store page. Baseline, changed, unchanged and valid 304 checks count. A 404/410 for an already tracked page also counts because it contributes removal evidence. A 404/410 with no prior snapshot fails and is not charged.

Failed, blocked, oversized or budget-skipped pages do not produce a page-check fee. There are no separate fees for Markdown, diffs, summary rows or change records. Successful pages in a mixed-failure run are charged and retain their results. Use Apify's maximum charge setting to cap billable checks.

Output, billing and state storage are separate operations. A crash can leave partial output, replay records or a charged check whose state update did not complete. Deduplicate by `event_id`; upserts using `document_id` should be idempotent. No exactly-once settlement guarantee is made.

### Supported scope and limits

- 1–50 explicit public HTTPS URLs per run, port 443, no credentials or custom authentication headers.
- Same-host redirects only; enter the final URL for cross-host redirects. URLs differing by query string are different documents; fragments are ignored.
- Public network destinations only. DNS is checked and the selected address pinned while TLS verifies the hostname.
- `robots.txt` is honored, including wildcard rules. Site requests are sequential with a minimum one-second spacing; crawl delays above 15 seconds are unsupported.
- Response cap: 3 MiB; extracted Markdown cap: 150,000 characters per page; retained state cap: 20 MiB. No silent truncation of a content snapshot.
- The Actor stops starting checks after eight minutes. Unfinished URLs are reported as skipped and the run fails after preserving successful results. Use smaller batches for slower sites.
- No browser rendering, PDF/OCR, private knowledge bases, historical changes from before your first run, or full-site discovery. Automatic extraction cannot identify every soft error page; review representative output before connecting an index.

Independent tool maintained by Apter Enterprises. No affiliation with the documentation publishers is implied. Use the Store Issues tab for a reproducible extraction problem without posting credentials or private content.

# Actor input Schema

## `urls` (type: `array`):

1–50 public HTTPS page URLs (port 443). No login, cookies, cross-host redirects or JavaScript rendering. Use final page URLs. Only these pages are checked; links are not crawled.

## `monitorName` (type: `string`):

Reuse the same name for persistent history. Use a new name for a fresh baseline. State is retained in your own Apify account. Omitted URLs keep their history.

## `includeUnchanged` (type: `boolean`):

Off: emit content updates, confirmed removals, pending-removal notices and a summary. On: also export unchanged page snapshots. No extra event fee.

## `contentSelector` (type: `string`):

Select exactly one main-content element, such as article or main. Blank uses article, main, role=main, .document or body. Changing extraction settings requires a new monitorName.

## `excludeSelectors` (type: `array`):

Remove elements inside the content before converting to Markdown, e.g. .last-updated. Navigation, scripts, forms and buttons are removed automatically.

## `minContentChars` (type: `integer`):

Reject shorter content as a possible JavaScript shell or failed extraction. Oversized pages and sudden shrinkage greater than 80% also retain the previous baseline.

## `stateStoreId` (type: `string`):

Usually leave blank: a named store is created automatically. Select only a DocDelta state store that belongs to this monitor.

## Actor input object example

```json
{
  "urls": [
    "https://docs.apify.com/platform/actors/running/actors-in-store"
  ],
  "monitorName": "main",
  "includeUnchanged": false,
  "minContentChars": 100
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

## `changes` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://docs.apify.com/platform/actors/running/actors-in-store"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("apterlabs/documentation-change-monitor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": ["https://docs.apify.com/platform/actors/running/actors-in-store"] }

# Run the Actor and wait for it to finish
run = client.actor("apterlabs/documentation-change-monitor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://docs.apify.com/platform/actors/running/actors-in-store"
  ]
}' |
apify call apterlabs/documentation-change-monitor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,apterlabs/documentation-change-monitor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/DYqAVMLdePXIZ4eLG/builds/I5pdzjvKRnBvzveX4/openapi.json