# CleanMeta Crawler (`stefano_seggio/primer-actor`) Actor

Clean, structured page metadata in seconds: title, description, canonical, Open Graph, H1, word count. Built-in retries and pagination. Ready for SEO audits and LLM/RAG pipelines - pay only per result, never per wasted run.

- **URL**: https://apify.com/stefano\_seggio/primer-actor.md
- **Developed by:** [Stefano Seggio](https://apify.com/stefano_seggio) (community)
- **Categories:** SEO tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.50 / 1,000 extracted results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## CleanMeta Crawler — Apify Store Overview

**Actor:** `stefano_seggio/primer-actor` | **Actor ID:** `U9fUBHDngX6IyjzzF` | **Version:** 1.1
**Store URL:** https://apify.com/stefano\_seggio/primer-actor

***

### Executive Summary & Business Use Case

CleanMeta Crawler turns a list of caller-supplied start URLs into clean, structured page metadata: `<title>`, meta description, canonical URL, Open Graph title/image, HTML language, first H1, approximate word count and HTTP status code. It is built on Crawlee with a Cheerio (static HTML) crawler, and it works against **any website the caller points it at** — there is no fixed target site or vertical; it follows same-hostname links and, when a `paginationSelector` is supplied, paginated listing pages, up to a configurable request cap. In short: give it a URL, it hands back the seven fields that actually matter from that page and every page it discovers from there, already parsed and typed, instead of a blob of raw HTML someone still has to write a parser for.

Three concrete business use cases the data directly supports:

1. **SEO audits.** An SEO consultant or in-house marketing team points the crawler at their own site (or a competitor's) to find missing/duplicated `<title>` tags, meta descriptions and canonical URLs across hundreds of pages in one run — the exact class of technical-SEO defect that costs organic ranking and is otherwise found by clicking through pages by hand.
2. **LLM / RAG ingestion pipelines.** A team building a retrieval or agent pipeline needs `{title, description, h1}` per URL as lightweight, pre-parsed context instead of shipping raw HTML into a model and burning tokens on markup and boilerplate the model has to strip out itself.
3. **Social preview / site-health monitoring.** A content or growth team runs this on a schedule against their own domain to catch broken or stale `og:title`/`og:image` tags before a broken social card ships to production, with `statusCode` per crawled page surfacing dead internal links as a free side effect of the same run.

This grounding comes directly from the actor's own README ("Built for" section) — there is no monetary-value, tender-amount, or listing-price field anywhere in this actor's schema, so no pricing- or procurement-style use case is claimed here; the actor is a page-metadata extractor, not a registry monitor.

### Technical Features & V2 Architecture Highlights

CleanMeta Crawler shipped a **cross-run, per-URL change-detection layer** in v1.1.0 (2026-09-08), described in the actor's CHANGELOG.md and AGENTS.md. Two points are specific to this actor and should not be assumed to generalize from other actors in the same developer's portfolio:

- **Named key-value-store persistence, not the run-scoped default store.** State — one content fingerprint per URL, `{ entries: { [url]: { contentHash, lastSeenAt } }, lastRunAt }` under key `DELTA_STATE` — is persisted in an explicitly **named** key-value store, `primer-actor-delta-state`, opened via `Actor.openKeyValueStore('primer-actor-delta-state')`. This matters because the actor's own CHANGELOG (v1.1.1 fix entry) documents that the first attempt at this used `Actor.getValue()`/`Actor.setValue()`, which are shortcuts for the store associated with the *current run only* — real cloud verification (two separate runs against the same URLs) caught that every page was misclassified `NEW_URL` on the second run because state never actually carried over. The named-store fix is what makes delta state genuinely survive across separate runs, which is what makes this actor usable on a schedule.
- **Domain-specific event set — not the fleet's generic four-state taxonomy.** The `eventType` field takes exactly three values, per `dataset_schema.json`'s own enum and description: `NEW_URL` (first time this exact URL was scraped), `CONTENT_CHANGED` (extracted metadata differs from the last scrape of that URL), and `UNCHANGED`. There is **no `CLOSED` or `STATUS_CHANGE` event**, and the actor's own AGENTS.md is explicit about why: this actor crawls whichever URLs the caller supplies each run rather than discovering listings from an enumerable registry (the way a government-tenders portal has a walkable list of active tenders), so there is no trustworthy way to say a URL has "closed" or "disappeared" — a URL simply not appearing in one run's crawl could just as easily mean it wasn't linked from this run's start URLs, or sat past the `maxRequestsPerCrawl` cutoff.
- **`onlyChanged` input flag**, per its own `input_schema.json` description: when enabled, "a page is still crawled (its links are still followed) but is only added to the dataset — and charged — if it's the first time this exact URL has been scraped, or its extracted metadata differs from the last time this Actor scraped it. Pages with unchanged metadata are skipped." Every visited URL still gets its fingerprint updated in the delta store regardless of this flag, because the next run needs an accurate fingerprint for every visited URL — not just the ones actually delivered — to classify correctly.
- **Content fingerprint scope.** The hash used to detect `CONTENT_CHANGED` covers only the SEO-relevant extracted fields (title, meta description, canonical URL, OG title, OG image, language, H1, word count) — it deliberately excludes `statusCode`, `crawlDepth` and `scrapedAt`, none of which describe page content itself.
- **No 18-field Unified Metadata Schema claim.** Unlike this developer's registry-monitoring fleet (which shares a documented 18-field base envelope across a dual-floor delta engine), this actor's own schema files document a different, smaller field set (15 dataset fields total) and a deliberately different, domain-honest delta model — this document only claims what `dataset_schema.json` and the actor's own docs actually state.
- **Resilience:** automatic retries (`maxRequestRetries: 4`) with session rotation on suspected blocks, same-hostname link discovery, and a hard `maxRequestsPerCrawl` cap checked after every result so a run stops cleanly at the budget the caller set rather than continuing to burn compute.

### Input Schema & JSON Configuration Example

Fields exactly as declared in `.actor/input_schema.json`:

| Field | Type | Default | Description |
|---|---|---|---|
| `startUrls` | array | `[{"url": "https://apify.com"}]` (prefill) | URLs to start crawling from. At least one is required - the Actor charges an actor-start event just for running, so an empty list would bill the caller for zero output. |
| `maxRequestsPerCrawl` | integer | `100` | Hard limit on how many pages this run will fetch, across start URLs, discovered same-site links and pagination. |
| `paginationSelector` | string | *(none)* | Optional CSS selector for a next-page link, e.g. a\[rel=next] or .pagination .next. When set, the crawler follows it up to Max pagination depth pages per start URL, on top of normal same-site link discovery. |
| `maxPaginationDepth` | integer | `3` | Maximum number of paginated pages to follow per start URL when Pagination selector is set. Ignored otherwise. |
| `proxyConfiguration` | object | `{"useApifyProxy": true}` (prefill) | Proxies used to fetch pages. Apify Proxy (datacenter) is recommended as the default. |
| `onlyChanged` | boolean | `false` | When enabled, a page is still crawled (its links are still followed) but is only added to the dataset - and charged - if it's the first time this exact URL has been scraped, or its extracted metadata differs from the last time this Actor scraped it. Pages with unchanged metadata are skipped. Useful for scheduled re-runs where you only want to pay for what's new or different since the last run. |

`startUrls` carries `minItems: 1` in the schema, so at least one start URL must be supplied on every run.

A valid JSON input example (exercising pagination and delta mode, all real field names):

```json
{
    "startUrls": [
        { "url": "https://crawlee.dev" },
        { "url": "https://crawlee.dev/blog" }
    ],
    "maxRequestsPerCrawl": 50,
    "paginationSelector": "a[rel=next]",
    "maxPaginationDepth": 3,
    "proxyConfiguration": {
        "useApifyProxy": true
    },
    "onlyChanged": true
}
```

### Output Dataset Sample & Data Dictionary

Fields exactly as declared in `.actor/dataset_schema.json`:

| Field | Type | Description |
|---|---|---|
| `url` | string | Page URL |
| `title` | string | Page title tag |
| `metaDescription` | string or null | meta name=description content |
| `canonicalUrl` | string or null | link rel=canonical href |
| `ogTitle` | string or null | og:title meta content |
| `ogImage` | string or null | og:image meta content |
| `language` | string or null | html lang attribute |
| `h1` | string or null | First h1 text |
| `wordCount` | integer | Approximate visible body word count |
| `statusCode` | integer or null | HTTP status code of the response |
| `crawlDepth` | integer | Link-hops from the nearest start URL |
| `scrapedAt` | string | ISO timestamp of extraction |
| `eventType` | string (enum: `NEW_URL`, `CONTENT_CHANGED`, `UNCHANGED`) | NEW\_URL if this exact URL was never scraped before, CONTENT\_CHANGED if its metadata differs from the last scrape, UNCHANGED otherwise. No status/closure concept applies - this Actor crawls caller-supplied URLs, not a discoverable listing registry. |
| `contentHash` | string | Fingerprint over the page's extracted content fields, used to detect CONTENT\_CHANGED across runs. |
| `previousScrapedAt` | string or null | scrapedAt from the last time this URL was scraped, or null if this is the first time (NEW\_URL). |

A realistic example dataset record (field names are real, from `dataset_schema.json`; values are illustrative):

```json
{
  "url": "https://example.com/blog/post",
  "title": "How We Cut Page Load Time by 40%",
  "metaDescription": "A breakdown of the changes that moved the needle.",
  "canonicalUrl": "https://example.com/blog/post",
  "ogTitle": "How We Cut Page Load Time by 40%",
  "ogImage": "https://example.com/og/post.png",
  "language": "en",
  "h1": "How We Cut Page Load Time by 40%",
  "wordCount": 1284,
  "statusCode": 200,
  "crawlDepth": 1,
  "scrapedAt": "2026-09-08T11:04:27.177Z",
  "eventType": "CONTENT_CHANGED",
  "contentHash": "3f9a1c2b8e7d4f0a1b2c3d4e5f60718293a4b5c",
  "previousScrapedAt": "2026-09-01T09:12:03.501Z"
}
```

Note: a request that permanently fails after retries is recorded as its own dataset item with `url`, `error` and `failedAtRetry` fields instead of being silently dropped (documented in README.md; these fields are not part of the successful-page schema above).

### Multi-language Integration Snippets

#### cURL

```bash
curl -X POST "https://api.apify.com/v2/acts/stefano_seggio~primer-actor/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
        "startUrls": [{ "url": "https://crawlee.dev" }],
        "maxRequestsPerCrawl": 20,
        "onlyChanged": false
    }'
```

#### Python (apify-client)

```python
from apify_client import ApifyClient

client = ApifyClient("<APIFY_TOKEN>")

items = client.actor("stefano_seggio/primer-actor").call(run_input={
    "startUrls": [{"url": "https://crawlee.dev"}],
    "maxRequestsPerCrawl": 20,
    "onlyChanged": False,
})

for item in client.dataset(items["defaultDatasetId"]).iterate_items():
    print(item["url"], item["title"], item["wordCount"], item["eventType"])
```

#### Node.js (apify-client)

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('stefano_seggio/primer-actor').call({
    startUrls: [{ url: 'https://crawlee.dev' }],
    maxRequestsPerCrawl: 20,
    onlyChanged: false,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

### Pricing Model Explanation

CleanMeta Crawler runs on Apify's Pay-Per-Event (PPE) model with two named events, on a pricing scale that is independently set from the rest of this developer's actor fleet:

| Event | Price | What triggers it |
|---|---|---|
| `result` | $0.0005 per event ($0.50 per 1,000 results) | Charged once per dataset item actually delivered — one successfully crawled and extracted page. |
| `apify-actor-start` | $0.00005 per event | Charged once per run, simply for the Actor executing — this is why `startUrls` requires at least one entry: an empty run would otherwise bill the caller for zero output. |

There is only **one** result tier here — unlike fleet actors that split pricing across two different tiers of result (e.g. a lighter listing-only event vs. a fuller detail-fetch event), CleanMeta Crawler's `result` event is charged identically for every delivered page regardless of how it was reached (start URL, same-site link discovery, or pagination). The actor's own CHANGELOG (v1.1.0, "Not added and why") is explicit that the v1.1 delta feature deliberately did **not** introduce a second pricing tier: `onlyChanged` changes *how many* of the existing `result` events get charged, not the price or structure of the event itself, and Apify's rule allowing only one "significant pricing change" per Actor per month made a same-release tier addition a deliberate future decision rather than something bundled in here.

**What `onlyChanged` actually does to billing:** with `onlyChanged: true`, a page classified `UNCHANGED` (its extracted metadata is identical to the last time this Actor scraped that same URL) is not added to the dataset at all — it is simply never pushed, and therefore never generates a `result` event to charge for. This is not "billed at $0" or a discounted event; an unchanged page produces no chargeable event whatsoever. The page is still fetched and its links are still followed (so link discovery and pagination coverage are unaffected by enabling this flag) — only the delivery-and-charge step is skipped for that one URL. This makes `onlyChanged: true` the cheapest way to re-run an SEO audit or social-preview check on a schedule: the caller pays the flat `apify-actor-start` fee plus `result` only for pages that are new or that actually changed since the last run.

***

*Sources: `.actor/actor.json`, `.actor/input_schema.json`, `.actor/dataset_schema.json`, `.actor/output_schema.json`, `README.md`, `CHANGELOG.md`, `AGENTS.md` — all read directly from `C:\Users\Stef\apify-portfolio\primer-actor` on 2026-09-08.*

# Actor input Schema

## `startUrls` (type: `array`):

URLs to start crawling from. At least one is required - the Actor charges an actor-start event just for running, so an empty list would bill the caller for zero output.

## `maxRequestsPerCrawl` (type: `integer`):

Hard limit on how many pages this run will fetch, across start URLs, discovered same-site links and pagination.

## `paginationSelector` (type: `string`):

Optional CSS selector for a next-page link, e.g. a\[rel=next] or .pagination .next. When set, the crawler follows it up to Max pagination depth pages per start URL, on top of normal same-site link discovery.

## `maxPaginationDepth` (type: `integer`):

Maximum number of paginated pages to follow per start URL when Pagination selector is set. Ignored otherwise.

## `proxyConfiguration` (type: `object`):

Proxies used to fetch pages. Apify Proxy (datacenter) is recommended as the default.

## `onlyChanged` (type: `boolean`):

When enabled, a page is still crawled (its links are still followed) but is only added to the dataset - and charged - if it's the first time this exact URL has been scraped, or its extracted metadata differs from the last time this Actor scraped it. Pages with unchanged metadata are skipped. Useful for scheduled re-runs where you only want to pay for what's new or different since the last run.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://apify.com"
    }
  ],
  "maxRequestsPerCrawl": 100,
  "maxPaginationDepth": 3,
  "proxyConfiguration": {
    "useApifyProxy": true
  },
  "onlyChanged": false
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://apify.com"
        }
    ],
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("stefano_seggio/primer-actor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [{ "url": "https://apify.com" }],
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("stefano_seggio/primer-actor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://apify.com"
    }
  ],
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call stefano_seggio/primer-actor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,stefano_seggio/primer-actor"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/U9fUBHDngX6IyjzzF/builds/b0pgX8rH0eYPfJj1u/openapi.json