# Changelog of AI Search Visibility Tracker: ChatGPT, Perplexity & Gemini (`emoubarak/ai-search-visibility-tracker`) Actor - **URL**: https://apify.com/emoubarak/ai-search-visibility-tracker/changelog.md - **Full Actor documentation**: https://apify.com/emoubarak/ai-search-visibility-tracker.md ## Changelog All notable changes to this Actor are documented here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### \[0.4.11] — 2026-08-09 Gaps found by reading Apify's own publishing docs. Listing only. #### Added - **`.actor/key_value_store_schema.json`**, declared in `actor.json` under `storages.keyValueStore`. The quality score's *congruency* category names the key-value store schema explicitly alongside the input and dataset schemas, and it was the one this Actor did not have — the `SUMMARY` record the output schema links to was undeclared. Verified after deploy that `Actor.setValue('SUMMARY')` still writes cleanly under the schema's `application/json` content type. #### Changed - README images renamed from `hero.png` / `variance.png` to `ai-search-visibility-tracker-hero.png` and `ai-answer-variance-measurement.png`. Apify's README guide asks for descriptive, SEO-friendly file names and warns against generic ones; the first pass shipped exactly the generic names it warns about. - Both images are now wrapped in links to the Actor page, which the same guide recommends. #### Notes Two things checked and deliberately not done: - **No status badge.** It renders live `runs: N, users: N`. At two users that argues against the Actor on its own page. Worth adding once the numbers help. - **No `llms.txt` to write.** Apify serves one at `apify.com/llms.txt`, but it lists Apify's own pages only — no community Actor appears in it, ours or a competitor's. The LLM-facing surface is instead the per-Actor markdown at `apify.com//.md`, which is generated from this README. It strips images, so the variance evidence is carried by the markdown table rather than the chart alone — which it is. That same page currently reports **`0.0% runs succeeded`** to any model reading it: `publicActorRunStats30Days` is 5 TIMED-OUT and 0 succeeded, all of them the quality tests that failed before 0.4.7. Owner runs do not count toward it, so it clears only as the automated tests start passing. ### \[0.4.9] — 2026-08-09 Store listing and discoverability. One real bug fixed, no behaviour change. #### Fixed - **The MCP connection URL in the README was a 404.** It advertised `mcp.apify.com?tools=emoubarak/ai-brand-monitor`, but the published Actor slug is `ai-search-visibility-tracker` — verified against the API, which returns 404 for the former and 200 for the latter. Every agent that followed the README failed to connect, on the one section aimed squarely at agent traffic. Root cause was `.actor/actor.json` still carrying `"name": "ai-brand-monitor"` from before the Store rename. That is now aligned, which also removes a standing hazard: a bare `apify push` resolved that name to *no* Actor and would have created a second, duplicate listing rather than updating this one. - `test/docs.test.ts` gained a guard pinning every Apify URL in the README to the slug in `actor.json`, so the two cannot drift apart again. #### Added - **A measured variance example replaces the hypothetical one.** The "Why one query per prompt is not a measurement" section previously argued from an invented 2-of-5 example. It now shows a real run — `JuXrVfwWxMSfwaas5`, 2 prompts x 5 samples x 4 engines, 40/40 collected, 0 errors — where the commercial prompt was perfectly stable (5/5 on every engine) and the informational one was not (4/5, 4/5, 1/5, 0/5). The point that argument had been missing: the true mention rate there is 45%, and a one-sample audit across four engines can only ever return 0, 25, 50 or 75%. **45% is unreachable in principle**, and a flat "brand invisible" 0% comes up in 3.2% of the 5⁴ possible draws. Both figures are exact enumerations, not simulations. Keeping the stable prompt in the example is deliberate: sampling is not uniformly necessary, it is unequally necessary, and which prompts need it is not knowable until they have been sampled. - Two images, the first on this listing: a hero and the variance chart, served from the public key-value store `ai-visibility-tracker-assets`. - An explicit positioning line against the subscription platforms buyers compare this against (Profound, Otterly, AthenaHQ). #### Changed - Store categories moved from `AI, SEO_TOOLS, AGENTS` to `SEO_TOOLS, MARKETING, MCP_SERVERS` (applied via the API; categories are not a field `actor.json` carries). Apify caps an Actor at three. "AI" is already carried by the title, SEO title and description, so the text search finds it either way, whereas `MARKETING` (1,385 Actors) and `MCP_SERVERS` (1,687) can only be reached through the category filter — against `AI` at 7,622 and `AGENTS` at 2,902. ### \[0.4.7] — 2026-08-09 Fixes the Apify automated quality test, which had flagged the Actor as "under maintenance" after three consecutive days of failing runs. Input schema only — no code change. (Builds 0.4.5 and 0.4.6 were pushed on 2026-08-05 without a changelog entry and carried no behaviour change; the platform build counter had moved on, so this entry is numbered to match the build it ships in rather than to follow 0.4.4.) #### Fixed - **`maxPrompts` and `samplesPerPrompt` now carry a `prefill` of 1.** Neither field had one, so the automated test — which runs the Actor with the input schema's prefilled values — fell through to the Zod defaults of 10 and 3. Combined with the four prefilled engines that is 10 x 3 x 4 = **120 downstream Actor calls**, which at `MAX_CONCURRENCY = 5` and the ~60-100 s a downstream call takes is 24 waves, or 35-50 minutes. The test requires `Succeeded` with a non-empty dataset within **5 minutes**, so the default run could never pass it. Measured on real runs of build 0.4.4 (2026-08-05), which is where those per-wave figures come from: | Run | Samples | Duration | | ------------------- | ------- | -------- | | `SWc8fiUpAOx2pYygf` | 6 | 2 m 34 s | | `DpBVbEzl0JlwKMzhr` | 12 | 5 m 10 s | | `TSpxik0PwVCm1SF11` | 18 | 8 m 24 s | | `H0eEQNeBmGeFmihvO` | 30 | 8 m 49 s | The prefilled run is now 1 x 1 x 4 = **4 samples**, a single concurrency wave, ~2 minutes end to end. All four engines stay prefilled: multi-engine coverage is the Actor's headline claim, so the demo run has to show it. The `default` values are deliberately left at 10 and 3. A prefill is the starting point of a Store form, not the recommended configuration, and `samplesPerPrompt: 1` is explicitly not a measurement — both field descriptions now say so and point at 3 as the floor for a run worth acting on. ### \[0.4.4] — 2026-08-05 Documentation correction. No code change. #### Fixed - The 0.4.3 notes claimed Apify has no multi-example mechanism and that the competitors' "examples" were README content. **Both were wrong.** The claim came from a page fetch whose markdown conversion dropped the tab bar; the raw HTML shows an Examples tab on `apify/google-search-scraper` and on the competitor, and none on ours. The real mechanism: the Examples tab is backed by Apify **Tasks**. A task appears there once `publicConfig.publishedAt` is set. Creating tasks is API-writable and is now done — four exist, one per entry in `.actor/examples.json`. Setting `publicConfig` is not: the API accepts it and stores `null`, and no publish route exists in the OpenAPI spec, so publishing is a Console step. `exampleRunInput` is a separate, older field that prefills the Input tab rather than the Examples tab. Corrected in `.actor/examples.json`, `OPERATIONS.md` and the 0.4.3 entry, with the four task IDs and the exact Console path recorded. ### \[0.4.3] — 2026-08-05 Ready-to-run examples. Listing content only. #### Added - `.actor/examples.json` — four input examples, one per differentiator: the full audit, a cheap Google-AI-Overviews-only weekly watch, a genuinely localized French run, and collect-only. Each carries its BRONZE-tier price. - README **Recipes** section rendering all four, and the first one published to the Actor's `exampleRunInput`. - `test/examples.test.ts` — validates every example against `ActorInputSchema`, recomputes its advertised price from `pay_per_event.json` and fails above 15% drift, asserts each appears in the README, keeps the localized example under $2.50, and rejects an example that lists the tracked brand as its own competitor. #### Changed - The Actor page's Examples tab is backed by Apify **Tasks**: a task whose `publicConfig.publishedAt` is set appears as an example. Creating the task is API-writable; publishing it is Console-only (the API nulls `publicConfig` and exposes no publish route). Four tasks were created from this file; publishing them is a Console step. `exampleRunInput` is a separate, older field that prefills the Input tab, and carries example 1. - The French example's competitor list was `["Axonaut", "Sellsy"]` — it listed the tracked brand as its own competitor. Now `["Axonaut", "noCRM.io"]`, both confirmed by a real French AI Overview that names "Axonaut, Sellsy et noCRM.io". Prompts reduced from 5 to 4 to keep the run under $2.50, since localized Google costs ~13x the default engine. - README no longer claims an exact unit-test count. The guard could not verify it: `it.each` blocks expand at runtime, so a source-level count is always wrong. A precise number that cannot be checked mechanically should not be published. ### \[0.4.2] — 2026-08-05 Listing content only. No code, schema or pricing change. #### Added - **"Use from an AI agent (MCP)"** section near the top of the README: the `https://mcp.apify.com?tools=emoubarak/ai-brand-monitor` connection URL, three example prompts a user can say to their agent, and the agent-relevant facts (pay-per-event, limited permissions, no Standby, collect-only) with the two cost anchors taken from the existing table — $0.99 for 5 prompts x 2 samples across four engines, $0.43 for a Google AI Overviews-only check. The `?tools=` query-parameter convention was checked against the current Apify MCP documentation rather than assumed. One claim was corrected in the process: the docs state the MCP server only accepts unauthenticated requests for discovery tools, so running an Actor does require authorising Apify once in the browser. The section says "no OpenRouter key" rather than "no API key". #### Changed - The "Can AI agents run this Actor?" FAQ entry now links to that section instead of restating it. ### \[0.4.1] — 2026-08-05 Listing consistency fix. No behaviour change. #### Fixed - The published README contradicted itself: the opening said "No API key required" while the input section still carried the pre-BYOK paragraph "The OpenRouter key is yours … without a key every metric that depends on analysis is empty". The second was false as of 0.4.0. Root cause was a stale README in the repo, not a stale build — the live build already matched HEAD. - Input table was missing `analysisMode` entirely, and described `analysisModel` as accepting "any OpenRouter model ID", which is only true under BYOK. - SUMMARY example showed `schemaVersion: 1` and omitted the v2 fields, so it contradicted the webhook section that tells consumers to branch on version 2. - Stale test count in "Run it locally". #### Added - `test/docs.test.ts` — a drift guard that fails the suite when the README's input table diverges from `input_schema.json`, when it names an event absent from `pay_per_event.json` or omits a billable one, when the SUMMARY example falls behind the schema version, or when the documented test count goes stale. - A one-line price anchor near the top: a 3-sample Google AI Overviews prompt costs about $0.027, which is comparable at a glance against the per-check prices competitors display. ### \[0.4.0] — 2026-08-05 Analysis key included by default, collect-only mode, per-engine cost breakdown. #### Added - **Included analysis key.** `ANALYSIS_OPENROUTER_KEY` (secret Actor env var) scores answers when the user supplies none, so the Actor now runs with **no API key required** — the last onboarding step is gone and it is callable by an agent with nothing to configure. `openRouterApiKey` becomes optional BYOK: the user then pays OpenRouter directly, may use any model, and is billed the reduced `response-analyzed-byok` event (~1/3 price). `SUMMARY.analysisKeySource` reports which was used. - **Model allowlist on the included key** — `openai/gpt-5.6-luna` (default) and `deepseek/deepseek-v4-flash-0731`, both validated with real calls against `AnalysisResultSchema` at ~$0.0002/call. Anything else falls back to the default with a warning, so one input field cannot point the owner's key at an expensive model. Two candidates were rejected by that validation: `openai/gpt-5-nano` returned an empty completion (reasoning tokens consumed its budget) and `openai/gpt-4.1-nano` returned schema-valid but wrong extractions. - **Per-run analysis spend ceiling**, mirroring the downstream circuit breaker. On trip, remaining answers are collected but left unscored. - **`analysisMode: "collect-only"`** — no LLM call, no analysis charge, raw answers and citations only. Collection metrics and their deltas are still computed; analysis-derived metrics are absent rather than zero. - **`SUMMARY.costByEngine`** — samples, billable samples, estimated cost, cost share and mention share per engine, for deciding which engines earn their place. #### Changed - **`SUMMARY.schemaVersion` is now `2`.** SUMMARY is a webhook payload, so its shape is a contract; consumers should branch on this field. Version 2 adds `analysisMode`, `analysisKeySource` and `costByEngine`, and makes analysis-derived metrics absent in collect-only mode. - **`report-generated` raised** from $0.08 to $0.20 (FREE), with proportional tiers ($0.125 BRONZE). The aggregation and delta layer is the differentiated value; samples stay cheap. A default run moves +$0.12 at FREE, +$0.075 at BRONZE — $2.64 to $2.72. - A sample with no analysis is no longer marked `analysis_failed` when the run is in collect-only mode; leaving it `ok` keeps the coverage report meaningful. ### \[0.3.0] — 2026-08-05 Fourth engine, webhooks, and a listing rewritten around what buyers actually search for. #### Added - **Gemini** as a fourth answer engine, via `apify/gemini-search-scraper` — the official Apify Actor, same `{ queries }` input and same sectioned-source output shape as ChatGPT Search, so it reuses that parser. Measured unit cost $0.00505 (run `gePTBPiX5pFk8wSo5`), billed as `prompt-sampled-gemini` at $0.015 BRONZE. `engines` now defaults to all four. - **Optional webhook.** Set `webhookUrl` and the finished summary — metrics, coverage and deltas — is POSTed there as JSON at end of run: 10s timeout, two retries, 4xx not retried. Failures are warned about and never fail the run. Deliberately **not** billed: it is a retention feature, not a metered one. - README sections answering the queries buyers actually search: how to track brand mentions in AI search, how to improve brand visibility in AI search engines, and what to check before buying an AI search monitoring tool. Plus a worked example of why one query per prompt is not a measurement, and a cost-per-check table so the price can be compared against per-check tools. #### Changed - Actor title and short description rewritten from keyword data, leading with "AI Search Visibility Tracker" and covering AI brand monitoring. Title kept under Apify's 63-character limit. - Run cost examples recomputed for four engines. ### \[0.2.0] — 2026-08-05 Production hardening. Billing model, per-engine pricing and real market targeting. #### Changed — billing (breaking for the operator, invisible to users) - Downstream engine Actors are now called with the **developer's** Apify token (`DOWNSTREAM_APIFY_TOKEN`, a secret environment variable) instead of the token injected into the user's run. An Actor started via the API runs under the account of the token used, so the previous behaviour billed users at retail for the engine scrapers *on top of* this Actor's pay-per-event charges. A run without the token now fails at startup rather than silently double-charging. See OPERATIONS.md for the cash-flow and concurrency exposure this creates. - `prompt-sampled` is replaced by four per-engine events: `prompt-sampled-google-aio` (primary), `prompt-sampled-google-aio-localized`, `prompt-sampled-perplexity` and `prompt-sampled-chatgpt`. One Perplexity call costs ~4x one Google AI Overview and a localized one ~13x; a single blended price made cheap-engine users subsidize expensive ones. Every price is 3x the measured downstream cost, floor $0.005. #### Added - **Market targeting for Google AI Overviews.** Any market other than `US` is served by `johnvc/Google-AI-Overview-API`, which accepts Google's own `gl` and `hl` parameters. Verified: 10/10 French queries and 3/3 German queries returned localized AI Overviews with local sources. The official Apify Actor exposes only `queries` and cannot localize; Perplexity and ChatGPT Search expose no country parameter at all. - `geoApplied` on every sample row, plus `sourceActor`, so a user can tell which answers were genuinely localized instead of inferring it. A run warns explicitly about engines that could not honour the requested market. - `maxTotalChargeUsd` on every downstream call, plus a per-run circuit breaker (`src/budget.ts`) ceilinged at `samples x worst-case unit cost x 1.5`. On trip, collection stops and the run reports partial coverage. - `OPERATIONS.md`: billing exposure, measured costs with run IDs, monitoring to configure in Console, and the deploy procedure. #### Fixed - The downstream circuit breaker under-counted spend by ~170x. `run.usageTotalUsd` is not settled when `.call()` returns — it holds only the $0.00005 actor-start charge, with the per-result event landing moments later. The budget now charges `max(reported, worst-case unit cost)`. - Answer analysis was truncated at `max_tokens: 800`, which silently lost samples whose answers named many brands: the JSON came back cut off mid-object and failed validation twice. Raised to 2000, and a truncated completion now reports itself as truncated instead of as malformed JSON. ### \[0.1.0] — 2026-08-04 Initial release. #### Added - Sampling across three answer engines via the official Apify Actors: `apify/perplexity-search-scraper`, `apify/google-ai-overviews-scraper` and `apify/chatgpt-search-scraper`. Every prompt is asked `samplesPerPrompt` times per engine, with bounded concurrency (max 5), 3 retries with exponential backoff and a per-call timeout. - Stratified prompt generation across commercial, informational, transactional, brand-defensive and comparison intents, from one LLM call. User-supplied `queries` bypass generation entirely. - Structured answer analysis via OpenRouter (default `openai/gpt-5.6-luna`): brand mention, position, sentiment with evidence, competitors named, and which brand the answer recommends. Invalid JSON is retried once with the validation error fed back into the prompt. - Metrics per engine and overall: answer trigger rate, mention rate, visibility rate, average position, share of voice, recommendation rate, sentiment distribution and top cited domains — each with its denominator. - Week-over-week deltas from a named key-value store (`ai-brand-monitor-state`, keyed by brand + market), including new and lost cited domains. - Pay-per-event monetization: `prompt-sampled`, `response-analyzed` and `report-generated`. Failed engine calls are not charged. Collection stops gracefully when the run's spending limit is reached. - Graceful partial failure throughout: engine errors, analysis failures, a missing OpenRouter key and an exhausted budget all degrade the output and add a warning instead of aborting the run. - 103 unit tests (Vitest) covering metrics, schemas, delta logic, adapters against a mocked ApifyClient, analysis retry behaviour and the collection layer. No test touches the network. - `npm run local` for a fully mocked end-to-end run at zero cost. #### Known limitations at 0.1.0 - No market targeting (addressed in 0.2.0). - A single blended `prompt-sampled` event across all engines (addressed in 0.2.0).