# Changelog of AI Multimodal Analyzer — powered by Gemini (`badr_xba/ai-multimodal-analyzer-gemini`) Actor - **URL**: https://apify.com/badr_xba/ai-multimodal-analyzer-gemini/changelog.md - **Full Actor documentation**: https://apify.com/badr_xba/ai-multimodal-analyzer-gemini.md ## Changelog ### 2026-11-10 (rejected API key: same behaviour in the Claude, OpenAI and Gemini Actors; local, not pushed) - A rejected key (HTTP 401, HTTP 400 `API_KEY_INVALID` ("API key not valid", expired key) / 403 `PERMISSION_DENIED`) is a configuration error of the run: it stops after the FIRST such error (no further AI call; until the key has answered once, items are sent one at a time so a bad key costs one request whatever `concurrency`), every item not analysed goes to `FAILED_INPUTS` with one shared code `invalidApiKey` (or `apiKeyNoAccess` for 403 / permission) and a fix hint (where to create/check the key, the "Gemini API key" field, nothing billed), `RUN_SUMMARY` gets `stoppedReason` + `stoppedHint` + `apiKeyError`, no dataset row, and the run ends FAILED via `Actor.fail` with the status "Stopped: Gemini rejected the API key — nothing was analysed or billed. ..." (schedules and integrations notice a broken key). Only Apify's run start event is charged. - Before: Google's real answer to a wrong key (HTTP 400 INVALID_ARGUMENT "API key not valid", reason API_KEY_INVALID) was classified `invalidRequest` (an input error), so the run ended SUCCEEDED with 0 rows and a generic "Done" message; 401/403 used code `auth` + `skipped`. - Unchanged: 429 and 5xx are retried and never treated as key errors; a missing key still gives the successful estimate-only demo run. - Shared wording in `src/api_key_errors.py` (identical in the three Actors except the provider constants). New tests: 401, 403, stop after the first call (concurrency 1 and 5), 429 not a key error, Gemini 400 API_KEY_INVALID / expired / PERMISSION_DENIED. ### 2026-11-10 (fixes from real Gemini API tests; local, not pushed) - Fix: task `detect_key_info` was refused by Gemini on every item (HTTP 400 INVALID_ARGUMENT "Request contains an invalid argument", real test with gemini-3.5-flash-lite). Every keyword of its schema is in Google's supported subset; the difference with the tasks that work was `"maxItems": 60` on `result.facts` (array of objects with a 10-value `type` enum), which Gemini compiles into a too-large decoding automaton. `maxItems` removed from `facts`; the prompt asks for at most 60 facts and the list is capped at 60 after the answer (warning when facts are dropped). - New test: the exact `response_json_schema` sent for every task (and the chunk / compare calls) is checked against Google's documented responseJsonSchema subset (keywords, enum only for strings/numbers and non-empty, `$ref` without siblings, `required` names existing properties) + arrays of objects <= 30 items. - Fix: task `compare` numbered the sources in completion order (the second text could become "Item 1"). Rows are now compared in INPUT order: prompt labels "Item N", `comparedSources` and the model's `item` numbers all follow the input. Done message: "2 of 2 item(s) analysed and compared (3 rows: 2 items + 1 comparison)" instead of "3 of 2 item(s) analysed"; RUN_SUMMARY `itemsAnalysed` counts items only, new `rowsSaved`. - Checked (no code change): a local `datasetIds: ["gta_local"]` run exits 1 because crawlee 1.11 rejects the storage name `gta_local` (names: a-z, 0-9, hyphens in the middle). Same layout with `gta-local` (`datasets/gta-local/000000001.json`... + `__metadata__.json`) reads fine. On Apify the dataset is opened by id with force_cloud. ### 2026-11-09 (billed tokens of failed items; local, not pushed) - Fix: an item that failed after Google processed it (`maxTokens`, `blocked` refusal/safety, cut stream, `providerNoResponse`/errors after partial data, budget stop mid-chunks) was shown as "0 input + 0 output tokens, est. $0.0000" although Google bills those tokens. Every attempt that reported usage is now recorded once per item, at the price of the model that answered it (fallback and audio rates included), and added to the run totals, RUN_SUMMARY and status/done messages; the FAILED_INPUTS entry gets `tokens` and `costUsd` (omitted when nothing was reported). Successful rows also include a cut attempt made before a fallback. Managed mode: the shown tokens are the ones charged per attempt; no double counting across retries/fallback. ### 2026-11-09 (level fallback model; local, not pushed) - New model `gemini-3.6-flash` (tier `36-flash`, same prices as 3.8-flash incl. the 2027-01-01 increase; capabilities copied from 3.8-flash, not re-verified on the models page; `managedEnabled: false`, proposed events only). - `levels.standard.fallback = ["gemini-3.6-flash"]`: own key + no explicit `model` only. On `providerNoResponse`, HTTP 503 or HTTP 429 RESOURCE_EXHAUSTED from the level model, the item is retried once on the fallback (no explicit cache, thinking level kept when supported). Rows get `model` = model used, `requestedModel`, `fallbackReason`; cost at the fallback's prices; RUN_SUMMARY `fallbacks`/`fallbackModels`; status and final messages mention it. If the fallback fails, the primary's code is kept with a note. Once the primary is skipped after 2 no-responses, later items go straight to the fallback. Managed mode: no fallback. - After the first real test (198 s): with a fallback available for the item, the level model gets NO same-model retry on no-response / 503 / 429 (straight to the fallback); the single retry stays when there is no fallback. `firstResponseTimeoutSecs` default 90 -> 60. Done message and RUN_SUMMARY name the model(s) that actually answered (`model`, `requestedModel`, `modelsUsed`), e.g. "analysed with gemini-3.6-flash (fallback for gemini-3.8-flash)". ### 2026-11-09 (first-response timeout; local, not pushed) - Fix: `gemini-3.8-flash` (free tier) accepted requests but sent no bytes, and an item hung > 12 min (per-call timeout sized for max output, repeated by retries). - New input `firstResponseTimeoutSecs` (default 90, 15–600): a streamed call with no chunk at all (no text, thinking or usage) within it is aborted; calls already streaming keep the full per-call timeout. - Such a call fails as `providerNoResponse` after at most ONE retry (none in managed mode: nothing reported, nothing charged). After 2 consecutive no-responses on the model, remaining items are skipped with `providerNoResponse` (status message says so); other items' results still make the run SUCCEEDED. ### 2026-11-09 (empty-input demo + own-key fee by level; local, not pushed) #### Fixed - A run without any source (no startUrls/texts/documentUrls/imageUrls/mediaUrls/datasetIds), e.g. Apify's automatic Store test with `{}` or defaults only, no longer ends with "Nothing to analyse": it uses the demo sources = the `prefill` of those fields in `.actor/input_schema.json` (read at runtime; `src/demo.py` keeps a fallback checked against the schema by a test), logs "No source given: running the demo input" and continues. Without a key it is an estimate-only run: SUCCEEDED, 1 row, nothing billed. Runs with explicit sources are unchanged. #### Changed (owner decision 2026-11-09) - Own-key fee scales with the analysis level: one event `item-analyzed` ($0.0015) charged with count 1 (quick), 2 (standard), 4 (deep) per analysed item = $0.0015 / $0.003 / $0.006. Never charged for errors, refusals, estimates or skipped items. README launch text ("free ... no fee of its own") replaced; PRICING.md and models.json (`item-analyzed` $0.002 -> $0.0015) updated. The input schema `startUrls` prefill (a large Wikipedia page) is replaced by a short `texts` prefill so the demo stays cheap. ### 0.2.3 — 2026-11-09 (commercial safety, local, not published) Audio pricing from 0.2.2 (audio rates, managed audio refusal on `gemini-3.1-flash-lite`) is unchanged. #### Fixed (owner-loss risks from the finance audit) - **No free managed usage:** managed runs are refused before any AI call unless the run is pay-per-event (`ChargingManager.get_pricing_info().is_pay_per_event`), every event of the model is in the run's live `per_event_prices` with a price > 0, and each live token price covers Google's highest price for the model (any `from` date, long context) × 1.625. The proposed prices in `models.json` are no longer used for any charging decision (`Biller.event_price` returns live prices only; a missing price makes the reservation refuse: `missingEventPrice`). `charge_room()` returns 0 (refuse) when the max total charge cannot be read. - **Per-model `managedEnabled`** flag in `models.json` (opt-in: missing or `false` = managed refused for that model; validated by `load_catalog`). - **Timeouts:** generation is streamed (`generate_content_stream`), the usage metadata is kept as chunks arrive, so a cut-off call still reports and charges its tokens. Per-call timeout ≥ `maxOutputTokens` / 20 tok/s + 120 s. In managed mode a timeout / network drop is retried only when the failed attempt's usage was reported and fully charged. SDK automatic continuation switched off (each continuation hop is a separately billed request that could exceed `max_output_tokens`). - **Exact input in managed mode:** every managed call is sized with `count_tokens` over the same parts (inline media, Files API uploads, YouTube) plus the system instruction and schema as text parts (CountTokensConfig refuses `system_instruction` on the Gemini API); a failed count means no call (`countUnavailable`). Managed URL context is refused (fetched pages cannot be counted or bounded below the context window). - **Context cache:** no explicit cache in managed mode (owner-paid storage, no event bills it, may survive an aborted run). Own-key storage estimate now uses the price in effect today (`price_on`) instead of the last (future) price entry. - **Redaction:** the per-item crash traceback is logged redacted (was `Actor.log.exception`, raw); `main()` catches every uncaught error, logs a redacted traceback (customer key, managed key and key-looking unknown-field values scrubbed) and fails the run without the SDK's raw traceback. Unknown input fields holding key-looking values get a warning that names the field, never the value. - `split_pdf` with 0 / negative pages per chunk produced empty PDFs; now one page per chunk. #### Tests - 168 → 202 (line coverage 84 % → 87 %; billing 95 %, main 92 %, provider 91 %): live-price fallbacks, charge_room branches, missing-event / not-PPE / `managedEnabled` / URL-context refusals, exact-count reservation, count failure, charge limit reached mid-run (fake Actor applies the SDK's limit arithmetic), managed timeout retry rules, partial-stream charging, Files API polling (PROCESSING → ACTIVE, FAILED, 5-minute timeout, upload error), PDF split edge cases, item and run crash paths redacted, unknown key-field warning, double-charge checks in the "never raises" billing tests. ### 0.2.1 — 2026-11-09 (model audit, local, not published) #### Fixed - `gemini-3.1-flash-lite` cached-input price: was $0.25 (worst-case placeholder, "not confirmed"); the official pricing page (aistudio.google.com/docs/pricing, read 2026-11-09) lists $0.025 (text/image/video; $0.05 audio). Added `shutdownDate: 2027-05-07` from aistudio.google.com/docs/deprecations (replacement `gemini-3.5-flash-lite`). README price table and QUALITY.md updated. - `gemini-4-argon` `defaultThinkingLevel`: was null; aistudio.google.com/docs/thinking lists "On (medium)". Informational field (not read by the code). #### Known, not fixed - `gemini-3.1-flash-lite` audio input is $0.50/1M (cache $0.05), twice the text rate; the code prices all input at one rate, so cost estimates for audio on this model are 2x low. Managed billing charges `input-tokens-1k-31-flash-lite` at $0.0005/1k = $0.50/1M, which covers Google's audio cost at zero margin. ### 0.2 — 2026-11-09 (local build, not published) #### Security - DNS rebinding closed: each hop's host is resolved once, every answer must be public, and the connection is pinned to that exact IP (`PinnedIPTransport`: URL rewritten to the IP, original host kept in the `Host` header and as the httpx/httpcore `sni_hostname` extension, so TLS SNI and certificate hostname verification still use the real name; TLS verification unchanged). Keep-alive off so a TLS connection is never reused across host names; proxies from the environment ignored (`trust_env=False`). Same `blockedAddress` code, redirect and size caps. #### Removed - Google Search grounding. Google's terms for Grounding with Google Search forbid storing/caching grounded results and Search Suggestions and require showing them unmodified only to the end user who sent the prompt; an Actor stores every result in a dataset. The `useGoogleSearch` input is gone (a run that still sends `true` is refused with a clear message before any AI call), the Google Search tool is never sent, and `searchSuggestionsHtml` / `webSearchQueries` are no longer stored or shown in the report. `groundingSources` now lists URL context retrievals only. The grounding price was removed from `models.json` and PRICING. - URL context is kept: neither the Gemini API Additional Terms nor the URL context guide set a storage or display restriction on it. ### 0.1 — 2026-11-09 (local build, not published) #### Added - Gemini analysis of web pages, texts, PDF/DOCX/TXT/MD/CSV documents, images, audio/video files, public YouTube links and rows of other Apify datasets. - Tasks: summarize, extract (customer JSON Schema, validated up front and checked after the answer), classify (labels with model-reported confidence and quoted basis), analyze, detect_key_info, compare (+ comparison row), report, custom. - Analysis levels quick / standard / deep mapped in `src/models.json` (gemini-3.5-flash-lite, gemini-3.8-flash, gemini-4-argon), model override, date-effective prices (gemini-4-argon from 2026-12-01, gemini-3.8-flash from 2027-01-01). - Cost guards: estimateOnly (automatic without a key), map-reduce over long texts and PDFs with explicit `truncated` flag, video clip length, max output/total tokens, max estimated cost, managed-mode ceilings and max-charge check before every call. - Key modes: own key (flat `item-analyzed` event) and managed key (per-1k-token events per tier, paid Apify users only, key from an Actor secret env var). - Optional URL context and Google Search grounding (own key), explicit context cache for long shared instructions, Files API for large media. - Visual report (kit 1.1.0), RUN_SUMMARY, FAILED_INPUTS, review invitation on successful runs, secret redaction. - SSRF guard: http(s) only, private/loopback/link-local/metadata/unique-local addresses refused after DNS resolution and on every redirect hop (max 5), also for URLs handed to Gemini (YouTube, URL context); one billing tier (pair of token events) per model. - Google Search grounding: Search Suggestions kept unmodified (`searchSuggestionsHtml`, `webSearchQueries`) and shown next to the grounded answer in the report (Gemini API terms); YouTube items label their image "YouTube thumbnail" (`imageLabel`). - Local mock mode (`MOCK_PROVIDER=1`, ignored on the Apify platform).