Chrome Extension Update Impact & Review Regression Intelligence

What this Actor does

This Actor does not simply scrape Chrome Web Store reviews. It compares public review quality before and after an extension update, then reports rating shifts, negative-review changes, emerging complaints, worsening or decreasing themes, feature requests, bounded evidence, coverage, and confidence.

Signals are observational. For example: “negative review signals increased after version 4.9.129 was published.” The Actor does not claim that a version caused a bug.

Who it is for

Browser-extension developers, QA and support teams, product managers, agencies, researchers, and teams evaluating extension quality.

Supported input

See sample-input.json for the Store-test default. extensions accepts one to five Chrome Web Store IDs or listing URLs. The default is Dark Reader. comparisonMode supports latest_update, custom_date, and recent_vs_previous; custom_date requires comparisonDate in YYYY-MM-DD format. Each period contains 3–30 calendar days, and the deterministic analysis sample is capped at 10–200 reviews per period.

language selects the Chrome Web Store rendered locale/feed. Review-language detection is not independently verified, so non-English runs may include LANGUAGE_FILTER_UNVERIFIED.

How comparison works

latest_update reads the current profile version and calendar update date. custom_date uses the supplied calendar date without claiming it is a published version. Both exclude the boundary day: with a July 15 boundary and seven-day windows, BEFORE is July 8–14 and AFTER is July 16–22. If the latest update is too recent, AFTER coverage is partial and the report includes AFTER_WINDOW_INCOMPLETE; it is not silently moved to another update. recent_vs_previous compares the two most recent complete calendar windows.

The collector uses the public profile over HTTP and the public rendered reviews page with bounded Playwright pagination. No Google login, CAPTCHA bypass, extension installation, source-code download, proxy, LLM, or user-supplied API key is required.

Output

One Dataset item is emitted per requested extension, including failed extension items so other targets can finish. The report includes the current extension identity, boundary, BEFORE/AFTER metrics, impact direction and level, theme deltas, feature requests, short evidence excerpts, coverage, warnings, deterministic summary, and runtime statistics.

Output fields

Field	Meaning
status	ok or failed report status.
recordType	Stable dataset record type.
extension.id	Chrome Web Store extension ID.
extension.name	Current listing name, when available.
extension.version	Current listing version, when available.
extension.updatedDate	Current listing update date, calendar precision.
boundary.effectiveMode	Comparison mode actually used after metadata fallback.
periods.before.reviewsAnalyzed	Deterministic BEFORE sample size.
periods.after.reviewsAnalyzed	Deterministic AFTER sample size.
periods.before.averageRating	Mean star rating in BEFORE.
periods.after.averageRating	Mean star rating in AFTER.
impact.direction	negative_shift, stable, positive_shift, or insufficient_data.
impact.level	low, medium, high, or unknown.
impact.ratingDelta	AFTER minus BEFORE average rating.
impact.negativeRateDelta	AFTER minus BEFORE share of 1–2 star reviews.
impact.confidence.score	Evidence-quality score from 0 to 1; not causality probability.
impact.confidence.level	low, medium, or high.
emergingThemes	New themes meeting bounded mention/share safeguards.
worseningThemes	Existing themes with a material share increase.
decreasingThemes	Themes with a material share decrease; not proof of improvement.
featureRequests	Bounded request titles, counts, and evidence.
coverage.before.coverageStatus	BEFORE full, partial, or insufficient.
coverage.after.coverageStatus	AFTER full, partial, or insufficient.
warnings	Collection, source, locale, sample, or coverage warnings.
stats.reviewsScanned	Reviews retained from bounded source pagination.
stats.reviewsAnalyzed	BEFORE plus AFTER analyzed samples.
stats.pagesFetched	Rendered review pages fetched.
stats.durationMs	Per-extension runtime.
summary	Deterministic, non-causal human-readable summary.
fetchedAt	Run timestamp.

Limitations and error semantics

Chrome Web Store exposes a current profile update date rather than a trustworthy historical release list. Review version attribution is not treated as reliable in V1, so comparisons are date-based. Calendar dates do not provide an exact rollout timestamp. Review IDs may be unavailable; in that case the Actor uses a deterministic SHA-256 content fallback. Source markup, localized labels, and review pagination can change; the report preserves partial coverage and warnings when bounded collection cannot reach the requested window.

Hard per-extension failures use bounded error codes such as PROFILE_UNAVAILABLE, REVIEWS_UNAVAILABLE, and NO_REVIEWS. Low samples still produce a report with insufficient_data impact when possible.

Cost and publishing

Playwright is used for review pagination, so the Actor is configured at 512 MiB. Pricing is unchanged by this implementation and was not modified automatically. The measured release gate is recorded in BENCHMARK_NOTES.md and docs/publish-readiness-report.md. This Actor is published in Apify Store under DEVELOPER_TOOLS; no pricing change is part of this release.

Local development

npm install
npm test
npm run build
npm run lint
npm run format:check
npm run validate:schema

The runtime entrypoint is src/main.js; use apify run --purge --input-file storage/key_value_stores/default/INPUT.json for a local Actor smoke. No external model is used.

FAQ

Does this prove that an update caused a bug?

No. It identifies changes in public review signals around a calendar boundary. Engineers should reproduce and verify individual reports.

What does insufficient_data mean?

Fewer than eight analyzed reviews were available for at least one period, or the source could not provide enough coverage. The report remains useful as a transparent collection result but should not drive a strong directional conclusion.

Does the Actor collect reviewer profiles?

No. It collects only bounded review fields needed for aggregate analysis and short evidence excerpts.

Benchmark results

Local and cloud results are recorded in BENCHMARK_NOTES.md. The companion sample-benchmark.json is a fixture-shaped benchmark record, while the publication decision is recorded in docs/publish-readiness-report.md.