# Changelog of Micro Center In-Store AI-Build Sniper (`pyralislabs/microcenter-ai-instore-sniper`) Actor

- **URL**: https://apify.com/pyralislabs/microcenter-ai-instore-sniper/changelog.md
- **Full Actor documentation**: https://apify.com/pyralislabs/microcenter-ai-instore-sniper.md

## Changelog

### \[1.7.5] - 2026-06-15

#### Revert the 40s warm-up — it was the regression

- 1.7.3's "uninterrupted 40s warm-up" was wrong: Cloudflare's challenge clears on the `/search/` page, not the homepage, so a long homepage wait just burns up to 40s per session before the productive page is even tried. 1.7.4 runs went 0/11 with it.
- **Reverted the homepage warm-up to a short 15s best-effort** (the config that previously got 10/11 + a build). It's an opportunistic cookie grab; the real clearance happens on the search page.
- **Internal deadline 200s→180s** for drain margin — 200s drained to ~240s (too close to the 300s cap); 180s lands ~210s.

### \[1.7.4] - 2026-06-15

#### Remove the open-box keyword fallback (the last big time-waster)

- Diagnostics on 1.7.3 runs showed the main crawl now clears Cloudflare and assembles a build in ~80s, but the **open-box keyword fallback** — a second `crawler.run()` — then ran **~145s**, grabbing cold sessions and re-fighting Cloudflare from scratch (4×403 + an SSL error), producing nothing and dragging the run to ~225s (near the 300s cap).
- **Removed it.** The open-box *facet* search already runs inside the main crawl on the warmed `cf_clearance` session; if it finds no open-box GPU we simply report none. No second crawl, no cold-session re-fight. Removed the now-dead `keyword_fallback` warning branch.
- Net: runs should finish in ~100–130s with builds, well clear of the cap.

### \[1.7.3] - 2026-06-15

#### Headful Cloudflare clearing VERIFIED on platform + speed fixes

- **Confirmed live: headful (1.7.2) clears Cloudflare's managed challenge.** Test runs obtained the `cf_clearance` cookie and produced real builds with 0 failed requests — the 0-build problem is solved. Remaining work was purely speed.
- **Run memory raised** (default 4096→8192, max 4096→16384, min 1024→4096). Headful's challenge solve is CPU-bound; 4 GB pinned one core and throttled the autoscaler to concurrency 1. 8 GB lifts it to 2–3 (verified). The daily auto-test uses the default.
- **Warm-up now solves the challenge UNINTERRUPTED** (clear-wait 15s→40s, goto→30s). It previously gave up after 15s and navigated to `/search/`, aborting Cloudflare's challenge JS mid-solve and forcing a restart — first-clearance dragged to ~100s.
- **Open-box keyword fallback gated to >90s budget left** (was 45s). It is a second `crawler.run()` that grabs cold sessions and re-fights Cloudflare (~97s wasted, 5×403, 0 results on June 15 02:46); it never contributes to builds.

### \[1.7.2] - 2026-06-15

#### Headful browser to actually clear Cloudflare (the real root cause)

- **Run logs revealed the true cause of the 0-build runs: headless Chromium cannot clear Cloudflare's managed challenge.** Every warm-up hit `waitForFunction: Timeout 30000ms exceeded`, 0/11 requests ran, 0 `cf_clearance` cookies across 8 exits. Two of my own bugs compounded it:
  - 1.7.0/1.7.1 passed the warm-up `waitForFunction` timeout as the **2nd** arg (the page-function argument) instead of the **3rd** (options), so Playwright silently used its 30s default and every un-cleared warm-up stalled a full 30s
  - 1.7.0/1.7.1 made warm-up failure **fatal** (retire + throw), so the crawler never reached the search pages at all; the original code "continued to search anyway" and got results
- **Fixes:**
  - Run the browser **headful** (`headless: false`) under the image's xvfb display — headful passes Cloudflare's managed challenge where headless does not (top-level `headless` overrides the platform's `APIFY_HEADLESS` env)
  - Warm-up is **non-fatal** again — best-effort `cf_clearance` collection that continues to the search on failure
  - `waitForFunction` timeout signature corrected; search-page selector wait 15s→25s so a challenge can clear there; internal deadline 210s→200s for drain margin
- See `docs/session-findings.md` §10.5

### \[1.7.1] - 2026-06-15

#### Resilience to bad residential-exit batches (follow-up to 1.7.0)

- **1.7.0's `maxPoolSize: 1` was an overcorrection.** It's perfect when the first residential exit is clean (verified run: 12/12, ~94s), but on a Cloudflare-flagged batch it hunts exits one at a time and makes zero progress — the no-storeId run (what the daily auto-test fires) finished 0/11 requests, 0 builds, though it still exited gracefully in 235s (the 210s deadline held)
- **Fix:** small pool instead of a pinned single exit — `maxPoolSize 1→8`, keep `maxUsageCount: 200`, add `retryOnBlocked: true`. Concurrency now tries several exits in parallel, the first clean one wins and is ridden for the whole run, bad exits retire fast
- Warm-up timeouts shortened (homepage goto 25s→12s, clear-check 15s→8s) so a flagged exit is discarded quickly; `maxRequestRetries 3→5` to allow more fresh-exit attempts within the deadline
- See `docs/session-findings.md` §10.2

### \[1.7.0] - 2026-06-15

#### Cloudflare-aware crawl + hard internal deadline (fixes the 300s auto-test timeout)

- **Root cause:** MC is behind Cloudflare's *managed* challenge. The old config rotated the residential exit every 4 requests (`maxUsageCount: 4`), so every new exit got re-challenged/403'd; the run ground past 300s and was killed **before any `pushData`** (empty dataset = auto-test failure)
- **One sticky exit per run:** `sessionPoolOptions.maxPoolSize: 1` + `maxUsageCount: 200` — solve the challenge once on warm-up, then ride that cf\_clearance cookie + IP for every search. A 403/challenge retires the session so the retry hunts a fresh exit; a working exit is never rotated away. Verified run: 11/11 first-try, zero 403s, ~96s total (was: timeout)
- **Warm-up now confirms it cleared** (waits past the "Just a moment" challenge page before hitting `/search/`); if an exit can't clear in 15s it retires and retries on a fresh IP
- **210s internal deadline:** `crawler.stop()` + handler no-op guarantee assemble/push/charge/exit run inside the 300s cap; open-box keyword fallback only fires when >45s budget remains
- Faster knobs now that we cooperate with Cloudflare: `maxRequestRetries 5→3`, `maxConcurrency 2→3`, `sameDomainDelaySecs 2→1`, search nav `domcontentloaded` (grid is present there)
- See `docs/session-findings.md` §10 for the full investigation

### \[1.6.0] - 2026-06-11

#### Per-record dataset output (output\_schema\_version 2.0.0)

- Each build is now its own dataset record (`record_type: "build"`); Micro Center additionally emits `open_box_deal` records; every run ends with one `run_summary` record; error records carry `record_type: "error"`
- Why: a single summary record made the monetization screen's per-result usage metric read ~$96/1k (one record per run) and starved the Output-tab table views; per-record output also makes CSV export and agent consumption natural
- Dataset views and output schema rewritten for the flat record shape; README and AGENTS.md updated

### \[1.5.0] - 2026-06-11

#### Final pre-launch polish (from the clean 3m43s platform run)

- GPU targets refreshed: discontinued 40-series cards (RTX 4080 Super / RTX 4080) replaced with current-gen RTX 5070 Ti / RTX 5060 Ti across goals (RTX 4090 kept for LLM\_TRAINING — used stock lingers and empty searches are now cheap)
- Default run memory 2048 → 4096 MB — the platform log showed CPU pinned at 2 GB, capping concurrency at 1; 4 GB roughly halves wall time at near-neutral compute cost
- README: new "What to expect from a run" section (runtime, pacing rationale, proxy + cost expectations, zero-result and open-box semantics)
- Log strings de-dashed (em-dashes rendered as `?` in the platform console)

### \[1.4.1] - 2026-06-11

#### Proxy cost fix (the $5 free-plan burn)

- **Images/media/fonts are no longer fetched** — page assets through residential proxy cost ~$1/run (0.13 GB × $8/GB) and consumed most of the monthly platform credit in one afternoon; DOM-text parsing needs none of it (~10× bandwidth cut, faster runs)
- Failed warm-up navigations now settle on about:blank so a pending chrome-error page can't interrupt the next request (MC)

### \[1.4.0] - 2026-06-11

#### Runtime fix after first successful platform run (zero 403s, but 6.6 min > 5-min auto-test limit)

- **Zero-result searches are now a valid empty answer, not a retried failure** — the discontinued `RTX 4070 Ti Super` query burned ~2 min in 5 retries (each with a session warm-up) because the no-results page has no product grid
- Replaced the discontinued `RTX 4070 Ti Super` GPU target with `RTX 5070 Ti` (shared goal config, both actors)
- Session reuse capped at 4 requests (platform log proved residential sessions reuse fine; single-use would double runtime via warm-ups)

### \[1.3.0] - 2026-06-11

#### Platform 403 hotfix

- **Force Apify RESIDENTIAL (US) proxy** unless the user explicitly picks proxy groups — the first platform run showed Micro Center hard-blocks datacenter IPs (every request 403'd instantly)
- **Session warm-up**: each fresh session visits the homepage before its first search request, collecting anti-bot cookies (cold hits on /search/ trigger 403s)
- Request pacing: `sameDomainDelaySecs: 2`, `maxRequestRetries: 5`, handler timeout 120s
- Input schema: proxy prefill now `RESIDENTIAL`, description documents the datacenter block

### \[1.2.0] - 2026-06-11

#### Pre-publish hardening + component data enrichment (competitor-parity pass)

- SSD search target and capacity floor now follow the AI goal (`LLM_TRAINING` → 4TB NVMe, others 2TB) via shared goal config
- Defensive `data-price` parsing (strips locale formatting before parseFloat)
- README: Newegg-vs-Micro-Center comparison table
- Opus improvement plan fully audited and retired: remaining Phase C ideas (multi-store fan-out, pickup-ETA enrichment, open-box freshness tracking, open-box PPE event) moved to the ROADMAP enhancement queue; open-box deals stay deliberately unbilled (rare post-title-guard + a free differentiator)
- **GPU market-drift fallback:** when no GPU clears the goal's budget ratio (2026 street prices for RTX 5080 exceed the 55% cap at a $2,000 budget), the cap relaxes toward the full budget instead of returning zero builds, with an explicit warning
- **Open-box precision guard:** clearance-facet results are only emitted as open-box deals when the listing title actually says "open box" — Micro Center's facet can fall through to regular search results, which previously risked mislabeling new items as open-box
- Env-gated `DEBUG_DUMP_PRODUCTS=1` sample logging for production diagnosis
- `sku` and `availability` (stock text) parsed from listing cards now actually ship on every component record (previously parsed but dropped during build assembly)
- Open-box `condition` field rides along on open-box GPU components
- README: removed private-actor link, issue reports now point to the Apify Store Issues tab

### \[1.1.0] - 2026-06-10

#### Pre-launch hardening

- Dataset output schema added (per-build and open-box-deal table views + run summary) and wired into actor metadata
- High request-failure-rate warning (≥25% failed requests) surfaced in output `warnings`, recommending Residential proxies
- Local-run warning when `ACTOR_TEST_PAY_PER_EVENT` is unset, so PPE billing is never silently skipped during testing
- Error records now include `incomplete_build_notice` stating nothing was charged
- Store categories added to actor metadata; cross-links to companion PyralisLabs actors in README

### \[1.0.0] - 2026-05-08

#### Added

- Initial release: AI-optimized PC build scouting from Micro Center
- Store-specific pricing via cookie mechanism
- Open-box GPU deal discovery
- Pay-per-Event pricing at $0.05/build
- GPU VRAM filtering with numeric minimum
- NPU CPU filtering for Intel Core Ultra and AMD Ryzen AI
- Socket-compatible motherboard matching (LGA 1851 / AM5)
- Pre-built PC/laptop exclusion filter
- GPU fallback chain for out-of-stock targets
- Quality-gated PPE charging
- Residential proxy enforcement for anti-bot defense
- Crawl stats in output for agent observability
- MCP tool compatibility via Apify MCP Server