# Changelog of Quora Questions & Answers Scraper (`devilscrapes/quora-questions-scraper`) Actor - **URL**: https://apify.com/devilscrapes/quora-questions-scraper/changelog.md - **Full Actor documentation**: https://apify.com/devilscrapes/quora-questions-scraper.md ## Quora Questions & Answers Scraper — Changelog ### 0.2.3 — 2026-08-31 Cloud QA on build `0.2.2` reached `SUCCEEDED` with 3 real rows (run `5A12dw3zifEfsU6R5`) — the first genuinely working cloud run — but eyeballing the dataset found two silent data-quality bugs the parser tests hadn't caught because the earlier fixture's structure was simplified in a way that hid them: - **`answer_author_name` shipped as `""` on every row.** Each answer card actually carries TWO `a[href*="/profile/"]` links — a document-order- first avatar-only link with empty text, then the name-bearing link. `css_first` always grabbed the avatar. New `_author_link_with_name` prefers whichever profile link actually has visible text. - **`answer_upvotes` was `None` on every row.** The upvote button does NOT live inside the same ancestor as the profile link — it sits TWO levels further out. `_answer_card_ancestor` required only a profile link, so it returned a scope too small to ever contain the upvote button. It now requires BOTH a profile link and the upvote button before accepting an ancestor as the answer's card. - `tests/fixtures/question_page.real_structure.html` rebuilt to actually reproduce both structural facts (two profile links, upvote button two levels out) instead of the simplified single-level card the previous fixture used — which is exactly why these two bugs passed local tests but failed on real data. New assertions lock in the real author name and upvote count. 67 tests green, ruff clean, pyright clean. ### 0.2.0 — 2026-08-31 Root cause of the 0/0 rows that closed out 2026-08-30's investigation: `0.1.3`'s cloud run (`P4dbLhqPt6uFChk1l`) confirmed Camoufox clears Quora's Cloudflare challenge, but the JSON-LD-primary parser design in design.md turned out to be a wrong guess — it just failed silently instead of loudly (question parsed via H1, `suggestedAnswer` empty, 0 answer rows, run correctly failed loud on REQ-6 rather than reporting a hollow success). Ground-truthed against a REAL page this time (`src/recon_capture.py`, a one-off Camoufox cloud capture persisted to KVS — see `docs/specs/quora-questions-scraper/notes.md`): - **No JSON-LD, no `__NEXT_DATA__`, no usable data island exists on the real render.** `window.ansFrontendGlobals` only carries page config, never question/answer content. The question and every answer are plain DOM. - **Quora's own `puppeteer_test_*` classes are the stable target** — test hooks the site ships for its own end-to-end tests, present on the question title (`puppeteer_test_question_title`) and every answer body (`puppeteer_test_answer_content`), unlike the surrounding CSS-in-JS classes (`s1e38h8g` etc.) which are hashed and unusable. `src/parser.py`'s extraction priority is now: test-id CSS (primary, ground-truthed) → JSON-LD (secondary, kept for resilience) → bare H1 - guessed selectors (last resort, unchanged from before). - **Two real bugs the guessed selectors had baked in, both found from the recon capture:** 1. The "Continue Reading" gate is a CSS toggle, not a real content gate — Quora ships a short preview AND the complete answer text in the same DOM node, joined by that marker. The old code always kept the text *before* the marker and flagged it truncated; that discarded the real content on every answer that had it. `_split_truncated` now prefers the text *after* the marker, and only falls back to "before + truncated=True" when nothing follows (the genuinely login-gated case, still covered by the existing `question_page_gated.synthetic.html` fixture). 2. The upvote count renders as a digit-rolling ticker: a same-length `visibility--hidden` ghost span (a width placeholder) plus the real visible span, both inside the same button. Reading all `` text concatenated them into a garbled number (`9953` instead of the real `53`, confirmed against the recon capture). `_extract_upvotes` now skips the hidden span. - **Fault isolation, not misattribution.** Author/timestamp/upvotes are read from the nearest ancestor (within `ANSWER_CARD_SEARCH_DEPTH = 6`) that links to an author profile. An unbounded walk on the real recon page mis-attributed a "related answers" section's author to an unrelated answer further up the page; bounding the search means that case now correctly returns `None` for those fields while still emitting the real answer text — a partial row, never a dropped one. - New fixture `tests/fixtures/question_page.real_structure.html` models the real recon structure (not a byte-for-byte copy — trimmed for repo size) and 3 new tests in `tests/test_parser.py` cover the primary path, the full-text-after-marker behaviour, and the bounded-ancestor-search fault isolation. 66 tests green (was 63), ruff clean, pyright clean. - `.actor/pay_per_event.json` (`actor-start: $0.20`, `result-scraped: $0.002` → $2.20/1,000) is unchanged and already correct; still a publish-time activation on the Console Monetization tab, out of this fix's scope. **Second bug, found by cloud QA on build `0.2.1` (run `JeBdVCVejlO6YWFvC`, FAILED):** Camoufox cleared the Cloudflare challenge (`status=200`) but the parser STILL found no title — a genuine hydration race, not a selector bug. `page.content()` was captured right after `wait_until="domcontentloaded"` + a fixed 1.5s sleep; Quora's Relay app (10+ webpack chunks) doesn't always finish rendering the question/answer DOM in that window, even though the same wait pattern happened to work during the manual recon capture. `src/browser.py::fetch_page` now waits (best-effort, `h1` marker, 8s cap, never raises) for the page to actually hydrate before reading `content()` — `_wait_for_hydration`. A timeout just means proceed with whatever DOM exists; the parser layer still decides whether that's enough. 1 new test (`test_fetch_page_swallows_hydration_timeout_and_still_returns_html`, 67 total). Pushed as build `0.2.2`. ### 0.1.1 — 2026-08-30 Root cause of all 12/12 cloud FAILED runs, found by reading the actual platform run logs (`P4oObkxiIk9z7OdHw`, then a fresh reproduction on build 0.0.2 — run `ncnbKI3rjyndmAav5`) instead of trusting the 2026-08-27 UNPROVEN writeup: - **The 08-27 residential + Camoufox-escalation fix was never actually live.** It was merged to `main` but the account was over its usage cap from 2026-08-27 through 2026-08-29, so `apify push` never ran — every cloud run still executed the OLD build (`0.0.1`, datacenter proxy, no Camoufox path). Re-pushed as build `0.0.2` now that the cap is lifted ($80 ceiling, $41.37 used). - **A real bug the 08-27 fix didn't anticipate.** Once actually run in the cloud, `ncnbKI3rjyndmAav5` showed curl-cffi rotate through 4 browser profiles: 3 hit Cloudflare's managed challenge (`cf-mitigated: challenge` / "Just a moment"), but the 4th (`safari180`) got a bare `200` that triggers neither the challenge nor geo-splash marker — confirmed by local reproduction (RESIDENTIAL, US-pinned) that Quora served an un-hydrated React/Relay shell (`