Brave Discussions Scraper

Scrapes discussion threads and answer excerpts that Brave Search displays in its Discussions panel, with support for inline Reddit and forum Q&A results.

Input

Provide a query of up to 300 characters. maxItems defaults to 5 (range 1–100), maxPages to 1 (range 1–10), and maxRequestRetries to 0 (range 0–5). Only ordinary server errors may use explicitly requested retries. navigationTimeoutSecs defaults to 60 seconds, and requestTimeoutSecs bounds each page's full processing at 90 seconds. Country defaults to us, language to en, and the browser runs headlessly. The checked-in input searches react vs vue, directly, for at most five records on one page.

The runtime uses bundled stock Firefox 1.61.1 with TLS verification enabled, the matching official Apify Firefox image, and exact lockfile dependencies installed with npm ci. It does not change browser identity or weaken the browser sandbox. A requested but unavailable proxy fails instead of silently changing route. Pagination follows only an observed next link retaining the requested origin, query, locale, and page.

Output

Each record includes position and page, thread title and canonical URL, source and favicon, community and thread ID when available, question or answer excerpt, published date, comment and upvote counts, replies, top answer data for inline Q&A, discussion type, forum flag, localized search context, provenance, and scrape metadata. The example uses content from the September 22 browser inspection with the publication date representation corrected; it is not a fresh Actor dataset. Comments are not labeled as answers, and answer votes remain scoped to individual answers. Top-level vote fields preserve upvotes shown in the discussion panel metadata, without claiming they belong to an individual answer.

publishedAtRaw retains the displayed date label. A complete calendar date becomes publishedAt: "YYYY-MM-DD" with publishedAtPrecision: "day", without inventing midnight or a timezone. An ISO timestamp is retained only if the source supplies a timezone, with publishedAtPrecision: "timestamp" and its original offset and fractional-second precision. Partial, relative, invalid, ambiguous, or timezone-less time labels do not produce a normalized publication value. scrapedAt is the actual retrieval timestamp.

Local validation

npm test
npm run lint
npm run validate:dataset
npm run audit:dataset

The extractor accepts both the current #discussions panel and the older web-result inline Q&A layout. It removes duplicate canonical thread URLs and stops when maxItems is reached.

Terminal access restrictions

HTTP 401/403/407/429, explicit CAPTCHA, and access-denial pages stop later actor-initiated requests. They are never treated as temporary retries, and no alternate source, session, or proxy is selected. Existing admitted requests observe the shared stop state before continuing or accepting results; ordinary transport and server failures remain separately bounded. Dataset dedupe and accepted-row counts advance only after a successful write, so an unsuccessful write can be retried without silently dropping its records.

Candidate verification status

The October 2 direct run MhNa2JaAd4mVTO77B on build 2.1.2 succeeded with four unique Reddit rows from five extracted records. Independent review found that its date-only labels had been emitted as midnight-UTC timestamps, so that build is not promotion-ready. The precision correction and stricter audit are tested offline; a fresh successful cloud run and independent audit of the corrected build remain required. Original run evidence has not been rewritten. Unknown or unrecognized empty results fail closed.