Google Search Scraper

Fetch bounded, localized Google web-search HTML and publish source-backed organic results only. People Also Ask questions, related searches, featured text, and estimated totals are retained as page evidence in the OUTPUT and OUTPUT_SUMMARY records rather than mixed into the result dataset.

Behavior

• Returns at most 100 unique organic results across at most 10 result pages.
• Preserves the original query, maxItems, localization, time-filter, nfpr, filter, timeout, retry, and proxy inputs, including the legacy time_period, time_period_min, and time_period_max aliases.
• Uses Apify's GOOGLE_SERP proxy group by default. Direct mode is useful for local QA, while standard/custom proxy settings remain accepted.
• Resolves Google's protected result redirects with bounded, redirect-only probes; destination pages are never scraped.
• Publishes normal rows atomically only after all requested pages are source-backed and valid. A challenge, consent page, oversized response, malformed source, or failed page produces no normal rows.
• Treats Google's explicit no-results page as a successful valid-empty run.
• Normal rows include source type, host, HTTP status, content type, response size, retry/proxy metadata, SHA-256 source hash, extraction method, and stable IDs. Runtime/debug fields are removed before publication.
• Optional failure rows contain exactly url, error, errorCode, and scrapedAt.

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

Input

Field	Type	Default	Notes
query	string	required	1–300 characters
maxItems	integer	10	1–100
gl	string	us	Two-letter country code
hl	string	en	Language such as en or en-us
lr	string	omitted	Result-language filter such as lang_en
timePeriod	string	none	last_hour, last_day, last_week, last_month, last_year, or custom
timePeriodMin / timePeriodMax	string	omitted	MM/DD/YYYY, required together for custom
time_period, time_period_min, time_period_max	aliases	omitted	Backward-compatible snake-case aliases
nfpr / filter	integer	omitted	Google values 0 or 1
requestTimeoutSecs	integer	45	5–120 seconds per page
maxRequestRetries	integer	1	0–2 retries after the first request
requestDelayMs	integer	0	0–5000 ms between successful pages
maxResponseBytes	integer	6000000	1024–12000000 byte response cap
includeDiagnostics	boolean	true	Emit minimal diagnostics on failure
proxySessionId	string	omitted	Optional stable proxy session
proxyConfiguration	object	Google SERP proxy	Apify, custom, or direct settings
fixtureJson / fixtureFile	string	omitted	Local QA only; fixture files must be under test/fixtures or .actor/fixtures

Example:

{
  "query": "OpenAI API documentation",
  "maxItems": 15,
  "gl": "us",
  "hl": "en",
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["GOOGLE_SERP"]
  }
}

Dataset and run metadata

Each normal row is an organic-result with recordId, position, title, url, domain, searchQuery, searchPage, localization fields, searchUrl, scrapedAt, fieldCoverage, and dataQuality. Optional source-backed fields such as displayedUrl, snippet, date, siteLinks, rating, and reviewCount are omitted when unavailable. Provenance fields identify the exact HTML source and request attempt.

OUTPUT and its identical OUTPUT_SUMMARY alias report status, counts, page-level SERP feature evidence, proxy mode, redirect-resolution count, request attempts, failures, and elapsed time. A failed run is fail-closed: diagnostics may be present, but guessed or partial normal rows are not.

Local verification

npm ci
npm run check
npm exec --yes --package=apify-cli -- apify validate-schema
npm exec --yes --package=apify-cli -- apify run --input-file qa-inputs/local-fixture.json --purge
npm run validate

The local fixture covers pagination, canonical URL deduplication, and SERP feature capture. qa-inputs/cloud-validation.json exercises bounded direct live access; Google may return a challenge, which is recorded truthfully as a diagnostic rather than bypassed.