Search & Read - Web Search Results with Page Content

Search the web for a batch of queries, pinned to a country and language, and read the top result pages as clean text and markdown, in one run. One dataset record per query: its results, the content behind each result, and an explicit reason wherever there is no content.

• Country and language pinned per query. Google through Apify's Google SERP proxy with the country Google domain, gl, hl and a proxy in that country, so results do not change language between runs.
• Pages read in the same run. No child Actor per query or per page. A URL returned by several queries is fetched once; every query keeps its own rank and reference to it.
• Explicit statuses. Search failures and page failures are separate. A page that could not be read keeps its search title and snippet with contentSource: snippet_only and null content, never an invented or partial claim.
• Bounded. Limits on results, pages, bytes, time and concurrency. At the time limit, finished results are returned and unfinished steps say deadline_exceeded.
• Browser when the page needs it. renderMode: auto reads over HTTP and switches to Chromium only for JavaScript shells, very thin pages and bot challenges; browser renders every page. Each record says which mode produced it.
• Honest coverage. Every query record carries search.coverage: how many organic results there were, whether the engine showed local (map) listings that are not extracted, and whether the engine ran the query as sent.

Built for lead research, market and competitor monitoring and RAG pipelines, where a list of search results is only useful once you have the text behind it. The page reader inside it is LeadProof's Web Page Reader; the search is our own, not a third-party Actor.

Try it

The prefilled input searches two Kenyan company names, pinned to Kenya and English, and reads up to four pages per query. One run, two queries, and the pages come back in the same dataset: a query record per query with its results, each result carrying the page record and the reason if the page could not be read.

Input

{
  "queries": [
    {"id": "eon", "query": "\"Eon Energy\" Kenya"},
    {"id": "kavirondo", "query": "\"Kavirondo Chemist\" Homa Bay Kenya"},
    {"id": "uganda-row", "query": "hardware store Kampala", "country": "UG"}
  ],
  "country": "KE",
  "language": "en",
  "maxResultsPerQuery": 4,
  "maxPagesPerQuery": 4,
  "contentScope": "page"
}

Field	Default	Notes
mode	search	search needs queries; urls reads urls directly and never searches.
queries		Strings or {id, query, country, language}, up to 200. Missing ids become q1, q2... Identical queries keep their own ids; the search runs once and is shared (search.sharedWithQueryId).
urls		URL mode: strings or {id, url}, up to 500 public http(s) URLs.
country, language		Required in search mode, at the top level or per row. ISO code or English country name; language tag such as en, sw, en-KE. A row with an unknown country fails alone (invalid_locale).
maxResultsPerQuery	5	1-20 search results kept per query.
maxPagesPerQuery	5	0-20 top results whose pages are read. Others keep search metadata with page status skipped (read_not_requested). 0 = search only.
contentScope	auto	page keeps headers and footers (contact details), main keeps the main content only.
renderMode	http	http (served HTML only), auto (browser only when the page needs it), browser (always render).
browserConcurrency	2	Pages rendered at once in browser and auto modes. Give the run 2 GB or more for browser work.
includeHtml	false	Adds size-bounded HTML to each page.
searchTimeoutSecs, pageTimeoutSecs	30	Per attempt.
searchConcurrency, pageConcurrency	3, 5	Parallel searches and page reads. The reader also limits requests per host.
maxRetries	1	Only failures marked retryable (timeouts, 429, 5xx). Never 4xx, unsupported content, login walls or bot blocks.
maxPagesTotal	200	Unique page reads per run.
maxTotalMegabytes	50	No new read starts after this much text, markdown and HTML. Reads already running finish.
maxPageKilobytes	2000	Download limit per page.
maxRunSeconds	900	Time limit; the run's own timeout is also respected (30 s kept for saving results).
saveFailedSearchPages	false	Diagnostics: keep the raw result page of a search the provider could not parse (markup_changed, unrecognized_page) in this run's key-value store.

Output

One dataset item per query (search mode) or per URL (URL mode), including failed and skipped rows. Every item follows the LeadProof contract v1: schemaVersion: "1.0", inputId, status (succeeded, partial, failed, skipped), warnings and error: null | {code, message, retryable}.

A query item (abbreviated from a real local run, 2026-09-19):

{
  "schemaVersion": "1.0", "recordType": "query", "inputId": "kavirondo", "queryId": "kavirondo",
  "query": "\"Kavirondo Chemist\" Homa Bay Kenya", "country": "KE", "language": "en",
  "provider": "google-serp-apify-proxy", "status": "partial", "warnings": ["some_pages_unreadable"], "error": null,
  "search": {"status": "succeeded", "resultCount": 4, "attempts": 1, "sharedWithQueryId": null,
             "appliedSettings": {"googleHost": "www.google.co.ke", "gl": "ke", "hl": "en", "proxyCountry": "KE"},
             "usage": {"billableRequests": 1, "estimatedProviderCostUsd": 0.0025}},
  "results": [
    {"resultId": "kavirondo#1", "contentSource": "page", "pageKey": "p786dd5aead6089e3d9d1a195",
     "searchResult": {"rank": 1, "url": "https://keonline.biz/business/121561-kavirondo-chemist-ltd", "title": "KAVIRONDO CHEMIST LTD - Contacts, Career, Services ...", "snippet": "..."},
     "page": {"schemaVersion": "1.0", "status": "succeeded", "httpStatus": 200, "title": "...", "language": "en", "text": "...", "markdown": "...", "contentHash": "...", "links": [], "metadata": {}, "structuredData": {"jsonLd": [], "jsonLdErrors": []}, "warnings": [], "error": null}},
    {"resultId": "kavirondo#2", "contentSource": "snippet_only",
     "searchResult": {"rank": 2, "url": "https://www.facebook.com/groups/.../posts/...", "title": "Staff at Kavirondo Chemist and Agrovet in Homabay ...", "snippet": "..."},
     "page": {"status": "failed", "httpStatus": 400, "text": null, "markdown": null, "error": {"code": "http_error", "message": "HTTP 400", "retryable": false}}}
  ],
  "stats": {"pagesRead": 3, "pagesFailed": 1, "pagesSkipped": 0},
  "completedAt": "2026-09-19T19:46:32.517Z"
}

• searchResult is SearchResult v1: rank is one-based within the returned results, not a global Google rank. Additive fields from the search library (urlSource, page, displayUrl) are kept.
• page is PageRecord v1 from the page reader, always with every field present. Records we create ourselves (skipped or failed reads) have null content.
• contentSource: page (readable content present), snippet_only (only the search title and snippet), none.
• Status of a query: failed when the search failed, succeeded when every requested page was read, partial when some pages could not be read or the run stopped early. An empty result list from a successful search is succeeded, not a failure.
• search.coverage and the warnings say what the search could not show:
  • local_results_excluded (and, with no organic results, no_organic_results_local_listings_not_read): the engine showed local business listings from its map product, which are not extracted. Zero organic results then means "we did not read what exists", not "the business has no web presence".
  • exact_phrase_not_found, query_changed_by_engine, query_auto_corrected: the engine ran another query (for example it dropped the quotes). The results are kept and search.coverage.queryAsSent is false. They are not evidence that the name matched; judge identity from the pages themselves.
• search.providerPages and search.providerAttempts keep the search provider's own per-page and per-request diagnostics.
• URL mode items are PageRecords with recordType: "page" and pageKey.

The run's SUMMARY record has counts, stop reason, engine statistics (searches, shared searches, page requests, shared pages, bytes, retries, limits hit), backends and charged events. Views: overview (one line per query) and pages (URL mode).

How it works and what it depends on

Part	What	Where it runs	Paid to
Web search	LeadProof's own websearch library, Google through Apify's Google SERP proxy	In this run	Covered by query-searched
Page reader	LeadProof's Web Page Reader engine, in a Node process next to the Python run	In this run	Covered by page-read / page-read-rendered
Orchestration	The search_and_read package	In this run	Covered by the same events

Nothing is charged twice: the search proxy and the page reads are the platform usage of this run, and under pay-per-event that usage is ours to cover, not an extra line on your bill.

No third-party Actor is called at runtime. Search requests and page reads are ordinary network traffic of this run. The page reader enforces the network policy (public destinations only, including redirects and browser subrequests); this Actor also rejects private, loopback, link-local, credentialed and non-HTTP(S) URLs in the input and in search results before anything is read. Robots rules for single result pages are the reader's policy; this Actor does not crawl sites.

Page text is untrusted data. It is returned as data and never interpreted as instructions.

What you pay for

Pay per event, so an empty search and an unreadable page cost nothing:

Event	When it is charged	Price
query-searched	One per query whose search completed, including one that found nothing	$0.005
page-read	One per unique page read over HTTP with readable content	$0.002
page-read-rendered	One per unique page that Chromium had to render (renderMode: browser, or auto falling back to it)	$0.006
apify-actor-start	Apify's standard start event, once per GB of the run's memory	$0.00005

What that means in practice:

• Each page is charged once, under one event. A page read over HTTP is page-read. A page the browser rendered is page-read-rendered and only that: never both events for the same page.
• You are not charged for what you did not get. A failed search, a query skipped by a limit, a page that returned 404, was blocked, held no text, or was withheld by your spending limit: free. A result that keeps only its search title and snippet (contentSource: snippet_only) is free too - it was not read, so it is not charged as a read.
• Repeats are charged once. Two rows with the identical query share one search and one charge; a URL returned by five queries is read once and charged once, and every query still gets its own record pointing at it.
• You choose whether the browser can run. renderMode: http (the default) never charges the rendered price. auto charges it only for the pages where the HTTP result was not the page; the pages it kept on HTTP stay at $0.002. browser renders everything. A rendered page costs us about four times an HTTP one, which is why it is priced apart rather than averaged into one number.
• Your spending limit is respected before the work starts. No query begins unless the remaining limit covers its search and its page reads, and a restarted run never charges the same query or page twice.
• Examples at 1 GB memory:
  • The example task "Read the pages behind a company-name search (Kenya)", a real run: 2 queries, 8 results, 7 readable pages, one result that kept its snippet because the site returned an error, 52 seconds: 2 x $0.005 + 7 x $0.002 + $0.00005 = $0.02405.
  • 10 queries, 3 readable pages each, HTTP: 10 x $0.005 + 30 x $0.002 + $0.00005 = $0.11005.
  • The same 10 queries where half the pages cannot be read: 10 x $0.005 + 15 x $0.002 = $0.08005.
  • 50 queries, search only (maxPagesPerQuery: 0): $0.25005.

Measured performance

Owner-account runs, September 2026 (run IDs in DEPLOYMENT.md):

• 3 Kenyan queries, 3 results each: 9 pages read in 31 s, $0.0104 total usage (of which Google SERP proxy $0.0075).
• 27-row company list searched through this Actor (8 runs, 37 queries including retries without the city, 79 page reads, 44 readable): $0.1106, about $0.0041 per row in http mode ($0.1310, $0.0049 per row in auto). 79% of it is the Google SERP proxy at $0.0025 per request, 13% compute, 7% key-value checkpoint writes.
• A JavaScript page returns 67 characters over HTTP and 1,433 characters in the browser; in auto the fallback happened on its own and the static page stayed on HTTP.

Limitations

• The browser renders one page at a time per slot and costs more: on a 27-row business list auto cost 18% more than http and found nothing extra, because business directories are server-rendered. Use auto for sites that are JavaScript applications.
• Local (map) listings are never extracted, only reported (local_results_excluded).
• Only public pages. No login, CAPTCHA solving or block evasion. Directory and social sites often return 403 (blocked_by_site).
• One Google results page per 10 results; very specific queries can return very few results.
• The byte limit is checked when a read starts, so the total can exceed it by the reads already running.
• Content fields are capped at 400,000 characters each and trimmed further if an item would exceed Apify's 9 MB item limit; contentHash always describes the full text before trimming.

Development

python -m unittest discover -s test (from this folder) covers batch order, locale propagation, shared URLs and searches, search versus page failures, retries, timeouts, the deadline, unsupported content, limits, resume from checkpoints, billing idempotency and budget exhaustion, input safety, contract schema compatibility, the Node bridge protocol (against a fake reader) and the web search adapter. search_and_read/contracts/ holds the JSON schemas; test/fixtures/contract/ holds contract fixtures. The engine package has no Apify dependency and can be imported by other Actors.

---

Built by LeadProof - verified local-business lead lists, built to order.