Yandex Search Scraper

Yandex web results as a spreadsheet. Each row carries its rank counted across pages, the title, the destination address, the breadcrumb Yandex shows under it, the snippet, and the sitelinks nested beneath it — plus the count of organic and paid rows on the page it came from, and Yandex's own phrasing of how many results the query has.

And one flag that is the reason to take this from a dataset rather than from a screenshot.

Yandex does not label its adverts. A paid placement is an ordinary result row with the same markup and no class of its own, sitting among the organic ones. The same query answered from a US route put five of them above the first organic result. Every row here carries is_advert, and every row also carries organic_count beside count, so the split never has to be inferred.

Accepted input

queries is required and takes one search term per line. A term is never split on a comma, so a phrase containing one stays a single search. Each term is walked separately and every row names the term it answered, so a run over twenty terms splits back apart cleanly.

regionId is optional and takes Yandex's own region number — 213 is Moscow, 84 is the USA. It changes which results come back and how many adverts sit above them. It does not translate them; it is a market selector, not a language one.

startPage defaults to 0 and accepts 0 to 24. It is where each query's walk begins, not the page it fetches — the run continues forward from there. Page 25 serves a shell rather than results, which is why the ceiling is what it is.

maxItems defaults to 100 and caps each query separately, so twenty terms with a cap of 100 returns up to 2,000 rows rather than 100 shared between them. 0 means no limit, which here means 250 organic results plus whatever adverts sit among them.

{
  "queries": ["stripe payments", "payment gateway"],
  "regionId": 213,
  "startPage": 0,
  "maxItems": 100
}

Response fields

{
  "query": "stripe payments",
  "region_id": 213,
  "page": 0,
  "page_exists": true,
  "count": 14,
  "organic_count": 10,
  "advert_count": 4,
  "results_found_text": "stripe payments — Яндекс: нашлось 28 тыс. результатов",
  "position": 5,
  "is_advert": false,
  "title": "Stripe | Financial Infrastructure to Grow Your Revenue",
  "url": "https://stripe.com/",
  "displayed_path": "stripe.com › payments",
  "snippet": "Millions of companies use Stripe to accept payments online…",
  "sitelinks": [{ "title": "Pricing", "url": "https://stripe.com/pricing" }]
}

Fields absent from a result are returned as null rather than omitted, so every row has the same shape.

is_advert is the column to read first. On an advert url is null, because Yandex does not state a paid result's destination — its link goes through a click counter instead.

url on an organic result is the destination Yandex names, not the counter redirect the link on the page points at. That means a row is usable as a rank-tracking record without following anything.

position is the rank across pages with adverts included, so page 1's first row is position 11. That is the position as the page renders it, which is what a rank-tracking caller means by rank.

results_found_text is Yandex's own approximate phrase, published verbatim and unparsed. It moves: three fetches of one query said 26, 28 and 29 thousand. A parsed integer would look authoritative and be none of those things.

page_exists false means the query has fewer pages than the run asked for. It is a real answer rather than a failure — Yandex serves a shell past the end of a result set — and the walk stops there.

Behaviour on partial results

One fetch is a sample of a page, not the page. Fetched twice from one pinned route, the first page shared 7, 9 and 9 of its ten destinations across three pairs, and never came back in the same order. So this Actor does not promise a stable ordered set and you should not build one on it: treat a run as a sample of the results, and take a repeated run as a second sample rather than a correction of the first.

Paging is real and clears that noise easily — page 0 against page 1 shared 0, 1 and 1 of 10 over the same pairs — which is why a walk is worth making at all.

A destination already collected in the same walk is skipped rather than saved twice, and the run log says how many it skipped. Repeats across pages are rare here, so that is a safety net rather than the main event.

A query that answers with nothing is skipped with a line in the log and the rest of the list still runs.

Deeper pages take more work to obtain and are the one place this source gives up. Measured per attempt over two runs of ten per page, the first page was served twenty times out of twenty, while pages 1, 3, 10 and 24 all landed between 30% and 55%. Read that as one step rather than a curve — ten samples is nowhere near enough to rank the deep pages against each other, and the two runs disagreed by 3 of 10 on page 1 alone. A deep-page run is slower and more likely to end early than a first-page run.

Frequently asked questions

Why does the same query return different results each time? Because Yandex resamples its own first page. Two fetches from one pinned route shared 7 to 9 of ten destinations and never repeated their order. This is a property of the search engine rather than of the Actor, and pretending otherwise — by promising a stable ranked set — would be the wrong thing to do with it. Take a run as a sample. If you need a rank you can defend, take several runs and use the median position for a destination.

How do I tell an advert from an organic result? is_advert. There is nothing else to go on: Yandex renders both in the same markup with the same classes. The discriminator used is that an organic result names its own destination and an advert does not, sending the click through a counter instead — which is also why url is null on an advert.

Why does position start at 11 on the second page? Because it is the rank as rendered, across pages and adverts included. Page 0 holds positions 1 to about 14 depending on how many adverts sat in it; page 1 continues from there. A per-page position would make two rows from different pages look equally ranked.

Can I get more than 250 results for one query? No. Yandex serves 25 pages of 10 organic results and page 25 is a shell. Narrowing the query reaches different results; going deeper does not exist.

Does regionId translate the results? No. It changes which results Yandex serves and how many adverts sit above them — a market selector. The results come back in whatever language the pages are written in.

Is a Yandex account or API key required? No. No account, session cookie or key of any kind is supplied to the Actor or needed by it.

Related Actors

Yandex Maps Scraper is the other half of this site and a completely different dataset: organisations with ratings, phone numbers, websites, opening hours and coordinates, searched by area rather than by keyword. Unlike the web results, the map surface is stable — one page fetched twice returned all 25 rows in the same order.

Yandex Maps Place Scraper reads one organisation in full, and Yandex Maps Reviews Scraper exports its reviews.