Yandex AI Overview Scraper

Scrape AI-generated responses (Alice AI, Yandex Neuro summary, and smart answer blocks) from Yandex Search results in real-time. Extracts the full AI answer text, structured Markdown format, key takeaway bullet points, cited source URLs, follow-up suggested queries, and top organic SERP results.

---

🌟 Key Features

• ⚡ Real-time AI Overview Extraction: Captures live AI-generated summaries (Neuro / Alice AI / Smart Answers) directly from Yandex Search.
• 📝 Formatted Markdown & Key Takeaways: Returns both plain text and structured Markdown summaries with parsed bullet points (keyPoints).
• 🔗 Cited Sources & Domains: Extracts all citation links with title, clean destination URL, domain, position rank, and context snippets.
• 💡 Suggested Follow-up Questions: Gathers related queries and prompt suggestions generated for the search topic.
• 🌐 Top Organic SERP Results: Optionally extracts top organic search results (title, link, domain, snippet) for complete SERP context.
• 🎯 Dynamic Multi-Query & Batch Processing: Run single queries or batch lists of topics in a single run.
• 🌍 International & Localized Search: Supports multi-language and country region codes (e.g. en, ru, us, by, kz).
• 🛡️ Anti-Detection & Proxy Ready: Built-in stealth headers, anti-bot evasions, and native Apify residential proxy integration.

---

📋 Sample Output Record

{
  "query": "what is artificial intelligence",
  "searchUrl": "https://yandex.com/search/?text=what+is+artificial+intelligence&lr=84",
  "language": "en",
  "country": "us",
  "hasAiOverview": true,
  "aiOverview": "Artificial intelligence (AI) is the capability of computational systems to perform tasks typically associated with human intelligence, such as learning, reasoning, problem-solving, and natural language processing.",
  "aiOverviewMarkdown": "Artificial intelligence (AI) is the capability of computational systems to perform tasks typically associated with human intelligence.\n\n**Key Takeaways:**\n- Enables machine learning and neural network processing\n- Widely applied in computer vision, robotics, and natural language processing\n- Powers predictive models and autonomous decision-making systems",
  "aiBlockType": "NeuroSummary",
  "keyPoints": [
    "Enables machine learning and neural network processing",
    "Widely applied in computer vision, robotics, and natural language processing",
    "Powers predictive models and autonomous decision-making systems"
  ],
  "sources": [
    {
      "position": 1,
      "title": "Artificial intelligence - Wikipedia",
      "url": "https://en.wikipedia.org/wiki/Artificial_intelligence",
      "domain": "wikipedia.org",
      "snippet": "Artificial intelligence (AI) is the intelligence of machines or software..."
    },
    {
      "position": 2,
      "title": "What is Artificial Intelligence (AI)? | IBM",
      "url": "https://www.ibm.com/topics/artificial-intelligence",
      "domain": "ibm.com",
      "snippet": "Artificial intelligence leverages computers and machines to mimic human problem-solving..."
    }
  ],
  "suggestedQueries": [
    "how does machine learning work",
    "types of artificial intelligence",
    "applications of ai in healthcare"
  ],
  "organicResults": [
    {
      "rank": 1,
      "title": "What is Artificial Intelligence? | IBM",
      "url": "https://www.ibm.com/topics/artificial-intelligence",
      "domain": "ibm.com",
      "snippet": "Artificial intelligence leverages computers and machines to mimic the problem-solving and decision-making capabilities of the human mind."
    }
  ],
  "searchMetadata": {
    "pageTitle": "what is artificial intelligence — Yandex: found 126 thousand results",
    "totalOrganicFound": 10,
    "sourcesCount": 2,
    "hasAiSummary": true
  },
  "scrapedAt": "2026-08-20T12:30:00.000Z",
  "status": "success"
}

---

⚙️ Input Parameters

Field	Type	Default	Description
query	string	"what is artificial intelligence"	Single search query to look up on Yandex.
queries	array	[]	Optional list of search queries to scrape in batch.
language	string	"en"	Language code (e.g. 'en' for English, 'ru' for Russian).
country	string	""	Country/region code for localized results (e.g. 'us', 'ru', 'kz', 'by').
includeOrganicResults	boolean	true	Include top organic search results alongside the AI overview.
maxItems	integer	10	Maximum number of queries to process (min: 1, max: 100).
proxyConfiguration	object	{ "useApifyProxy": true }	Proxy configuration. Residential proxies are strongly recommended.

Example Input JSON

{
  "queries": [
    "what is artificial intelligence",
    "quantum computing explained"
  ],
  "language": "en",
  "country": "us",
  "includeOrganicResults": true,
  "maxItems": 10,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

---

📊 Output Fields Reference

Field	Type	Description
query	string	The submitted search query.
searchUrl	string	The target search URL requested on Yandex.
language	string	Language code used for search.
country	string	Country/region code used for search (if specified).
hasAiOverview	boolean	true if an AI Overview/Neuro response was found, false otherwise.
aiOverview	string	Full plain text of the AI-generated answer summary.
aiOverviewMarkdown	string	Formatted Markdown text including paragraphs and bullet points.
aiBlockType	string	Identified AI block selector/type name.
keyPoints	array	Key takeaways and summary bullet points.
sources	array	Array of cited source objects (position, title, url, domain, snippet).
suggestedQueries	array	Follow-up search suggestions and related questions.
organicResults	array	Top organic search result previews (rank, title, url, domain, snippet).
searchMetadata	object	SERP metadata including page title and result counts.
scrapedAt	string	ISO 8601 timestamp when scraped.
status	string	Status of the record ("success").

---

🚀 How to Run

Run on Apify Platform

1. Go to the Yandex AI Overview Scraper Actor on Apify Console.
2. Enter your search query or list of queries.
3. Configure language and proxy settings (Residential proxy recommended).
4. Click Start.
5. Export your results in JSON, CSV, Excel, or HTML format.

Run Locally

# Install dependencies
npm install

# Run the actor
npm start

# Validate the generated dataset
node validate-datasets.js

---

💡 Best Practices & Tips

• Residential Proxies: Yandex SERP actively deploys SmartCaptcha challenges on datacenter IPs. Using Apify Residential Proxies ensures highest success rates.
• Language Selection: Yandex's Neuro AI summary is most active and rich on Russian (ru) and English (en) queries.
• Batch Processing: Use the queries input array to process multiple keywords seamlessly in a single Actor run.