Google AI Scraper

This actor is designed to scrape structured answers from Google's AI-powered search results (AI Overview). Simply provide your questions, and get back comprehensive, well-organized answers with source citations, highlighted key terms, and all answer components—all in a clean, structured JSON format.

---

🚀 Key Features

• ✅ Extracts AI-generated answers with paragraphs, lists, and headings
• ✅ Retrieves source citations with URLs, titles, snippets, and source names
• ✅ Captures highlighted key terms from the AI response
• ✅ Supports multiple questions in a single run
• ✅ Full proxy support for stable and geo-aware scraping
• ✅ Handles Google's dynamic content loading seamlessly

---

👤 Who Is This Actor For?

This actor is perfect for:

• 📊 Data analysts gathering AI-curated information at scale
• 🧠 Content creators researching topics with authoritative sources
• 🔍 SEO professionals analyzing Google's AI-generated content
• 📚 Knowledge base builders aggregating information from verified sources
• 🤖 AI/ML practitioners collecting training data from Google's AI responses
• 🎓 Researchers gathering comprehensive answers with source attribution

---

❓ Why Is This Actor Important?

Google's AI Overview is a powerful feature that synthesizes information from multiple authoritative sources to answer user questions. The content is dynamically loaded and rendered, making it challenging to access programmatically. This actor:

• Waits for the AI response to be fully loaded before extraction
• Extracts clean and structured data including paragraphs, lists, headings, and sources
• Saves you hours of manual research and data collection
• Provides access to Google's AI-curated knowledge at scale

---

Tech Notes

• It is recommended to set the actor memory to at least 4GB to avoid performance issues.

• It is recommended to set the actor's proxy type to residential.

---

Input Schema

Here's an example input you can pass to the actor:

{
  "questions": [
    "how many planets are there in the solar system?",
    "what is photosynthesis?",
    "who invented the internet?"
  ],
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

Input Parameters

• questions (required): An array of questions you want answers for. Each question will be submitted to Google's AI search.
• proxyConfiguration (required): Proxy settings for the scraper. Essential for avoiding rate limits and bypassing geo-restrictions.

---

Output Schema

{
  "question": "how many planets are there in the solar system?",
  "url": "https://www.google.com/search?udm=50&aep=46&source=25q2-US-SearchSites-Site-CTA&sei=agX6aMCZHYm15NoPl-7zwAQ&q=how%20many%20planets%20are%20there%20in%20the%20solar%20system%3F",
  "sections": [
    {
      "type": "paragraph",
      "content": "There are eight planets in our solar system: Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus, and Neptune. The International Astronomical Union (IAU) officially redefined what constitutes a planet in 2006, leading to Pluto's reclassification as a \"dwarf planet\"."
    },
    {
      "type": "paragraph",
      "content": "The eight planets are categorized into two main groups based on their composition:"
    },
    {
      "type": "list",
      "listType": "unordered",
      "items": [
        "Inner, terrestrial planets: These four planets are smaller and have solid, rocky surfaces...",
        "Outer, giant planets: These four are much larger than the terrestrial planets..."
      ]
    },
    {
      "type": "heading",
      "content": "Why Pluto is no longer considered a planet"
    },
    {
      "type": "list",
      "listType": "ordered",
      "items": [
        "It must orbit the sun.",
        "It must be massive enough that its own gravity pulls it into a nearly round shape.",
        "It must have \"cleared the neighborhood\" around its orbit..."
      ]
    }
  ],
  "highlightedTerms": ["eight"],
  "sources": [
    {
      "title": "About the Planets - NASA Science",
      "snippet": "About the Planets. Our solar system has eight planets: Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus, and Neptune...",
      "sourceName": "NASA Science (.gov)",
      "url": "https://science.nasa.gov/solar-system/planets/"
    },
    {
      "title": "Why is Pluto no longer a planet? - The Library of Congress",
      "snippet": "Answer. The International Astronomical Union (IAU) downgraded the status of Pluto to that of a dwarf planet because it...",
      "sourceName": "The Library of Congress (.gov)",
      "url": "https://www.loc.gov/everyday-mysteries/astronomy/item/why-is-pluto-no-longer-a-planet/"
    }
  ],
  "fullText": "There are eight planets in our solar system: Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus, and Neptune. The International Astronomical Union (IAU) officially redefined what constitutes a planet in 2006, leading to Pluto's reclassification as a \"dwarf planet\".\n\nThe eight planets are categorized into two main groups based on their composition:\n\n..."
}

Output Fields

• question: The question that was asked
• sections: Array of answer components (paragraphs, lists, headings) in order
  • type: Component type (paragraph, list, or heading)
  • content: Text content for paragraphs and headings
  • listType: Type of list (ordered or unordered)
  • items: Array of list items
• highlightedTerms: Key terms emphasized in the AI answer
• sources: Array of source citations with complete metadata
  • title: Article or page title
  • snippet: Preview text from the source
  • sourceName: Publisher name (e.g., "NASA Science (.gov)")
  • url: Full URL to the source
• fullText: Complete answer text with all sections combined

---

💡 Use Cases

1. Research & Fact-Checking

Get AI-curated answers from authoritative sources for research projects, fact-checking, and knowledge gathering.

2. Content Creation

Discover comprehensive answers to popular questions with verified sources to inform your content creation strategy.

3. SEO & Content Strategy

Understand how Google's AI answers questions to optimize your content for AI-powered search results and featured snippets.

4. Competitive Intelligence

Track how Google's AI presents information about your industry, competitors, or products from various sources.

5. Training Data Collection

Collect high-quality Q&A pairs with source attribution and highlighted key terms for training language models or building knowledge bases.

6. Academic Research

Gather structured information with proper source citations for academic papers, literature reviews, or educational content.

---

📊 Example Questions

Try these question types:

• Factual: "How many planets are there in the solar system?"
• Scientific: "What is photosynthesis?"
• Historical: "Who invented the internet?"
• Explanatory: "Why is the sky blue?"
• Biographical: "Who was Albert Einstein?"

---

👀 p.s.

Got feedback or need an extension?

Lexis Solutions is a certified Apify Partner. We can help you with custom solutions or data extraction projects.

Contact us over Email or LinkedIn

Support Our Work 💝

If you're happy with our work and scrapers, you're welcome to leave us a company review here and leave a review for the scrapers you're subscribed to. It will take you less than a minute but it will mean a lot to us!

Image Credit: https://www.google.com