Multi-Search Engine Scraper

a powerful and versatile search engine scraper that allows you to extract search results from multiple search engines simultaneously. Whether you're conducting market research, gathering data for analysis, or building a search aggregation service, this actor provides a robust solution for your web scraping needs.

🌟 Features

• Multi-Engine Support: Scrape results from the popular search engines including Google, Bing, Yahoo, DuckDuckGo, and more being added
• Customizable Scraping: Control the number of pages to scrape and request timeout
• Proxy Support: Optional proxy configuration for enhanced privacy and IP rotation
• Simple Integration: Easy-to-use interface with clear input/output schemas
• Reliable Performance: Built with stability and reliability in mind

📥 Input Schema

The scraper accepts the following parameters:

Parameter	Type	Description	Default
search_query	string	The search query to look for	Required
search_engines	string	Select one or more search engines to use	"google"
max_pages	integer	How many pages to crawl from search results	1
timeout	integer	Request timeout in seconds	10
proxy	string	Optional proxy URL to use for requests	None

Supported Search Engines

• Google
• Bing
• Yahoo
• AOL
• DuckDuckGo
• StartPage
• Dogpile
• Ask

Example Input

{
    "search_query": "learn python",
    "search_engines": ["google","bing"],
    "max_pages": 2,
    "timeout": 15,
    "proxy": "http://proxy.example.com:8080"
}

📤 Output Schema

The scraper returns results in the following format:

Field	Type	Description
host	string	The host domain of the result
link	string	The full URL of the result
title	string	The title of the webpage
text	string	The snippet/description of the result
engine	string	The search engine that provided this result
page	integer	The page number where this result was found

Example Output

{
    "results": [
        {
            "host": "python.org",
            "link": "https://www.python.org/about/gettingstarted/",
            "title": "Python For Beginners",
            "text": "Python is a programming language that lets you work quickly and integrate systems more effectively...",
            "engine": "google",
            "page": 1
        },
        {
            "host": "w3schools.com",
            "link": "https://www.w3schools.com/python/",
            "title": "Python Tutorial - W3Schools",
            "text": "Learn Python programming with our comprehensive tutorial...",
            "engine": "bing",
            "page": 2
        }
    ]
}