# Dogpile Search Scraper - Metasearch SERP API (`jaybird/dogpile-scraper`) Actor Scrapes organic search results from Dogpile.com metasearch engine. Returns structured data including title, URL, description, and position for each result. - **URL**: https://apify.com/jaybird/dogpile-scraper.md - **Developed by:** [Jaybird Technologies](https://apify.com/jaybird) (community) - **Categories:** AI, Lead generation, SEO tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $1.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Dogpile Search Scraper — Metasearch SERP API ### What does Dogpile Search Scraper do? **Dogpile Search Scraper** extracts search results from [Dogpile.com](https://www.dogpile.com), a **metasearch engine that aggregates results from Google, Bing, and other search engines** into a single results page. Use it as a Dogpile SERP API to get structured data from organic results, paid ads, and related searches — all from one scrape. Unlike single-engine scrapers, Dogpile gives you a **cross-engine view of search rankings** in a single request. ### Why scrape Dogpile? Why use a metasearch SERP API? Dogpile combines results from multiple search engines, providing broader visibility than any single engine alone: - **Cross-engine SEO monitoring** — See how content ranks across Google and Bing simultaneously without running separate scrapers - **Competitive analysis** — Discover competitors visible across multiple engines that you might miss on Google alone - **Ad intelligence** — Monitor paid search ads served through Google's syndication network on metasearch - **Market research** — Find content that ranks well across engines, indicating strong topical authority - **Related keyword discovery** — Extract related search suggestions for content strategy and keyword expansion - **SERP comparison** — Study differences between metasearch aggregated results and single-engine SERPs Built on the Apify platform, you get API access, scheduling, webhooks, and integrations with tools like Zapier, Make, Google Sheets, and Slack. ### What data can you extract from Dogpile? #### Organic results | Field | Type | Description | |-------|------|-------------| | searchQuery | object | Query term, page number, domain, engine | | position | number | Ranking position (1-based, continuous across pages) | | title | string | Result title/heading | | url | string | Destination URL | | displayedUrl | string | Breadcrumb-style URL shown in results | | description | string | Result snippet/description | | siteName | string | Website name (e.g., "YouTube", "CNN") | | type | string | Always `"organic"` | #### Paid results (ads) | Field | Type | Description | |-------|------|-------------| | searchQuery | object | Query term, page number, domain, engine | | position | number | Ad position (numbered separately from organic) | | title | string | Ad headline | | url | string | Actual advertiser destination URL (resolved from click tracker) | | displayedUrl | string | Display URL shown in the ad | | description | string | Ad copy/description text | | type | string | Always `"paid"` | #### Related searches | Field | Type | Description | |-------|------|-------------| | searchQuery | object | The original query that surfaced this suggestion | | title | string | Suggested related search term | | url | string | Dogpile search URL for this term | | type | string | Always `"relatedSearch"` | ### How to scrape Dogpile search results 1. Create a free Apify account 2. Open the Dogpile Search Scraper Actor 3. Enter your search queries in the **Search Queries** field (one per line) 4. Set **Max pages per query** (default: 3, each page has ~8-10 results) 5. Click **Start** and wait for the run to finish 6. Download your results in JSON, CSV, Excel, or other formats You can also run the scraper via API, schedule recurring runs, or integrate with your existing data pipeline. ### How much will scraping Dogpile cost? Each run scrapes approximately 8-10 organic results per page plus any paid ads and related searches present. A typical configuration of 10 queries across 3 pages (30 total page loads) completes in under 2 minutes. The scraper uses Apify Proxy (included) for reliable results. ### Input See the Input tab for full configuration options: - **Search Queries** (required) — List of search terms to scrape - **Max pages per query** — Result pages per query (default: 3, max: 25). ~8-10 organic results per page - **Max organic results per query** — Cap on organic results (0 = unlimited). Paid results and related searches are always returned - **Save HTML** — Save raw HTML to Key-Value Store for debugging/auditing ### Output You can download the dataset in various formats: JSON, CSV, Excel, XML, or HTML. The Output tab provides separate views for **Organic Results**, **Paid Results (Ads)**, **Related Searches**, and **All Results** combined. Example organic result (JSON): ```json { "searchQuery": { "term": "web scraping tools", "page": 1, "domain": "dogpile.com", "engine": "Dogpile (metasearch)" }, "position": 1, "title": "16 Best Web Scraping Tools In 2026 (Pros, Cons, Pricing)", "url": "https://www.scraperapi.com/web-scraping/tools/", "displayedUrl": "https://scraperapi.com > web-scraping > tools", "description": "Discover the top 16 web scraping tools for 2026. Compare features, pricing, and pros/cons to find the perfect tool for your needs.", "siteName": "scraperapi.com", "type": "organic" } ```` Example paid result (JSON): ```json { "searchQuery": { "term": "best credit cards", "page": 1, "domain": "dogpile.com", "engine": "Dogpile (metasearch)" }, "position": 1, "title": "Best Credit Cards of 2026 - Compare Top Offers", "url": "https://www.nerdwallet.com/credit-cards/best", "displayedUrl": "https://www.nerdwallet.com/credit-cards/best", "description": "Expert reviews of all the top-rated credit cards on the market.", "type": "paid" } ``` ### Tips and best practices - Start with 1-2 pages per query to estimate costs before scaling up - Each page returns ~8-10 organic results, so 3 pages yields ~25-30 results per query - Paid ads and related searches are bonus data — they don't count toward `maxResultsPerQuery` - Enable **Save HTML** when debugging to inspect the raw source - Use the separate dataset views (Organic, Paid, Related) for cleaner exports - Schedule recurring runs to monitor ranking changes over time ### Integrations Connect Dogpile Search Scraper with your tools: - **Google Sheets** — Auto-export results to a spreadsheet - **Zapier / Make** — Trigger workflows when new results appear - **Webhooks** — Get notified when a run completes - **API** — Run programmatically via REST API or Node.js/Python SDK ### FAQ and support #### What makes Dogpile different from Google Search? Dogpile is a metasearch engine — it queries Google, Bing, and other engines, then aggregates and deduplicates their results. You get cross-engine visibility in a single API call. #### How many results does Dogpile return per page? Typically 8-10 organic results per page, plus 2-4 paid ads and 5-10 related search suggestions on the first page. #### Can I scrape multiple queries at once? Yes. Add multiple queries to the input and they'll all be processed in a single run with automatic pagination. #### Do I get both organic AND paid results? Yes. The scraper captures organic results, paid ads (Google syndicated ads), and related search suggestions. Each type is clearly labeled with a `type` field. #### Are the ad URLs the real destination? Yes. The scraper resolves Google's click-tracking redirects to extract the actual advertiser landing page URL. #### I'm getting empty results This is rare since Dogpile has minimal anti-bot protection when accessed through Apify Proxy. If it happens, try reducing concurrency or check the Issues tab. #### Is it legal to scrape Dogpile? This Actor only extracts publicly available search results that anyone can access through a browser. It does not extract private user data. We believe our Actors, when used for ethical purposes, are safe. However, you should be aware that results could contain personal data. Personal data is protected by the GDPR in the European Union and by other regulations. You should not scrape personal data unless you have a legitimate reason to do so. If you're unsure whether your reason is legitimate, consult your lawyers. *** For issues and feature requests, use the Issues tab. For programmatic access, see the API tab. # Actor input Schema ## `queries` (type: `array`): List of search queries to scrape. Each query will be searched on Dogpile and results extracted across the configured number of pages. ## `maxPagesPerQuery` (type: `integer`): Maximum number of search result pages to scrape per query. Each page returns approximately 8-10 organic results. ## `maxResultsPerQuery` (type: `integer`): Maximum number of organic results to return per query. Set to 0 for unlimited (bounded by max pages). Paid results and related searches are always returned in full. ## `saveHtml` (type: `boolean`): Save the raw HTML of each scraped page to the Key-Value Store for debugging or auditing purposes. ## Actor input object example ```json { "queries": [ "web scraping tools", "best project management software" ], "maxPagesPerQuery": 3, "maxResultsPerQuery": 0, "saveHtml": false } ``` # Actor output Schema ## `allResults` (type: `string`): No description ## `organicResults` (type: `string`): No description ## `paidResults` (type: `string`): No description ## `relatedSearches` (type: `string`): No description ## `savedHtml` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "queries": [ "web scraping tools", "best project management software" ], "maxPagesPerQuery": 3, "maxResultsPerQuery": 0 }; // Run the Actor and wait for it to finish const run = await client.actor("jaybird/dogpile-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "queries": [ "web scraping tools", "best project management software", ], "maxPagesPerQuery": 3, "maxResultsPerQuery": 0, } # Run the Actor and wait for it to finish run = client.actor("jaybird/dogpile-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "queries": [ "web scraping tools", "best project management software" ], "maxPagesPerQuery": 3, "maxResultsPerQuery": 0 }' | apify call jaybird/dogpile-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=jaybird/dogpile-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Dogpile Search Scraper - Metasearch SERP API", "description": "Scrapes organic search results from Dogpile.com metasearch engine. Returns structured data including title, URL, description, and position for each result.", "version": "1.0", "x-build-id": "N8UpXsGacepYSMsL7" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/jaybird~dogpile-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-jaybird-dogpile-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/jaybird~dogpile-scraper/runs": { "post": { "operationId": "runs-sync-jaybird-dogpile-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/jaybird~dogpile-scraper/run-sync": { "post": { "operationId": "run-sync-jaybird-dogpile-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "queries" ], "properties": { "queries": { "title": "Search Queries", "type": "array", "description": "List of search queries to scrape. Each query will be searched on Dogpile and results extracted across the configured number of pages.", "items": { "type": "string" }, "default": [ "web scraping tools" ] }, "maxPagesPerQuery": { "title": "Max pages per query", "minimum": 1, "maximum": 25, "type": "integer", "description": "Maximum number of search result pages to scrape per query. Each page returns approximately 8-10 organic results.", "default": 1 }, "maxResultsPerQuery": { "title": "Max organic results per query", "minimum": 0, "type": "integer", "description": "Maximum number of organic results to return per query. Set to 0 for unlimited (bounded by max pages). Paid results and related searches are always returned in full.", "default": 0 }, "saveHtml": { "title": "Save HTML to Key-Value Store", "type": "boolean", "description": "Save the raw HTML of each scraped page to the Key-Value Store for debugging or auditing purposes.", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```