# 🔎 Yahoo Scraper (`scrapemesh/yahoo-scraper`) Actor

Scrape Yahoo Search results in bulk — titles, snippets, links, favicons, sub-links and a clean Markdown excerpt for every page. Smart proxy auto-escalation (direct → datacenter → residential) keeps you unblocked.

- **URL**: https://apify.com/scrapemesh/yahoo-scraper.md
- **Developed by:** [ScrapeMesh](https://apify.com/scrapemesh) (community)
- **Categories:** Automation, Lead generation, Social media
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.99 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## 🔎 Yahoo Search Scraper

Scrape **Yahoo Search** results at scale — titles, URLs, snippets, favicons, in-article sub-links, and a clean Markdown excerpt for every result. Bulk queries, time-window filtering and smart proxy auto-escalation (direct → datacenter → residential) keep your runs fast and unblocked.

---

### ⭐ Why Choose Us?

* **Bulk-first** — paste dozens of queries (or full Yahoo URLs) and walk every result page until the cap.
* **Smart proxy ladder** — starts direct, only escalates if Yahoo blocks. You don't pay for residential traffic you didn't need.
* **Rich back-fill** — when Yahoo's snippet is thin, the actor visits the result page and harvests in-article sub-links + a Markdown summary.
* **Live results** — rows stream to the dataset as they're scraped, so a mid-run interruption never loses your data.
* **Production-grade error handling** — 3-tier proxy retries, graceful PPE limit handling, exponential cool-downs.

---

### 🔑 Key Features

* 🌐 Bulk queries — plain keywords **or** full Yahoo SERP URLs, mixed freely.
* 📅 Time-window filter — Anytime / Past day / Past week / Past month.
* 🛡️ Auto-escalating proxy: **direct → Apify Datacenter → Apify Residential (3 retries)**, then sticky.
* 🧩 Optional second-pass back-fill of sub-links + Markdown excerpts.
* 📋 Per-section dataset views: **Overview**, **Snippet**, **Sub-links**.
* 🔄 Custom proxy URLs supported — they go first, then the smart ladder.

---

### 🧾 Input

```json
{
  "queries": [
    "java developer",
    "https://search.yahoo.com/search?p=python+jobs"
  ],
  "maxItems": 10,
  "timePeriod": "Anytime",
  "backfillEmptyResults": true,
  "backfillConcurrency": 8,
  "backfillMaxLinks": 10,
  "proxyConfiguration": { "useApifyProxy": false }
}
````

| Field | Type | Description |
|--|--|--|
| `queries` | `string[]` | One or more search terms **or** Yahoo SERP URLs. |
| `maxItems` | `integer` | Hard cap on unique results per query (1–500). |
| `timePeriod` | `string` | `Anytime` / `Past day` / `Past week` / `Past month`. |
| `backfillEmptyResults` | `boolean` | Visit each result page to harvest sub-links + Markdown excerpt. |
| `backfillConcurrency` | `integer` | Parallelism for back-fill (1–32). |
| `backfillMaxLinks` | `integer` | Max in-article sub-links per result page (1–50). |
| `proxyConfiguration` | `object` | Apify proxy config. Defaults to **direct** (no proxy). |

***

### 📤 Output

Each row matches the per-section views in the dataset.

```json
{
  "query": "java developer",
  "title": "How to become a Java Developer? - GeeksforGeeks",
  "url": "https://www.geeksforgeeks.org/gfg-academy/how-to-become-a-java-developer/",
  "description": "A Java developer is a software engineer who builds...",
  "text": "  * Core Java\n\nCore Fundamentals: Learn concepts and practice DSA...\n",
  "logo_url": "https://s.yimg.com/pv/.../32x32_7eae5aac8b7f7402.png",
  "links": [
    "https://www.geeksforgeeks.org/java/java",
    "https://www.geeksforgeeks.org/advance-java/spring"
  ],
  "domain": "www.geeksforgeeks.org"
}
```

| Field | Description |
|--|--|
| `query` | The query (or URL) the row was scraped under. |
| `title` | The result's headline. |
| `url` | The clean target URL (Yahoo's tracker is stripped). |
| `description` | Yahoo's SERP snippet, rendered as Markdown. |
| `text` | Markdown excerpt — either Yahoo's list block or, after back-fill, an in-article summary. |
| `logo_url` | The result's favicon. |
| `links` | Up to N harvested in-article sub-links (after back-fill). |
| `domain` | The host portion of `url`. |

***

### 🚀 How to Use (Apify Console)

1. Open **[Apify Console → Actors](https://console.apify.com/actors)**.
2. Find this actor and open it.
3. Paste your queries (one per line) into **🌐 Search Queries / URLs**.
4. Pick a **🎁 Maximum results** cap and a **📅 Time window**.
5. (Optional) Leave proxy on **direct** — the actor will auto-escalate only when needed.
6. Click **Start**.
7. Watch live logs — rows appear in the **Output** tab as they're scraped.
8. Export results as JSON / CSV / XLSX.

***

### 🤖 Use via API

```bash
curl -X POST "https://api.apify.com/v2/acts/<ACTOR_ID>/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "queries": ["java developer"],
       "maxItems": 10,
       "timePeriod": "Anytime"
     }'
```

***

### 💼 Best Use Cases

- SEO & SERP monitoring on Yahoo.
- Competitive intelligence — track who appears for a query over time.
- Lead generation — feed result URLs into your own enrichment pipeline.
- Content discovery — harvest in-article sub-links for further crawling.

***

### 💳 Pricing

This actor uses Apify's **Pay-per-event** model. The primary event is **`result-item`** — one charge per result row pushed to the dataset. You pay only for the rows you actually receive; back-fill, retries and failed attempts are **not** billed.

You also pay the underlying Apify platform usage (compute units, proxy traffic when used). Direct (no-proxy) requests cost no proxy traffic at all — which is why the actor stays on **direct** until Yahoo forces it to escalate.

***

### ❓ Frequently Asked Questions

**Does it work when Yahoo blocks me?**
Yes. The default no-proxy run is the fastest, but the moment Yahoo returns a block (HTTP 429/503 or a captcha page), the actor auto-escalates to the Apify Datacenter pool, then to Residential with up to 3 retries. Once a tier works, it's locked in for the rest of the run.

**Can I bring my own proxies?**
Yes — paste them into the proxy field's **Custom proxy URLs**. Your URLs are tried first (3 retries), then the datacenter → residential fallback ladder kicks in.

**Does it follow pagination?**
Yes. Yahoo returns ~7 results per page; the actor walks pages until your `maxItems` cap is hit or 3 consecutive pages return nothing.

**What about non-Latin queries?**
Yahoo handles UTF-8 queries natively — paste them as-is.

**Why is my back-filled `text` empty for some rows?**
Some sites block all bots (or render with JS only). In that case the actor falls back to a minimal block built from Yahoo's own title + description so the field is never blank.

***

### 📨 Support & Feedback

- Issues / feature requests → please open a thread on the actor's detail page.
- Custom solutions → **dev.scraperengine@gmail.com**.

# Actor input Schema

## `queries` (type: `array`):

Paste one or more **search queries** (e.g. `java developer`) **OR** ready-made **Yahoo Search URLs** (e.g. `https://search.yahoo.com/search?p=java+developer`). You can mix both freely — the actor auto-detects each entry. One entry per line.

## `maxItems` (type: `integer`):

Hard cap on the number of unique Yahoo result rows pushed for each query. Yahoo paginates ~7 results per page, so e.g. `maxItems: 30` walks ~5 pages.

## `timePeriod` (type: `string`):

Restrict results to a specific freshness window. `Anytime` keeps Yahoo's default ranking.

## `proxyConfiguration` (type: `object`):

Smart auto-escalation is built in. The actor starts **direct (no proxy)**. If Yahoo blocks, it switches to **Apify Datacenter**, then to **Apify Residential** with up to 3 retries — and **sticks with the working tier** for the rest of the run. If you provide **custom proxy URLs** here, those are tried first (3 retries) before the datacenter/residential fallback ladder.

## Actor input object example

```json
{
  "queries": [
    "java developer"
  ],
  "maxItems": 10,
  "timePeriod": "Anytime",
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "java developer"
    ],
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("scrapemesh/yahoo-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": ["java developer"],
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("scrapemesh/yahoo-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "java developer"
  ],
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call scrapemesh/yahoo-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=scrapemesh/yahoo-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "🔎 Yahoo Scraper",
        "description": "Scrape Yahoo Search results in bulk — titles, snippets, links, favicons, sub-links and a clean Markdown excerpt for every page. Smart proxy auto-escalation (direct → datacenter → residential) keeps you unblocked.",
        "version": "0.1",
        "x-build-id": "yd9n7oNcaC7Kbf1e8"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/scrapemesh~yahoo-scraper/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-scrapemesh-yahoo-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/scrapemesh~yahoo-scraper/runs": {
            "post": {
                "operationId": "runs-sync-scrapemesh-yahoo-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/scrapemesh~yahoo-scraper/run-sync": {
            "post": {
                "operationId": "run-sync-scrapemesh-yahoo-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "required": [
                    "queries"
                ],
                "properties": {
                    "queries": {
                        "title": "🌐 Search Queries / URLs (Bulk supported)",
                        "type": "array",
                        "description": "Paste one or more **search queries** (e.g. `java developer`) **OR** ready-made **Yahoo Search URLs** (e.g. `https://search.yahoo.com/search?p=java+developer`). You can mix both freely — the actor auto-detects each entry. One entry per line.",
                        "items": {
                            "type": "string"
                        }
                    },
                    "maxItems": {
                        "title": "🎁 Maximum results (per query)",
                        "minimum": 1,
                        "maximum": 50000,
                        "type": "integer",
                        "description": "Hard cap on the number of unique Yahoo result rows pushed for each query. Yahoo paginates ~7 results per page, so e.g. `maxItems: 30` walks ~5 pages.",
                        "default": 10
                    },
                    "timePeriod": {
                        "title": "📅 Time window",
                        "enum": [
                            "Anytime",
                            "Past day",
                            "Past week",
                            "Past month"
                        ],
                        "type": "string",
                        "description": "Restrict results to a specific freshness window. `Anytime` keeps Yahoo's default ranking.",
                        "default": "Anytime"
                    },
                    "proxyConfiguration": {
                        "title": "🛡️ Proxy configuration",
                        "type": "object",
                        "description": "Smart auto-escalation is built in. The actor starts **direct (no proxy)**. If Yahoo blocks, it switches to **Apify Datacenter**, then to **Apify Residential** with up to 3 retries — and **sticks with the working tier** for the rest of the run. If you provide **custom proxy URLs** here, those are tried first (3 retries) before the datacenter/residential fallback ladder.",
                        "default": {
                            "useApifyProxy": false
                        }
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```