# Startpage Search Extractor (`kawsar/startpage-search-extractor`) Actor

Startpage search scraper that extracts organic search results for any keyword list, with rank position, title, URL, domain, and description, so SEO teams can run keyword research and track competitor rankings without personalized results skewing the data.

- **URL**: https://apify.com/kawsar/startpage-search-extractor.md
- **Developed by:** [Kawsar](https://apify.com/kawsar) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.99 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Startpage Search Extractor: Scrape Startpage Search Results for Any Keyword

Startpage Search Extractor is a Startpage search scraper that turns a list of keywords into structured SERP data. For every organic search result it saves the rank position, page title, destination URL, domain, and snippet text, then pages through Startpage until it has collected as many results as you asked for. Because Startpage strips out personalization and search history, the rankings you pull are close to a neutral baseline, which makes this a practical way to do SEO keyword research and competitor tracking without your own browsing habits coloring the data.

Give it one keyword or two hundred. The actor handles pagination, retries, proxy rotation, and language or region settings on its own.

### Use cases

- **SEO keyword research**: collect the full first three pages for a keyword set and see which domains own the topic before you commit to writing anything
- **Rank tracking**: run the same keyword list on a schedule and watch your pages move up or down over weeks
- **Competitor monitoring**: filter to the past week and catch competitor articles the day they start ranking
- **Link prospecting**: pull hundreds of ranking URLs for a niche, then sort by domain to find outreach targets
- **Content gap analysis**: compare the URLs ranking for your terms against your own sitemap to find topics you never covered
- **Academic and market research**: gather search results in a specific language and country without a personalized feed skewing what comes back

### Input

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `keywords` | array | required | Search terms, one per line. Startpage operators such as `site:`, `intitle:`, quoted phrases, and `-exclusions` all work. |
| `maxItems` | integer | 10 | Total results to collect across all keywords. This is the only limit you need to set, and pagination follows from it. |
| `timeoutSecs` | integer | 600 | Time budget for the run. Anything collected before the cutoff is still saved. |
| `requestTimeoutSecs` | integer | 45 | How long to wait for a results page to render before retrying. |
| `language` | string | `english` | Language Startpage uses to read the query and return results. |
| `region` | string | `all` | Country to localize results to, such as `en_US`, `de_DE`, or `en_GB`. |
| `timeRange` | string | `any` | Limit results to the past day, week, month, or year. |
| `safeSearch` | string | `medium` | Adult content filter: off, moderate, or strict. |
| `includeAds` | boolean | `false` | Save sponsored placements too, flagged with `isSponsored`. |
| `delayBetweenRequestsSecs` | integer | 1 | Pause between page requests. Raise it if a run starts getting challenged, lower it to finish faster. |
| `proxyConfiguration` | object | Residential | Proxy type and location for requests. Startpage challenges datacenter IPs hard, so Residential is preselected and is what you want. |

#### Example input

```json
{
    "keywords": [
        "best web scraping tools",
        "site:apify.com web scraper"
    ],
    "maxItems": 10,
    "language": "english",
    "region": "en_US",
    "timeRange": "month",
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": ["RESIDENTIAL"]
    }
}
```

### What data does this actor extract?

Results land in a dataset, one row per search result. Here is a real row:

```json
{
    "keyword": "best web scraping tools",
    "rankPosition": 3,
    "resultTitle": "The 10 Best Web Scraping Tools in 2026",
    "url": "https://blog.apify.com/best-web-scraping-tools/",
    "displayUrl": "blog.apify.com/best-web-scraping-tools",
    "domain": "blog.apify.com",
    "site": "Apify",
    "resultDescription": "A breakdown of the most widely used scraping tools, what each one costs, and which sites they handle well.",
    "isSponsored": false,
    "pageNumber": 1,
    "searchLanguage": "english",
    "searchRegion": "en_US",
    "scrapedAt": "2026-09-17T09:14:02.481293+00:00",
    "error": null
}
```

| Field | Type | Description |
|-------|------|-------------|
| `keyword` | string | The search term that produced this result. |
| `rankPosition` | integer | Position within the keyword's result list, starting at 1. |
| `resultTitle` | string | Headline text shown on the results page. |
| `url` | string | Destination URL, with Startpage redirect wrappers already unwrapped. |
| `displayUrl` | string | Shortened URL text printed under the headline. |
| `domain` | string | Hostname of the destination URL, without `www.`. |
| `site` | string | Readable site name derived from the domain, for example `Goodreads`. |
| `resultDescription` | string | Snippet text beneath the headline. |
| `isSponsored` | boolean | True for paid placements, false for organic results. |
| `pageNumber` | integer | Which results page the row came from. |
| `searchLanguage` | string | Language setting used for the search. |
| `searchRegion` | string | Region setting used, or `all` for worldwide. |
| `scrapedAt` | string | UTC timestamp in ISO 8601 format. |
| `error` | string | Why a keyword failed, if it did. Null on successful rows. |

Export any of it as CSV, Excel, JSON, XML, or HTML, push it to Google Sheets, or read it straight from the Apify API.

### How it works

1. Reads your keyword list and drops duplicates.
2. If the `SCRAPE_DO_TOKEN` environment variable is set, every results page is fetched through the remote renderer (no local browser). Live tests filled most of a 100-item run this way; blocked pages are retried, then skipped so later pages can still count.
3. If that token is not set, launches Playwright, clears Startpage's verification check, and pages with Startpage's pagination buttons.
4. Parses each organic result into title, URL, domain, and snippet.
5. Saves each row to the dataset. If a keyword fails outright, it saves a row with the reason instead of ending the run.

### FAQ

**How many search results can I get per run?**
Up to 1000 rows per run, spread across as many keywords as you like. `maxItems` is the only limit to set, and the actor pages through Startpage until it gets there.

**Why did one keyword return fewer results than `maxItems`?**
Startpage stops serving new results for a single query somewhere around 50 to 60 organic hits, usually after five or six pages. Past that it repeats what you already have. If you set `maxItems` to 100 against one keyword you will get roughly 56, and the log will say the keyword is exhausted. To reach higher totals, spread `maxItems` across several keywords rather than pushing one term deeper.

**Do I need a proxy?**
If `SCRAPE_DO_TOKEN` is set, Apify proxy is optional (remote fetch brings its own residential pool). Without that token, yes — use Residential.

**What happens if Startpage refuses to serve a page?**
With remote fetch, that page is retried, then skipped so later pages can still fill `maxItems`. Without it, page 1 may restart the browser once; a later-page suspension keeps collected rows and moves on.

**What decides how long a run takes?**
With remote fetch, mostly page count (~10s per page in live tests, plus retries). Without it, mostly Startpage's verification check and pagination blocks.

**How much memory should I give this actor?**
With remote fetch, 2 GB is usually enough because no local browser runs. Without it, leave the default 8 GB so verification hashing has CPU.

**Why not plain HTTP requests?**
Startpage answers plain HTTP with a proof-of-work or CAPTCHA page. The remote renderer or Playwright is what clears that and returns real results.

**Can I use search operators?**
Yes. `site:`, `intitle:`, `inurl:`, quoted phrases, and `-word` exclusions are passed through to Startpage untouched, so `site:competitor.com pricing` works exactly as it would in the search box.

**Are the results personalized to my location?**
No, and that is the point. Startpage does not profile the caller, so what you get back is the unpersonalized result set. Set `region` if you specifically want results as they appear in one country.

**Why would a keyword return fewer results than I asked for?**
Either Startpage ran out of results for that query, or the run hit `maxItems` or `timeoutSecs` first. The log says which one it was.

### Integrations

Connect Startpage Search Extractor with other apps and services using [Apify integrations](https://apify.com/integrations). You can integrate with Make, Zapier, Slack, Airbyte, GitHub, Google Sheets, Google Drive, and many more. You can also use [webhooks](https://docs.apify.com/integrations/webhooks) to trigger actions whenever results are available.

Point it at your keyword list and you have a Startpage search scraper producing clean, exportable SERP data on whatever schedule you set.

# Actor input Schema

## `keywords` (type: `array`):

Search terms to run on Startpage, one per line. Every Startpage search operator works here, so you can use site:apify.com, "exact phrase", intitle:, or -excluded to narrow the results.

## `maxItems` (type: `integer`):

Total number of results to collect in one run, counted across all keywords. The actor pages through Startpage until it reaches this number. Note that Startpage returns roughly 50 to 60 organic results for a single keyword before it runs out, so use several keywords when you want more than that.

## `timeoutSecs` (type: `integer`):

Overall time budget for the run. Whatever has been collected when the budget runs out is still saved to the dataset.

## `requestTimeoutSecs` (type: `integer`):

How long to wait for a Startpage results page to render before giving up on it and retrying.

## `language` (type: `string`):

Language Startpage uses to interpret the query and return results.

## `region` (type: `string`):

Country the results should be localized to. Pick a region when you want to see the rankings a searcher in that market would get rather than worldwide results.

## `timeRange` (type: `string`):

Restrict results to pages Startpage has seen recently. Useful for news monitoring and for spotting freshly published competitor content.

## `safeSearch` (type: `string`):

Adult content filter strength applied by Startpage.

## `includeAds` (type: `boolean`):

Save paid placements alongside organic results and mark them with isSponsored. Leave off for clean organic ranking data.

## `delayBetweenRequestsSecs` (type: `integer`):

Pause added between page requests. A short wait keeps the request pattern human and cuts down on CAPTCHA pages. Raise it if a run starts getting challenged, lower it to finish faster.

## `proxyConfiguration` (type: `object`):

Select proxies to use for requests. Startpage challenges datacenter IP ranges aggressively, so Residential is preselected and is what you want for reliable results. Datacenter is cheaper but is usually served a CAPTCHA instead of search results.

## Actor input object example

```json
{
  "keywords": [
    "best web scraping tools"
  ],
  "maxItems": 10,
  "timeoutSecs": 600,
  "requestTimeoutSecs": 45,
  "language": "english",
  "region": "all",
  "timeRange": "any",
  "safeSearch": "medium",
  "includeAds": false,
  "delayBetweenRequestsSecs": 1,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# Actor output Schema

## `searchResults` (type: `string`):

Organic Startpage search results collected for your keywords, with rank position, title, URL, domain, and description for each result.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "keywords": [
        "best web scraping tools",
        "site:apify.com web scraper"
    ],
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "RESIDENTIAL"
        ]
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("kawsar/startpage-search-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "keywords": [
        "best web scraping tools",
        "site:apify.com web scraper",
    ],
    "proxyConfiguration": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["RESIDENTIAL"],
    },
}

# Run the Actor and wait for it to finish
run = client.actor("kawsar/startpage-search-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "keywords": [
    "best web scraping tools",
    "site:apify.com web scraper"
  ],
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}' |
apify call kawsar/startpage-search-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,kawsar/startpage-search-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/c30CNhhW6NlERaZ9O/builds/iWWztzpKeh3qjBeqb/openapi.json