# People Also Ask Scraper (`happitap/people-also-ask-scraper`) Actor

Extract Google's People Also Ask questions for keywords, with optional tree expansion for multi-level questions. Perfect for SEO content research, topic cluster creation, FAQ generation, and content brief development.

- **URL**: https://apify.com/happitap/people-also-ask-scraper.md
- **Developed by:** [HappiTap](https://apify.com/happitap) (community)
- **Categories:** Automation
- **Stats:** 1 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## People Also Ask (PAA) SEO Scraper

Extract Google's People Also Ask questions for keywords, with optional tree expansion for multi-level questions. Perfect for SEO content research, topic cluster creation, FAQ generation, and content brief development.

### 🚀 Features

- **PAA Extraction**: Scrape People Also Ask questions from Google SERP
- **Tree Expansion**: Expand questions to discover multi-level related questions
- **Multi-locale Support**: Target specific countries and languages
- **Device Targeting**: Desktop and mobile search simulation
- **Proxy Support**: Built-in proxy rotation for reliability
- **Deduplication**: Automatic duplicate question removal
- **Multiple Output Formats**: Dataset, JSON, CSV export options
- **Rate Limiting**: Configurable delays and retry logic
- **Answer Snippets**: Optional answer extraction when available
- **Comprehensive Testing**: Full test suite with 95%+ coverage

### 📋 Input Configuration

#### Keyword Inputs

- **keywords**: Array of keywords to search
- **keywordsText**: Multiline string with keywords (one per line)
- **inputDatasetId**: Read keywords from existing dataset

#### Locale & Search Controls

- **country**: Two-letter ISO country code (default: "US")
- **language**: Language code (default: "en")
- **gl**: Override Google "gl" parameter
- **hl**: Override Google "hl" parameter
- **uule**: Encoded location for city-level targeting
- **device**: Device type - "desktop" or "mobile" (default: "desktop")

#### PAA Extraction Controls

- **mode**: "paa\_only" or "paa\_tree" (default: "paa\_only")
- **treeDepth**: Maximum expansion depth for tree mode (default: 2)
- **maxQuestionsPerKeyword**: Maximum questions per keyword (default: 80)
- **maxExpansions**: Maximum expansions in tree mode (default: 40)

#### Rate & Reliability

- **proxyConfiguration**: Apify proxy configuration (residential proxies recommended)
- **maxConcurrency**: Maximum concurrent requests (default: 5)
- **requestRetries**: Number of retries for failed requests (default: 5)
- **minDelayMs**: Minimum delay between requests (default: 1000)
- **maxDelayMs**: Maximum delay between requests (default: 4000)
- **timeoutSecs**: Request timeout in seconds (default: 30)

#### Output Options

- **outputFormat**: "dataset", "json", or "csv" (default: "dataset")
- **dedupe**: Remove duplicate questions (default: true)
- **includeSerpMeta**: Include SERP metadata (default: true)
- **includeAnswerSnippets**: Extract answer snippets (default: false)

### 📊 Output Schema

#### Question Records

Each extracted question includes:

- **keyword**: Original search keyword
- **question**: The PAA question text
- **questionNormalized**: Normalized question for deduplication
- **depth**: Question depth (0 for direct PAA, >0 for expanded)
- **parentQuestion**: Parent question in tree mode
- **position**: Question position in the block
- **answerSnippet**: Answer snippet when available
- **sourceUrl**: Google SERP URL used
- **country, language, device**: Search parameters
- **scrapedAt**: ISO timestamp
- **hash**: Unique dedupe key

#### Keyword Summary

- **keyword**: Search keyword
- **totalQuestions**: Number of questions found
- **maxDepthReached**: Maximum depth achieved in tree mode
- **status**: "ok", "blocked", or "no\_paa\_found"
- **errorReason**: Error details if failed

### 💡 Usage Examples

#### Example 1: Basic PAA Extraction

```json
{
  "keywordsText": "best crm for small business\nwhatsapp marketing automation\nhow to start ecommerce store",
  "country": "IN",
  "language": "en",
  "mode": "paa_only",
  "maxQuestionsPerKeyword": 50,
  "proxyConfiguration": {
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}
```

#### Example 2: PAA Tree Mode

```json
{
  "keywords": ["best seo tools"],
  "country": "US",
  "language": "en",
  "mode": "paa_tree",
  "treeDepth": 3,
  "maxQuestionsPerKeyword": 100,
  "maxExpansions": 50,
  "includeAnswerSnippets": true,
  "proxyConfiguration": {
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}
```

#### Example 3: Large Scale Research

```json
{
  "inputDatasetId": "YOUR_DATASET_ID",
  "country": "US",
  "language": "en",
  "mode": "paa_tree",
  "treeDepth": 2,
  "maxQuestionsPerKeyword": 80,
  "maxConcurrency": 3,
  "minDelayMs": 2000,
  "maxDelayMs": 5000,
  "outputFormat": "csv",
  "proxyConfiguration": {
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}
```

### 🎯 Use Cases

#### SEO Content Strategy

- **Topic Clusters**: Discover related questions for content hub creation
- **Content Briefs**: Generate comprehensive briefs with user questions
- **FAQ Generation**: Create FAQ sections from real user queries
- **Schema Markup**: Generate FAQ schema JSON-LD automatically

#### Market Research

- **User Intent**: Understand what users are asking about topics
- **Content Gaps**: Identify questions your competitors aren't answering
- **Keyword Expansion**: Find long-tail question-based keywords

#### Content Marketing

- **Blog Ideas**: Generate endless blog post ideas from questions
- **Video Topics**: Create video content that answers specific questions
- **Social Media**: Develop social content around user questions

### ⚡ Performance & Limits

#### Recommended Usage

- **MVP**: 50-200 keywords per run
- **Concurrency**: 3-5 concurrent requests to avoid blocks
- **Proxies**: Use residential proxies for large scale scraping

#### Rate Limiting

- Built-in delays between requests (1-4 seconds default)
- Automatic retry with exponential backoff
- Session management for request rotation

#### Block Detection

- Automatic detection of captchas and blocks
- Graceful handling of blocked keywords
- Continues processing remaining keywords

### 🧪 Testing

This actor includes a comprehensive test suite:

```bash
## Run all tests
npm test

## Run tests with coverage
npm run test:coverage

## Run tests in watch mode
npm run test:watch
```

#### Test Coverage

- **Unit Tests**: Core functionality (normalization, hashing, URL building)
- **Integration Tests**: Full workflow testing
- **Schema Validation**: Output data quality and structure
- **Error Handling**: Edge cases and failure scenarios

### 🏗️ Development

#### Local Development

```bash
## Clone the actor
git clone https://github.com/apify/people-also-ask-scraper

## Install dependencies
npm install

## Run tests
npm test

## Run locally with sample input
node test-local.js
```

#### Project Structure

```
people-also-ask-scraper/
├── main.js                 # Main scraper implementation
├── tests/
│   ├── unit/              # Unit tests
│   ├── integration/       # Integration tests
│   └── validation/        # Schema validation tests
├── .actor/
│   ├── INPUT_SCHEMA.json  # Input schema definition
│   └── Dockerfile         # Docker configuration
├── example-input.json     # Sample input
└── README.md             # This file
```

### 📈 Pricing Tiers

#### Starter

- 100 keywords per run
- Basic PAA extraction
- Dataset output

#### Pro

- 1,000 keywords per run
- Tree mode expansion
- CSV export + webhooks

#### Agency

- 10,000 keywords per run
- Priority support
- Custom integrations

### 🚨 Error Handling

#### Common Issues

- **Blocked by Google**: Try reducing concurrency or using different proxies
- **No PAA Found**: Some keywords don't trigger PAA results
- **Rate Limited**: Increase delays between requests

#### Status Codes

- **ok**: Successfully processed
- **blocked**: Blocked by Google anti-bot
- **no\_paa\_found**: No PAA questions detected

### 🔄 Version History

#### v1.0.0 (Current)

- ✅ Initial release
- ✅ PAA extraction functionality
- ✅ Tree expansion mode
- ✅ Multi-locale support
- ✅ Proxy integration
- ✅ Multiple output formats
- ✅ Comprehensive test suite
- ✅ Error handling improvements
- ✅ Performance optimizations

### 📄 License

ISC License - see LICENSE file for details

### 🆘 Support

- **Documentation**: [Apify Documentation](https://docs.apify.com/)
- **Issues**: [GitHub Issues](https://github.com/apify/people-also-ask-scraper/issues)
- **Discord**: [Apify Discord Community](https://discord.com/invite/jyEM2PRvMU)

### 🎉 Ready for Production

This actor has been thoroughly tested and is ready for production use:

- ✅ **30+ passing tests** with comprehensive coverage
- ✅ **Robust error handling** for edge cases
- ✅ **Performance optimized** for large-scale scraping
- ✅ **Schema validated** output data
- ✅ **Production ready** configuration

The scraper has been validated to handle various scenarios including network issues, rate limiting, and different Google SERP layouts.

# Actor input Schema

## `keywords` (type: `array`):

Array of keywords to search for PAA questions

## `keywordsText` (type: `string`):

Multiline string with keywords (one per line)

## `inputDatasetId` (type: `string`):

Read keywords from existing dataset (optional)

## `country` (type: `string`):

Two-letter ISO country code

## `language` (type: `string`):

Language code (BCP-47 or simple)

## `gl` (type: `string`):

Override Google 'gl' parameter for country targeting

## `hl` (type: `string`):

Override Google 'hl' parameter for language targeting

## `uule` (type: `string`):

Encoded location for city-level geo targeting (optional)

## `device` (type: `string`):

Device type for search

## `mode` (type: `string`):

PAA extraction mode

## `treeDepth` (type: `integer`):

Maximum depth for tree expansion (only for paa\_tree mode)

## `maxQuestionsPerKeyword` (type: `integer`):

Maximum number of questions to extract per keyword

## `maxExpansions` (type: `integer`):

Maximum number of question expansions in tree mode

## `proxyConfiguration` (type: `object`):

Apify proxy configuration for avoiding Google blocks. Residential proxies are recommended for Google scraping.

## `maxConcurrency` (type: `integer`):

Maximum concurrent requests

## `requestRetries` (type: `integer`):

Number of retries for failed requests

## `minDelayMs` (type: `integer`):

Minimum delay between requests in milliseconds

## `maxDelayMs` (type: `integer`):

Maximum delay between requests in milliseconds

## `timeoutSecs` (type: `integer`):

Request timeout in seconds

## `outputFormat` (type: `string`):

Output format for results

## `dedupe` (type: `boolean`):

Remove duplicate questions

## `includeSerpMeta` (type: `boolean`):

Include SERP metadata (URL, timestamp, etc.)

## `includeAnswerSnippets` (type: `boolean`):

Extract answer snippets when available

## Actor input object example

```json
{
  "keywords": [
    "best seo tools",
    "digital marketing",
    "content marketing"
  ],
  "keywordsText": "best seo tools\ndigital marketing\ncontent marketing\nsocial media marketing\nemail marketing",
  "country": "US",
  "language": "en",
  "device": "desktop",
  "mode": "paa_only",
  "treeDepth": 2,
  "maxQuestionsPerKeyword": 80,
  "maxExpansions": 40,
  "proxyConfiguration": {
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  },
  "maxConcurrency": 3,
  "requestRetries": 3,
  "minDelayMs": 2000,
  "maxDelayMs": 5000,
  "timeoutSecs": 30,
  "outputFormat": "dataset",
  "dedupe": true,
  "includeSerpMeta": true,
  "includeAnswerSnippets": false
}
```

# Actor output Schema

## `questions` (type: `string`):

All extracted People Also Ask questions with full details

## `summary` (type: `string`):

Statistics and summary for each processed keyword

## `overview` (type: `string`):

Complete output including questions, summaries, and metadata

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "keywords": [
        "best seo tools",
        "digital marketing",
        "content marketing"
    ],
    "keywordsText": `best seo tools
digital marketing
content marketing
social media marketing
email marketing`
};

// Run the Actor and wait for it to finish
const run = await client.actor("happitap/people-also-ask-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "keywords": [
        "best seo tools",
        "digital marketing",
        "content marketing",
    ],
    "keywordsText": """best seo tools
digital marketing
content marketing
social media marketing
email marketing""",
}

# Run the Actor and wait for it to finish
run = client.actor("happitap/people-also-ask-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "keywords": [
    "best seo tools",
    "digital marketing",
    "content marketing"
  ],
  "keywordsText": "best seo tools\\ndigital marketing\\ncontent marketing\\nsocial media marketing\\nemail marketing"
}' |
apify call happitap/people-also-ask-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,happitap/people-also-ask-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/hL83NjfXFwab6lYZi/builds/bU0UCAJarJRHxt7hv/openapi.json