# Stack Exchange Questions and Answers Scraper (`automation-lab/stackexchange-questions-answers-export`) Actor

Search Stack Exchange sites by query, tags, author, score, date, or question URL and export normalized questions and answers with bodies, acceptance, authors, dates, and links.

- **URL**: https://apify.com/automation-lab/stackexchange-questions-answers-export.md
- **Developed by:** [Automation Lab](https://apify.com/automation-lab) (community)
- **Categories:** Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.96 / 1,000 item extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Stack Exchange Questions and Answers Scraper

Search selected Stack Exchange sites and export normalized questions and answers with bodies, tags, scores, authors, acceptance status, dates, and source links.

This Actor is designed for recurring developer research datasets, Q\&A trend analysis, knowledge-base preparation, and source-backed topic monitoring.

It uses the official public Stack Exchange API and does not require a Stack Exchange login.

### What does Stack Exchange Questions and Answers Scraper do?

The Actor turns Stack Exchange search results into integration-ready dataset rows.

You can:

- search one or several Stack Exchange network sites;
- combine a text query with tags, author, score, and date filters;
- fetch known Stack Overflow or Stack Exchange question URLs;
- include complete answer bodies and acceptance state;
- preserve both source HTML and readable plain text;
- schedule the same input to build a repeatable research dataset;
- export results as JSON, JSONL, CSV, XML, Excel, or RSS through Apify.

Question and answer rows use one stable schema, with `recordType` identifying the entity.

### Who is this Stack Exchange questions and answers export for?

**Developer-relations teams** can identify recurring questions around an API, SDK, or framework.

**Product researchers** can compare highly voted questions, accepted answers, and topic activity.

**Data teams** can create normalized Q\&A datasets without maintaining API pagination and backoff logic.

**Technical writers** can find source-linked questions that reveal documentation gaps.

**AI and search teams** can collect public text with IDs and canonical provenance for evaluation or retrieval workflows.

### Why use this Actor?

The output keeps questions and answers separate while retaining their relationship.

Every answer includes its parent question ID and question title.

Every row includes a canonical source link.

All search and URL inputs use the same filters.

The Actor honors Stack Exchange API backoff instructions and applies bounded retries to temporary failures.

It stops at your requested question limit and does not use browser or proxy traffic.

### Supported Stack Exchange sites

Use the site's API identifier in `sites`.

Common values include:

- `stackoverflow`
- `math`
- `serverfault`
- `superuser`
- `askubuntu`
- `salesforce`
- `worldbuilding`
- `softwareengineering`

For sites hosted at `<name>.stackexchange.com`, the identifier is usually `<name>`.

Stack Overflow, Server Fault, Super User, and Ask Ubuntu use their familiar identifiers shown above.

### What data can be exported?

| Field | Meaning |
| --- | --- |
| `recordType` | `question` or `answer` |
| `site` | Stack Exchange API site identifier |
| `questionId` | Source question ID |
| `answerId` | Answer ID, or null for questions |
| `parentQuestionId` | Parent question ID on answer rows |
| `title` | Question title |
| `bodyHtml` | Original API body HTML |
| `bodyText` | Readable plain-text body |
| `tags` | Parent question tags |
| `score` | Question or answer score |
| `viewCount` | Question views |
| `answerCount` | Number of question answers |
| `isAnswered` | Stack Exchange answered state |
| `isAccepted` | Accepted state for an answer |
| `acceptedAnswerId` | Accepted answer ID when available |
| `authorId` | Public numeric author ID |
| `authorName` | Public display name |
| `authorUrl` | Public profile URL |
| `authorReputation` | Reputation reported by the API |
| `createdAt` | ISO 8601 creation time |
| `lastActivityAt` | ISO 8601 activity time |
| `lastEditedAt` | ISO 8601 edit time, when present |
| `sourceUrl` | Canonical question or answer URL |
| `query` | Query that produced the row |
| `fetchedAt` | Retrieval timestamp |

Fields unavailable from the source are returned as null rather than invented.

### How to run the Actor

1. Open the Actor in Apify Console.
2. Add one or more Stack Exchange site identifiers.
3. Enter a query, tags, an author ID, or known question URLs.
4. Choose whether to include answers.
5. Set the maximum number of questions.
6. Click **Start**.
7. Open the Dataset tab to preview or download the rows.

Start with 5 questions while refining filters, then increase the limit for production runs.

### Input parameters

| Input | Type | Default | Description |
| --- | --- | --- | --- |
| `sites` | string\[] | `['stackoverflow']` | Up to 10 sites searched in order |
| `query` | string | — | Words in question title or body |
| `tags` | string\[] | — | Up to 5 required tags |
| `authorId` | integer | — | Question author user ID |
| `minScore` | integer | — | Minimum question score |
| `dateFrom` | date | — | Earliest creation date |
| `dateTo` | date | — | Latest creation date |
| `questionUrls` | string\[] | — | Known public question URLs |
| `includeAnswers` | boolean | `true` | Add answer rows |
| `maxQuestions` | integer | `25` | Question limit, 1–500 |
| `sort` | string | `relevance` | Relevance, activity, creation, or votes |
| `order` | string | `desc` | Descending or ascending |
| `maxRequestRetries` | integer | `2` | Temporary request retries, 0–5 |

Provide at least one of `query`, `tags`, `authorId`, or `questionUrls`.

Search filters are combined with AND.

All supplied tags must be present.

Answer rows do not count against `maxQuestions`.

### Example input: API topic research

```json
{
  "sites": ["stackoverflow"],
  "query": "Stack Exchange API",
  "includeAnswers": true,
  "maxQuestions": 5,
  "sort": "relevance"
}
```

This input exports up to five matching questions plus their available answers.

### Example input: tag and score monitoring

```json
{
  "sites": ["stackoverflow"],
  "query": "fetch",
  "tags": ["javascript"],
  "minScore": 5,
  "includeAnswers": true,
  "maxQuestions": 10,
  "sort": "votes",
  "order": "desc"
}
```

Save this input as an Apify Task and schedule it to create periodic snapshots.

### Example output

```json
{
  "recordType": "answer",
  "site": "stackoverflow",
  "questionId": 12345678,
  "answerId": 12345679,
  "parentQuestionId": 12345678,
  "title": "How can I retry a fetch request?",
  "bodyHtml": "<p>Use a bounded retry loop...</p>",
  "bodyText": "Use a bounded retry loop...",
  "tags": ["javascript", "fetch"],
  "score": 12,
  "viewCount": null,
  "answerCount": null,
  "isAnswered": null,
  "isAccepted": true,
  "acceptedAnswerId": 12345679,
  "authorId": 246810,
  "authorName": "Example Author",
  "authorUrl": "https://stackoverflow.com/users/246810/example-author",
  "authorReputation": 4200,
  "createdAt": "2025-01-15T12:00:00.000Z",
  "lastActivityAt": "2025-01-16T09:30:00.000Z",
  "lastEditedAt": null,
  "sourceUrl": "https://stackoverflow.com/questions/12345678/example#12345679",
  "query": "fetch",
  "fetchedAt": "2025-01-17T10:00:00.000Z"
}
```

The example uses anonymized values. Actual output contains current public source data.

### How much does it cost to export Stack Exchange questions and answers?

The Actor uses pay-per-event pricing.

A one-time **Start** event is charged once per run.

An **Item extracted** event is charged for each useful question or answer row written to the dataset.

Failed, rejected, duplicate, and empty records are not charged as items.

The current BRONZE price is **$0.001 per run start** plus **$0.0016 per item**. Higher and lower account tiers are shown in Console before a run.

For planning, total Actor charge is:

`$0.001 + (question rows + answer rows) × $0.0016` at BRONZE.

Example BRONZE Actor charges are approximately $0.009 for 5 rows, $0.041 for 25 rows, and $0.161 for 100 rows. Platform infrastructure usage, if applicable to your account, is shown separately by Apify.

Because answers are separate useful rows, a five-question run can contain more than five billable items.

Use `includeAnswers: false` when your workflow needs only question metadata and bodies.

### Using explicit question URLs

Known question URLs bypass search discovery.

Supported examples include:

- `https://stackoverflow.com/questions/QUESTION_ID/...`
- `https://math.stackexchange.com/questions/QUESTION_ID/...`
- `https://serverfault.com/questions/QUESTION_ID/...`
- `https://superuser.com/questions/QUESTION_ID/...`

The Actor derives the API site and question ID from each URL.

Query, tag, author, score, and date filters still apply to fetched questions.

Invalid or unsupported hosts fail closed with a clear input error.

### Recurring monitoring workflow

Create an Apify Task with a stable query and filters.

Schedule it daily or weekly.

Export each run's dataset to your warehouse or webhook destination.

Use `questionId`, `answerId`, `lastActivityAt`, and `lastEditedAt` as comparison keys.

The Actor returns snapshots; it does not maintain history or send alerts by itself.

Apify schedules, webhooks, and integrations provide the automation layer.

### API usage with cURL

Replace `APIFY_TOKEN` with your token in your environment.

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~stackexchange-questions-answers-export/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"sites":["stackoverflow"],"query":"Stack Exchange API","maxQuestions":5,"includeAnswers":true}'
```

To wait for results synchronously, use the run-sync-get-dataset-items endpoint.

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~stackexchange-questions-answers-export/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"sites":["math"],"tags":["calculus"],"maxQuestions":5}'
```

### JavaScript API example

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/stackexchange-questions-answers-export').call({
  sites: ['stackoverflow'],
  query: 'fetch',
  tags: ['javascript'],
  minScore: 5,
  includeAnswers: true,
  maxQuestions: 10,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

### Python API example

```python
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/stackexchange-questions-answers-export').call(run_input={
    'sites': ['stackoverflow'],
    'query': 'Stack Exchange API',
    'includeAnswers': True,
    'maxQuestions': 5,
})
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)
```

### Connect through MCP

Add the Actor to Claude Code:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/stackexchange-questions-answers-export"
```

#### Claude Desktop setup

Add this server object to Claude Desktop's MCP configuration:

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/stackexchange-questions-answers-export"
    }
  }
}
```

#### Cursor setup

Open Cursor settings, choose **Tools & MCP**, add a remote MCP server, and use the same Apify MCP URL shown above.

#### VS Code setup

Open VS Code's MCP configuration, add an HTTP server named `apify`, and use the same Actor-specific URL. Confirm that MCP support is enabled in your VS Code installation.

Example prompts:

- “Search Stack Overflow for recent questions about the Stack Exchange API and include answers.”
- “Export high-scoring JavaScript fetch questions and accepted answers.”
- “Collect five recent calculus questions from Math Stack Exchange.”

### Integrations

Send completed datasets to Google Sheets for editorial research.

Load JSONL into a warehouse for longitudinal analysis.

Trigger a webhook when a scheduled run completes.

Use Zapier or Make to route high-scoring questions to a review queue.

Use `sourceUrl` to preserve human-verifiable provenance in downstream systems.

### Reliability and retries

The Actor uses the official Stack Exchange API rather than page scraping.

It validates response shapes before producing data.

Temporary network and API failures are retried with bounded exponential delay.

When the API asks clients to back off, the Actor waits for the requested interval.

Deterministic input errors are not retried.

A run fails after retry exhaustion rather than returning misleading partial success.

### Limits and source behavior

Anonymous Stack Exchange API requests have a shared quota reported by the source.

Large or frequent jobs may encounter throttling.

The Actor supports up to 500 questions per run.

The API may omit deleted users, deleted posts, or fields unavailable to anonymous clients.

Search relevance and answered state are defined by Stack Exchange.

Result counts can change as posts are edited, scored, answered, or removed.

Answers are requested in batches of up to 100 question IDs.

No login-only, private, or deleted content is retrieved.

### Responsible use and legality

Stack Exchange content is publicly accessible but remains subject to source terms and content licensing.

Review the applicable Stack Exchange Terms of Service and content attribution requirements before republishing data.

Keep `sourceUrl` and author attribution when your use requires them.

Do not use public profile information for harassment, spam, or sensitive profiling.

Comply with privacy, database, copyright, and AI-training rules that apply to your jurisdiction and use case.

You are responsible for your inputs and downstream use.

### Troubleshooting

**The run says I must provide a search input.**

Add at least one of `query`, `tags`, `authorId`, or `questionUrls`.

**A site identifier fails.**

Use the Stack Exchange API site parameter, such as `stackoverflow` or `math`, not a full URL.

**I received fewer questions than requested.**

The source may have exhausted matching results, or filters may be narrow. Check tags, dates, author ID, and score.

**There are more rows than `maxQuestions`.**

The limit counts questions. Each included answer is saved as a separate row.

**A temporary API error exhausted retries.**

Run again later, reduce frequency, or split a large multi-site job into scheduled Tasks.

**An answer has no author ID.**

The source can omit owner information for removed accounts. Nullable fields preserve that state.

### FAQ

#### Does this scrape every Stack Exchange site?

It searches the site identifiers you provide. It does not crawl the entire network automatically.

#### Can I fetch one known question?

Yes. Put its public question URL in `questionUrls` and set `maxQuestions` to at least 1.

#### Are answer bodies included?

Yes when `includeAnswers` is true. Both HTML and plain text are returned.

#### Can I filter answers by score?

`minScore` filters questions. Answers are exported with their scores but are not independently filtered.

#### Does the Actor deduplicate historical runs?

No. Each run is an independent snapshot. Deduplicate downstream by site plus question or answer ID.

#### Does it need a proxy or Stack Exchange account?

No. It calls the public API directly.

#### Can I use the output for RAG?

The normalized text and canonical links are suitable building blocks, but you must handle licensing, attribution, chunking, and retrieval quality for your application.

### Related Automation Lab Actors

For scholarly literature rather than developer Q\&A, use [ArXiv Papers Scraper](https://apify.com/automation-lab/arxiv-paper-search-export).

For biomedical literature search, use [PubMed Article Search Scraper](https://apify.com/automation-lab/pubmed-article-search-scraper).

For extracting supplied article pages across domains, use [Multi-Site Article Content Extractor](https://apify.com/automation-lab/multi-site-article-content-extractor).

These Actors solve adjacent research workflows; they are not Stack Exchange data sources.

# Actor input Schema

## `sites` (type: `array`):

API site identifiers to search, such as stackoverflow, math, serverfault, or superuser.

## `query` (type: `string`):

Words to match in question titles and bodies.

## `tags` (type: `array`):

Return questions containing all supplied tags, for example javascript and fetch.

## `authorId` (type: `integer`):

Only return questions created by this numeric user ID.

## `minScore` (type: `integer`):

Only return questions with at least this score.

## `dateFrom` (type: `string`):

Earliest question creation date in YYYY-MM-DD format.

## `dateTo` (type: `string`):

Latest question creation date in YYYY-MM-DD format.

## `questionUrls` (type: `array`):

Optional Stack Overflow or Stack Exchange question URLs. Search filters are applied to fetched questions too.

## `includeAnswers` (type: `boolean`):

Export answer records after each matched question. Answers include body, score, author, acceptance, dates, and source URL.

## `maxQuestions` (type: `integer`):

Maximum number of unique question records to export (answers do not count against this limit).

## `sort` (type: `string`):

Stack Exchange search ordering.

## `order` (type: `string`):

Return highest/newest or lowest/oldest values first.

## `maxRequestRetries` (type: `integer`):

Retries for temporary API or network failures. The Actor also honors API-requested backoff.

## Actor input object example

```json
{
  "sites": [
    "stackoverflow"
  ],
  "query": "Stack Exchange API",
  "includeAnswers": true,
  "maxQuestions": 5,
  "sort": "relevance",
  "order": "desc",
  "maxRequestRetries": 2
}
```

# Actor output Schema

## `dataset` (type: `string`):

Questions and answers in the overview dataset view.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "sites": [
        "stackoverflow"
    ],
    "query": "Stack Exchange API",
    "maxQuestions": 5
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/stackexchange-questions-answers-export").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "sites": ["stackoverflow"],
    "query": "Stack Exchange API",
    "maxQuestions": 5,
}

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/stackexchange-questions-answers-export").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "sites": [
    "stackoverflow"
  ],
  "query": "Stack Exchange API",
  "maxQuestions": 5
}' |
apify call automation-lab/stackexchange-questions-answers-export --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,automation-lab/stackexchange-questions-answers-export"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/G1yhOQvTYcAGth413/builds/vnOdVQE7B2OxBssVx/openapi.json