# Stack Overflow Scraper — Q\&A, Tags, All 170 Sites | No Auth (`bovi/stack-exchange-scraper`) Actor

Scrape questions, answers, and users from Stack Overflow and all 170+ Stack Exchange sites via the official public API. No auth needed. Filter by tags, search queries, or question IDs. Pay per result.

- **URL**: https://apify.com/bovi/stack-exchange-scraper.md
- **Developed by:** [Vitalii Bondarev](https://apify.com/bovi) (community)
- **Categories:** Developer tools, MCP servers, AI
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.45 / 1,000 stack overflow scraper — q\&a, tags, all 170 sites | no auths

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Stack Overflow Scraper — Q\&A, Tags & All 170 Sites | from $1.50/1K No Auth

Used by developer-education platforms, AI coding assistants, and enterprise knowledge-base teams.

Scrape **questions, answers, and users** from Stack Overflow and all **170+ Stack Exchange sites**
via the official public API — no proxy, no auth required, no rate-limit tricks.

### What You Can Scrape

- **Questions** — titles, tags, scores, answer counts, full body HTML, owner info, dates
- **Answers** — body HTML, accepted status, vote score, linked question ID
- **Users** — display name, reputation, creation date, profile link

All 170+ Stack Exchange network sites are supported:
`stackoverflow`, `superuser`, `serverfault`, `askubuntu`, `math`, `physics`,
`datascience`, `security`, `unix`, `softwareengineering`, `codereview`, and more.

### Why Stack Exchange Scraper Beats Alternatives

| Feature | This actor | Typical alternatives |
|---|---|---|
| **parse\_confidence** field | ✅ every row | ❌ none |
| All 170+ SE sites | ✅ single input | ❌ SO-only |
| Questions + Answers + Users | ✅ all 3 | ❌ questions only |
| Official API (no proxy) | ✅ $0 infra | ❌ proxy required |
| Tag filter + search + ID fetch | ✅ all modes | ❌ limited |
| Body text (stripped HTML) | ✅ included | ❌ HTML only |
| Pay per result | ✅ PPE | ❌ rental / flat fee |

### How to Use

#### Scrape questions by tag (most common)

Set **Data type** = `questions`, **Tags** = `["python"]`, **Site** = `stackoverflow`.
Combine multiple tags for AND filtering: `["python", "pandas"]` returns questions that have both tags.

#### Full-text search

Set **Search queries** = `["python asyncio tutorial"]`. Uses the official `/search/advanced` endpoint
with relevance ranking. Supports quoted phrases.

#### Fetch answers for specific questions

Set **Data type** = `answers`, **Question IDs** = `["27435284"]` to get all answers
for a specific question sorted by votes.

#### Fetch top users

Set **Data type** = `users`, leave **User IDs** blank → returns users sorted by reputation.
Set specific **User IDs** to fetch exact user profiles.

### Output Schema

Every row (question, answer, or user) shares the same flat schema:

| Field | Type | Description |
|---|---|---|
| `site` | string | SE site slug (e.g. "stackoverflow") |
| `type` | string | "question" | "answer" | "user" |
| `question_id` | integer | Question ID (null for users) |
| `answer_id` | integer | Answer ID (null for questions/users) |
| `user_id` | integer | Owner's user ID |
| `title` | string | Question title (null for answers/users) |
| `body_html` | string | Full body HTML (null if includeBody=false) |
| `body_text` | string | HTML-stripped plain text body |
| `tags` | array | Tag list (questions only) |
| `score` | integer | Vote score (null for users) |
| `answer_count` | integer | Number of answers (questions only) |
| `is_answered` | boolean | Has accepted answer (questions only) |
| `is_accepted` | boolean | This answer is accepted (answers only) |
| `owner_name` | string | Display name of owner |
| `owner_id` | integer | User ID of owner |
| `owner_reputation` | integer | Reputation at scrape time |
| `creation_date` | ISO 8601 | When the item was created |
| `last_activity_date` | ISO 8601 | Last activity |
| `link` | string | Canonical URL |
| `query` | string | Provenance: the search query or tag |
| `parse_confidence` | float | 0–1 data quality score |
| `warnings` | array | Quality warning labels |
| `scraped_at` | ISO 8601 | Scrape timestamp |

### API Key (Optional)

Without an API key: **300 requests/day** (shared by IP).
With a free API key from [stackapps.com](https://stackapps.com/apps/oauth/register): **10,000 requests/day**.

Each page of 100 results = 1 API request. For most use cases the anonymous quota is sufficient.

### What Does `parse_confidence` Mean?

Every record gets a quality score from 0.0 to 1.0:

- **1.0** — all expected fields present
- **< 0.9** — minor missing fields (no title, no owner)
- **< 0.7** — major missing fields (no ID, no creation date)

Filter `parse_confidence >= 0.9` for clean downstream processing.

### Pricing examples

| Run | Items | Cost |
|---|---|---|
| 100 questions (metadata only) | 100 | ~$0.15 |
| 1,000 questions with body text | 1,000 | ~$2.00 |
| 500 answers for a tag | 500 | ~$0.75 |
| Daily monitoring (100 new questions) | 3,000/mo | ~$4.50/mo |

You only pay for items successfully pushed to the dataset. Body HTML is billed as an additional event only when non-null.

### FAQ

**Do I need an API key or proxy?**
No proxy required — the Stack Exchange API is public. Without an API key you get ~300 requests/day (shared by IP). For production runs get a free key at [stackapps.com](https://stackapps.com/apps/oauth/register) for 10,000 req/day.

**Which Stack Exchange sites are supported?**
All 170+ sites. Set the `site` field to the slug (e.g. `stackoverflow`, `superuser`, `datascience`, `security`). Full list at [stackexchange.com/sites](https://stackexchange.com/sites).

**Can I schedule it to monitor new questions?**
Yes. Use Apify Schedules + `fromDate` to pull only questions newer than your last run. Combine with a webhook to push new items into Slack, Airtable, or a database.

**What if it returns empty results?**
Check your `site` slug is correct and your `tags` filter is valid (tags are lowercase slugs, e.g. `python` not `Python`). For `answers` mode you must provide `questionIds` or leave it blank to browse recent answers. Errors are logged to the `OUTPUT` key-value store.

### Pricing

**Pricing:** $1.50 per 1,000 items. Enable `includeBody` for full Q\&A text — additional $0.50/1k for body content.

No API key required for standard usage (300 req/day anonymous). Supply a free Stack Apps key from [stackapps.com](https://stackapps.com/apps/oauth/register) for 10,000 req/day.

### Sites Reference

Full list at [stackexchange.com/sites](https://stackexchange.com/sites). Common slugs:

`stackoverflow` · `superuser` · `serverfault` · `askubuntu` · `math` · `physics` ·
`datascience` · `security` · `unix` · `softwareengineering` · `codereview` ·
`stats` · `dba` · `networkengineering` · `devops` · `gaming` · `electronics`

### Use with AI agents (MCP)

This actor is callable as a **tool by AI agents** (Claude Desktop, Cursor, VS Code,
n8n, LangGraph, CrewAI, or any MCP-compatible client) via Apify's hosted Model Context
Protocol server. An agent uses it to **fetch Stack Overflow and Stack Exchange Q\&A
mid-conversation** — e.g. "find the top-voted Python asyncio questions", "get all answers
for question 27435284", or "pull the latest security advisories tagged CVE from
security.stackexchange.com".

Point your MCP client at this single tool:

```json
{
  "mcpServers": {
    "apify": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://mcp.apify.com/?tools=bovi/stack-exchange-scraper",
        "--header",
        "Authorization: Bearer <YOUR_APIFY_TOKEN>"
      ]
    }
  }
}
```

Minimal call an agent can make (search mode — set `maxItems` low to keep responses token-lean):

```json
{ "site": "stackoverflow", "dataType": "questions", "searchQueries": ["python asyncio"], "sort": "votes", "maxItems": 10 }
```

Returns clean, flat rows the agent can reason over directly:

```json
{
  "site": "stackoverflow",
  "type": "question",
  "question_id": 49005651,
  "title": "How does asyncio actually work?",
  "tags": ["python", "python-asyncio", "python-3.x"],
  "score": 892,
  "answer_count": 7,
  "is_answered": true,
  "body_text": "I've been trying to understand asyncio...",
  "owner_name": "J_H",
  "link": "https://stackoverflow.com/questions/49005651",
  "creation_date": "2018-02-26T00:00:00Z",
  "parse_confidence": 1.0,
  "warnings": []
}
```

Reliability for agents: data comes from the **official Stack Exchange public API** (no
HTML scraping), so rows don't silently break on site redesigns. Every row includes a
`parse_confidence` score (0–1) — filter `parse_confidence >= 0.9` for clean downstream
processing. The `includeBody: false` option halves response size when only titles and
metadata are needed. No API key required inside the tool — auth is your Apify token in
the client config above.

### Integrations

Built for developer-education platforms and AI teams building knowledge bases from Stack Overflow and the broader network — the JSON/dataset output drops into the tools you already run, no glue code:

- **n8n / Make / Zapier** — trigger a run or pipe every new dataset item into 500+ apps (Google Sheets, Airtable, Slack, HubSpot, your database) with no code: [n8n](https://docs.apify.com/platform/integrations/n8n), [Make](https://docs.apify.com/platform/integrations/make), [Zapier](https://docs.apify.com/platform/integrations/zapier).
- **Webhooks** — fire your own endpoint the moment a run finishes, to push results straight into your pipeline ([docs](https://docs.apify.com/platform/integrations/webhooks)).
- **MCP server** — expose this actor as a tool to Claude, Cursor, or any [MCP client](https://mcp.apify.com) so an AI agent can pull this data mid-conversation ([guide](https://blog.apify.com/how-to-use-mcp/)).
- **API & SDKs** — fetch the dataset as JSON, CSV, or Excel through the Apify REST API or the Python / JS SDKs.

See all [Apify integrations](https://apify.com/integrations).

### Disclaimer

Not affiliated with Stack Exchange Inc. Uses the official public Stack Exchange API
under its terms of service. Data is licensed CC BY-SA 4.0 by contributors.

# Actor input Schema

## `site` (type: `string`):

Stack Exchange site slug to query. Use 'stackoverflow' for Stack Overflow; 'superuser', 'serverfault', 'askubuntu', 'math', 'physics', 'datascience', 'security', 'unix', 'softwareengineering' for other sites. Full list at stackexchange.com/sites.

## `dataType` (type: `string`):

Record type to fetch. 'questions' returns question threads with title, tags, score, body text. 'answers' returns answer bodies (use with questionIds for a specific thread). 'users' returns user profiles with reputation.

## `searchQueries` (type: `array`):

Keyword search strings. Each query runs against the Stack Exchange /search/advanced endpoint. Use quoted phrases for exact match (e.g. '"python asyncio"'). Mutually exclusive with tags; takes priority. Only used when dataType=questions.

## `tags` (type: `array`):

Filter questions by tag names (e.g. \['python'], \['python','pandas']). Multiple tags = AND filter — all tags must be present. Used only when dataType=questions and searchQueries is empty.

## `questionIds` (type: `array`):

Fetch specific items by Stack Exchange question ID (integer as string). When dataType=questions, returns those question records. When dataType=answers, returns all answers for those question IDs. Example: \['27435284', '1732348'].

## `userIds` (type: `array`):

Fetch specific user profiles by Stack Exchange user ID (integer as string). Leave empty to browse top users by reputation. Only used when dataType=users. Example: \['22656', '115145'].

## `sort` (type: `string`):

Sort order applied to results. 'activity': most recently active first. 'votes': highest vote score first. 'creation': newest items first. 'relevance': best search-match first (only valid in search mode).

## `maxItems` (type: `integer`):

Maximum total records to return across all queries combined. Keep low (50–200) for agent calls to avoid token bloat. Set 0 for no limit (use with caution on broad tags).

## `apiKey` (type: `string`):

Optional Stack Exchange API key. Anonymous quota: ~300 requests/day (shared by IP). With a free key from stackapps.com: 10,000 requests/day. Leave blank for most use cases — 100 results = 1 request.

## `includeBody` (type: `boolean`):

Include raw HTML in the body\_html field in addition to the stripped body\_text. Set false to halve response size when only body\_text is needed. body\_text (plain text) is always included.

## `fromDate` (type: `string`):

Return only items created on or after this date. ISO 8601 format (e.g. '2024-01-01'). Leave blank for no lower bound.

## `toDate` (type: `string`):

Return only items created on or before this date. ISO 8601 format (e.g. '2024-12-31'). Leave blank for no upper bound.

## Actor input object example

```json
{
  "site": "stackoverflow",
  "dataType": "questions",
  "searchQueries": [
    "python asyncio",
    "javascript promises"
  ],
  "tags": [
    "python"
  ],
  "questionIds": [
    "27435284"
  ],
  "userIds": [
    "22656"
  ],
  "sort": "activity",
  "maxItems": 100,
  "includeBody": true
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset containing Stack Exchange Scraper records (type, question\_id, title, tags, score, answer\_count, is\_answered, owner\_name, creation\_date, site, link, parse\_confidence).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "site": "stackoverflow",
    "dataType": "questions",
    "searchQueries": [
        "python asyncio",
        "javascript promises"
    ],
    "tags": [
        "python"
    ],
    "questionIds": [
        "27435284"
    ],
    "userIds": [
        "22656"
    ],
    "sort": "activity",
    "maxItems": 100,
    "includeBody": true
};

// Run the Actor and wait for it to finish
const run = await client.actor("bovi/stack-exchange-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "site": "stackoverflow",
    "dataType": "questions",
    "searchQueries": [
        "python asyncio",
        "javascript promises",
    ],
    "tags": ["python"],
    "questionIds": ["27435284"],
    "userIds": ["22656"],
    "sort": "activity",
    "maxItems": 100,
    "includeBody": True,
}

# Run the Actor and wait for it to finish
run = client.actor("bovi/stack-exchange-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "site": "stackoverflow",
  "dataType": "questions",
  "searchQueries": [
    "python asyncio",
    "javascript promises"
  ],
  "tags": [
    "python"
  ],
  "questionIds": [
    "27435284"
  ],
  "userIds": [
    "22656"
  ],
  "sort": "activity",
  "maxItems": 100,
  "includeBody": true
}' |
apify call bovi/stack-exchange-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,bovi/stack-exchange-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/7wYOe8pP8K5yiB8Cw/builds/l2DWh1HJu1JqucCKt/openapi.json