# Stack Overflow Scraper - Questions, Answers, Tags (`s-r/stackexchange-scraper`) Actor

Pull questions and answers from Stack Overflow and 170+ Stack Exchange sites by tag or search term. Returns question and answer bodies, score, views, accepted-answer status and closure reasons via the official API. Free API key optional.

- **URL**: https://apify.com/s-r/stackexchange-scraper.md
- **Developed by:** [SR](https://apify.com/s-r) (community)
- **Categories:** Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 run start fees

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Stack Overflow Scraper

Pull questions and answers from **Stack Overflow and 170+ other Stack Exchange
sites** by tag or search term. You get the question text, the accepted answer,
the highest-scoring answer, the score, the view count, and whether the question
was closed and why.

Filter by tag, search free text, pick any network site, sort by votes or
activity.

### The site is closed, the API is open

Worth stating plainly: **stackoverflow.com returns 403 to a plain HTTP client.**
across two network exits, the website refused every
request while `api.stackexchange.com` answered 200 on the same run.

So this Actor is not taking a shortcut by using the API. The API is the only
working door, and it is a documented, key-free one that returns better
structured data than the page ever did.

### The quota, stated up front

Stack Exchange allows **300 requests per day per IP** without a key, and
**10,000 with a free key**. One request returns up to 100 questions, so the
unauthenticated allowance is roughly 30,000 questions a day. Enough for most
jobs, not enough for a bulk harvest.

A key takes about a minute to create at
`stackapps.com/apps/oauth/register`, needs no approval, and goes in the
`api_key` field.

This Actor treats the quota as a real budget rather than something to route
around:

- `quota_remaining` is **read from every response**, not assumed, and reported
  in the run summary.
- When fewer than three requests remain, the run **stops and says so** instead
  of draining the last of the allowance.
- Throttling that arrives inside a `200` body, which is how Stack Exchange
  usually signals it, is detected and reported as `quota_exhausted` rather than
  parsed as an empty page.

Rows collected before a quota stop are complete and are returned.

### What comes back

Per question:

- **Identity**: `question_id`, `site`, `title`, `url`, `tags`
- **Signal**: `score`, `view_count`, `answer_count`, `comment_count`
- **Resolution**: `is_answered`, `has_accepted_answer`, `accepted_answer_id`
- **Bodies**: `question_body`, `accepted_answer_body`, `top_answer_body`,
  `top_answer_score` — all HTML stripped and entity decoded
- **Lifecycle**: `created_at`, `last_activity_at`, `closed_at`, `closed_reason`,
  `is_closed`
- **Asker**: `asker`, `asker_reputation`, `asker_url`

Two derived fields worth knowing about:

**`views_per_score`** is views divided by net score. A question with 400,000
views and a score of 12 is a problem thousands of people hit that the site never
rewarded — which is exactly what documentation and content research is hunting
for. Sort descending on it.

**`unanswered_but_popular`** flags questions with over 1,000 views and no
accepted-standard answer. On a tag you own, that list is a content backlog.

### Accepted is not the same as top

`accepted_answer_body` is the answer **the asker chose**, matched by id.
`top_answer_body` is the **highest scoring** answer. They are frequently
different, and the gap is informative: a low-scoring accepted answer next to a
high-scoring rival usually means the accepted one solved the asker's specific
case while the community preferred a better general answer.

Both are returned so you never have to guess which one you are reading.

### Input reference

| Field | Type | Default |
|---|---|---|
| `tagged` | list of tags, all must match | `["python", "asyncio"]` |
| `search` | free text, uses advanced search | — |
| `site` | any network site | `stackoverflow` |
| `sort` | votes, activity, creation, hot, week, month | `votes` |
| `order` | desc, asc | `desc` |
| `min_score` | integer | — |
| `api_key` | free key from stackapps.com | — |
| `limit` | 1-2000 | 100 |
| `retries` | 1-6 | 3 |

Give at least a tag or a search term. An empty input is rejected rather than
walking the whole site, because that would burn the daily quota on nothing.

Other network sites are just a parameter: `serverfault`, `superuser`,
`askubuntu`, `math`, `unix`, `dba`, `security`, `datascience`, and the rest.

### Typical uses

- **Content and docs research.** Take the tags for your product, sort by
  `views_per_score`, and you have a ranked list of things people struggle with
  and nobody answered well.
- **Support triage.** Watch `unanswered_but_popular` on your own tags.
- **Competitive and ecosystem research.** Compare question volume and
  answer rates across tags to see which tools people are actually adopting.
- **Training and evaluation data.** Question plus accepted answer, with score
  and view count as quality signals.

### Notes

`is_answered` is Stack Exchange's own flag and means an answer scored above the
site's threshold, which is not the same as the asker accepting one. Both are
returned separately for that reason.

Answer bodies are truncated at 4,000 characters. HTML is stripped, so code
blocks lose their formatting but keep their text.

Please respect the quota rather than working around it. This API is free, it
does not require an account, and it stays that way because people use it as
intended.

# Actor input Schema

## `tagged` (type: `array`):

Only questions carrying all of these tags, for example python and asyncio.

## `search` (type: `string`):

Free-text search instead of, or alongside, tags. Uses the advanced search endpoint.

## `site` (type: `string`):

Which Stack Exchange site to query: stackoverflow, serverfault, superuser, askubuntu, math, and 170 more.

## `sort` (type: `string`):

Which questions come first.

## `order` (type: `string`):

Sort direction. Descending puts the highest scoring or most recent questions first.

## `min_score` (type: `integer`):

Skip questions below this net vote count.

## `api_key` (type: `string`):

Without a key the API allows 300 requests per day per IP. A free key from stackapps.com/apps/oauth/register raises it to 10,000 and needs no approval.

## `limit` (type: `integer`):

How many questions to return. One API request covers 100, so this also decides how much daily quota the run uses.

## `retries` (type: `integer`):

Retries with backoff before a request is reported as an error.

## Actor input object example

```json
{
  "tagged": [
    "python",
    "asyncio"
  ],
  "search": "nginx reverse proxy",
  "site": "stackoverflow",
  "sort": "votes",
  "order": "desc",
  "limit": 100,
  "retries": 3
}
```

# Actor output Schema

## `questions` (type: `string`):

One row per question, with its bodies and answer stats.

## `summary` (type: `string`):

Counts, answer coverage and the remaining daily API quota.

## `errors` (type: `string`):

Failures with a code and a redacted message.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "tagged": [
        "python",
        "asyncio"
    ],
    "site": "stackoverflow",
    "limit": 100,
    "retries": 3
};

// Run the Actor and wait for it to finish
const run = await client.actor("s-r/stackexchange-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "tagged": [
        "python",
        "asyncio",
    ],
    "site": "stackoverflow",
    "limit": 100,
    "retries": 3,
}

# Run the Actor and wait for it to finish
run = client.actor("s-r/stackexchange-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "tagged": [
    "python",
    "asyncio"
  ],
  "site": "stackoverflow",
  "limit": 100,
  "retries": 3
}' |
apify call s-r/stackexchange-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,s-r/stackexchange-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Q0JAMUDeziYE68nAH/builds/ihVwW1dt7LXnZcD0Q/openapi.json