# Stack Overflow Scraper - Questions, Answers & Comments (`legend006/stackoverflow-scraper`) Actor
Scrape questions, answers, and comments from Stack Overflow and the Stack Exchange network. Filter by tag, search, or user. Returns body, score, votes, accepted-answer flag. Built for AI/LLM training datasets, dev research, and tag-trend monitoring.
- **URL**: https://apify.com/legend006/stackoverflow-scraper.md
- **Developed by:** [NIJ KANANI](https://apify.com/legend006) (community)
- **Categories:** Developer tools, AI
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks
- **User rating**: No ratings yet
## Pricing
from $0.30 / 1,000 results
This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event
## What's an Apify Actor?
Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.
In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):
```bash
npm install apify-client
```
In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):
```bash
pip install apify-client
```
In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):
````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash
In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).
If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).
For usage examples, see the [API](#api) section below.
For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).
# README
## 🟧 Stack Overflow Scraper
Scrape **Stack Overflow and the entire Stack Exchange network** — Server Fault, Super User, Math.SE, AskUbuntu, Data Science, etc. Pull questions by tag, search, user, or top-of-period. Optionally fetch full answers and comments.
> 🎯 The #1 source of high-quality technical Q&A on the internet — perfect for AI/LLM training, sentiment analysis, and dev research.
---
### ✨ What you can do
- 🏷️ **Tag-based pulls** — `python`, `machine-learning`, `react-native`, etc.
- 🔍 **Full-text search** — Stack Exchange's `/search/advanced`
- 👤 **User activity** — fetch any user's questions
- 🔥 **Top of period** — top of week/month/all-time
- 📅 **Date range filter** — `fromDate` / `toDate`
- 🎯 **Min score filter** — only quality content
- 💬 **Optionally fetch answers + comments** for each question
- 🌐 **Works on any Stack Exchange site** (`site` parameter)
---
### 🚀 Quick start
```json
{
"site": "stackoverflow",
"mode": "tag",
"tags": ["python", "machine-learning"],
"sort": "votes",
"fromDate": "2026-01-01",
"minScore": 5,
"includeAnswers": true,
"includeComments": false,
"maxItems": 500
}
````
***
### 📥 Input
| Field | Description |
|---|---|
| `site` | SE site (`stackoverflow`, `serverfault`, `superuser`, `math`, `datascience`, ...) |
| `mode` | `tag` / `search` / `user` / `top` |
| `tags` | Tag names (mode = tag) |
| `searchQueries` | Free-text queries (mode = search) |
| `userIds` | Numeric SE user IDs (mode = user) |
| `sort` | `activity` / `creation` / `votes` / `hot` / `week` / `month` |
| `fromDate`, `toDate` | ISO date range filter |
| `minScore` | Skip below this score |
| `includeAnswers` | Fetch all answers per question |
| `includeComments` | Fetch comments on Q (and on A if includeAnswers) |
| `maxItems` | Cap per target |
| `apiKey` | (optional) Free key from https://stackapps.com — boosts quota 300→10,000/day |
***
### 📤 Output (per question)
```json
{
"site": "stackoverflow",
"type": "question",
"questionId": 12345678,
"title": "How to do X in Python?",
"body": "HTML body
",
"bodyMarkdown": "Markdown body",
"tags": ["python", "machine-learning"],
"score": 42,
"viewCount": 9876,
"answerCount": 3,
"isAnswered": true,
"acceptedAnswerId": 99999,
"creationDate": "2026-04-15T...",
"owner": { "userId": 123, "displayName": "user", "reputation": 50000, "profileUrl": "..." },
"link": "https://stackoverflow.com/questions/12345678/...",
"answers": [
{
"answerId": 99999,
"body": "Answer HTML
",
"score": 78,
"isAccepted": true,
"creationDate": "...",
"owner": { "userId": 456, "displayName": "expert", "reputation": 100000 }
}
],
"comments": [
{ "commentId": 555, "body": "Comment text", "score": 5, "creationDate": "...", "owner": {...} }
]
}
```
***
### 🎯 Use cases
| Who | Why |
|---|---|
| 🤖 AI / LLM teams | Best-in-class technical Q\&A for fine-tuning code/expert models |
| 📊 Dev relations | Track which language/framework tags are heating up |
| 🎓 Researchers | Code-discussion datasets, error-pattern analysis |
| 🛠️ Tool builders | Mine common pain points around your stack |
***
### ⚙️ Tech notes
- Uses Stack Exchange API v2.3
- Without API key: 300 requests/day quota (per IP)
- With free API key: **10,000 requests/day** — strongly recommended for large pulls
- Auto-honors `backoff` field if API asks us to slow down
- Includes both HTML `body` and `body_markdown`
***
### ❓ FAQ
**Where do I get an API key?**
Free at https://stackapps.com/apps/oauth/register — fill in any name, leave most fields blank. Takes 30 seconds.
**Are deleted/closed questions included?**
The API skips deleted; closed questions are returned but flagged in the data.
**Schedule it?**
Yes. Daily pulls of new questions on your favorite tags is a perfect Apify Schedule use case.
# Actor input Schema
## `site` (type: `string`):
Stack Exchange site short name. Default 'stackoverflow'. Examples: 'serverfault', 'superuser', 'math', 'askubuntu', 'unix', 'datascience'.
## `mode` (type: `string`):
'tag' = questions by tag(s). 'search' = full-text search. 'user' = user's questions/answers. 'top' = top voted of period.
## `tags` (type: `array`):
Tag names without '\[' brackets, e.g. 'python', 'machine-learning'. Used in mode = tag.
## `searchQueries` (type: `array`):
Free-text search. Used in mode = search.
## `userIds` (type: `array`):
Numeric Stack Exchange user IDs. Used in mode = user. Get from a profile URL: stackoverflow.com/users/{ID}/...
## `sort` (type: `string`):
Sort order. 'activity' / 'creation' / 'votes' / 'hot' / 'week' / 'month' (some sorts are mode-specific).
## `fromDate` (type: `string`):
Optional lower bound on creation date.
## `toDate` (type: `string`):
Optional upper bound on creation date.
## `minScore` (type: `integer`):
Skip questions with score below this threshold (filters in code).
## `includeAnswers` (type: `boolean`):
Fetch all answers for each question (slower, much richer).
## `includeComments` (type: `boolean`):
Fetch comments on questions (and answers if includeAnswers is on).
## `maxItems` (type: `integer`):
Cap per tag / per query / per user.
## `apiKey` (type: `string`):
Free key at https://stackapps.com/apps/oauth/register — boosts daily quota from 300 to 10,000 requests/day. Highly recommended for large scrapes.
## Actor input object example
```json
{
"site": "stackoverflow",
"mode": "tag",
"tags": [
"python"
],
"searchQueries": [],
"userIds": [],
"sort": "votes",
"minScore": 0,
"includeAnswers": false,
"includeComments": false,
"maxItems": 100
}
```
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {};
// Run the Actor and wait for it to finish
const run = await client.actor("legend006/stackoverflow-scraper").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = {}
# Run the Actor and wait for it to finish
run = client.actor("legend006/stackoverflow-scraper").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
print(item)
# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{}' |
apify call legend006/stackoverflow-scraper --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.apify.com/?tools=legend006/stackoverflow-scraper",
"--header",
"Authorization: Bearer "
]
}
}
}
```
## OpenAPI specification
```json
{
"openapi": "3.0.1",
"info": {
"title": "Stack Overflow Scraper - Questions, Answers & Comments",
"description": "Scrape questions, answers, and comments from Stack Overflow and the Stack Exchange network. Filter by tag, search, or user. Returns body, score, votes, accepted-answer flag. Built for AI/LLM training datasets, dev research, and tag-trend monitoring.",
"version": "0.1",
"x-build-id": "QGMFXNuUCR1IkRvGC"
},
"servers": [
{
"url": "https://api.apify.com/v2"
}
],
"paths": {
"/acts/legend006~stackoverflow-scraper/run-sync-get-dataset-items": {
"post": {
"operationId": "run-sync-get-dataset-items-legend006-stackoverflow-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
},
"/acts/legend006~stackoverflow-scraper/runs": {
"post": {
"operationId": "runs-sync-legend006-stackoverflow-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor and returns information about the initiated run in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/runsResponseSchema"
}
}
}
}
}
}
},
"/acts/legend006~stackoverflow-scraper/run-sync": {
"post": {
"operationId": "run-sync-legend006-stackoverflow-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
}
},
"components": {
"schemas": {
"inputSchema": {
"type": "object",
"required": [
"site",
"mode"
],
"properties": {
"site": {
"title": "Site",
"type": "string",
"description": "Stack Exchange site short name. Default 'stackoverflow'. Examples: 'serverfault', 'superuser', 'math', 'askubuntu', 'unix', 'datascience'.",
"default": "stackoverflow"
},
"mode": {
"title": "Scrape mode",
"enum": [
"tag",
"search",
"user",
"top"
],
"type": "string",
"description": "'tag' = questions by tag(s). 'search' = full-text search. 'user' = user's questions/answers. 'top' = top voted of period.",
"default": "tag"
},
"tags": {
"title": "Tags",
"type": "array",
"description": "Tag names without '[' brackets, e.g. 'python', 'machine-learning'. Used in mode = tag.",
"default": [
"python"
],
"items": {
"type": "string"
}
},
"searchQueries": {
"title": "Search queries",
"type": "array",
"description": "Free-text search. Used in mode = search.",
"default": [],
"items": {
"type": "string"
}
},
"userIds": {
"title": "User IDs",
"type": "array",
"description": "Numeric Stack Exchange user IDs. Used in mode = user. Get from a profile URL: stackoverflow.com/users/{ID}/...",
"default": [],
"items": {
"type": "string"
}
},
"sort": {
"title": "Sort",
"enum": [
"activity",
"creation",
"votes",
"hot",
"week",
"month"
],
"type": "string",
"description": "Sort order. 'activity' / 'creation' / 'votes' / 'hot' / 'week' / 'month' (some sorts are mode-specific).",
"default": "votes"
},
"fromDate": {
"title": "From date (ISO YYYY-MM-DD)",
"type": "string",
"description": "Optional lower bound on creation date."
},
"toDate": {
"title": "To date (ISO YYYY-MM-DD)",
"type": "string",
"description": "Optional upper bound on creation date."
},
"minScore": {
"title": "Minimum score",
"minimum": 0,
"type": "integer",
"description": "Skip questions with score below this threshold (filters in code).",
"default": 0
},
"includeAnswers": {
"title": "Include answers",
"type": "boolean",
"description": "Fetch all answers for each question (slower, much richer).",
"default": false
},
"includeComments": {
"title": "Include comments",
"type": "boolean",
"description": "Fetch comments on questions (and answers if includeAnswers is on).",
"default": false
},
"maxItems": {
"title": "Max questions per target",
"minimum": 1,
"maximum": 10000,
"type": "integer",
"description": "Cap per tag / per query / per user.",
"default": 100
},
"apiKey": {
"title": "Stack Exchange API key (optional)",
"type": "string",
"description": "Free key at https://stackapps.com/apps/oauth/register — boosts daily quota from 300 to 10,000 requests/day. Highly recommended for large scrapes."
}
}
},
"runsResponseSchema": {
"type": "object",
"properties": {
"data": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"actId": {
"type": "string"
},
"userId": {
"type": "string"
},
"startedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"finishedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"status": {
"type": "string",
"example": "READY"
},
"meta": {
"type": "object",
"properties": {
"origin": {
"type": "string",
"example": "API"
},
"userAgent": {
"type": "string"
}
}
},
"stats": {
"type": "object",
"properties": {
"inputBodyLen": {
"type": "integer",
"example": 2000
},
"rebootCount": {
"type": "integer",
"example": 0
},
"restartCount": {
"type": "integer",
"example": 0
},
"resurrectCount": {
"type": "integer",
"example": 0
},
"computeUnits": {
"type": "integer",
"example": 0
}
}
},
"options": {
"type": "object",
"properties": {
"build": {
"type": "string",
"example": "latest"
},
"timeoutSecs": {
"type": "integer",
"example": 300
},
"memoryMbytes": {
"type": "integer",
"example": 1024
},
"diskMbytes": {
"type": "integer",
"example": 2048
}
}
},
"buildId": {
"type": "string"
},
"defaultKeyValueStoreId": {
"type": "string"
},
"defaultDatasetId": {
"type": "string"
},
"defaultRequestQueueId": {
"type": "string"
},
"buildNumber": {
"type": "string",
"example": "1.0.0"
},
"containerUrl": {
"type": "string"
},
"usage": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "integer",
"example": 1
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
},
"usageTotalUsd": {
"type": "number",
"example": 0.00005
},
"usageUsd": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "number",
"example": 0.00005
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
}
}
}
}
}
}
}
}
```