# Stack Exchange Questions Scraper (`fetch_cat/stack-exchange-questions-scraper`) Actor Collect public Stack Overflow and Stack Exchange questions by site, tag, keyword, date, score, and answers for SEO, DevRel, product, and support research. - **URL**: https://apify.com/fetch\_cat/stack-exchange-questions-scraper.md - **Developed by:** [Hanna Nosova](https://apify.com/fetch_cat) (community) - **Categories:** Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.02 / 1,000 question saveds This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Stack Exchange and Stack Overflow Questions Scraper Collect public Stack Overflow and Stack Exchange questions by site, tag, keyword, date range, and score. Use this actor to turn public developer Q&A activity into a clean dataset for SEO, product research, support insights, competitive monitoring, and developer-relations planning. --- ### What does Stack Exchange and Stack Overflow Questions Scraper do? Stack Exchange Questions Scraper extracts public question metadata from Stack Overflow and other Stack Exchange communities. It returns structured rows with question title, URL, tags, author, score, answer count, view count, answer status, dates, and license information. You can search one site or several sites in the same run. You can filter by tags, free-text query, date range, and minimum score. You can also enable answer summaries when you need more context around which questions received useful responses. --- ### Who is it for? #### SEO and content teams Find real questions people ask about programming languages, frameworks, libraries, tools, and errors. #### Developer-relations teams Track community pain points around your SDK, API, product, integration, or competitor. #### Product managers Discover recurring feature requests, confusion, migration problems, and adoption blockers. #### Support teams Monitor public troubleshooting topics and prepare proactive documentation. #### Market researchers Compare tag activity across technologies, products, ecosystems, and communities. --- ### Why use this actor? - ✅ Search public Stack Overflow and Stack Exchange questions without building your own collector - ✅ Export clean JSON, CSV, Excel, HTML, XML, or RSS from Apify datasets - ✅ Run scheduled monitoring jobs for tags and keywords - ✅ Combine multiple Stack Exchange sites in one run - ✅ Use date and score filters to focus on useful questions - ✅ Integrate results into spreadsheets, BI tools, notebooks, or automations --- ### What data can you extract? | Field | Description | | --- | --- | | `site` | Stack Exchange site name used for the question | | `questionId` | Stack Exchange question identifier | | `title` | Question title | | `url` | Public question URL | | `tags` | Question tags | | `ownerDisplayName` | Display name of the question owner when available | | `ownerUrl` | Public profile URL when available | | `score` | Question score | | `answerCount` | Number of answers | | `viewCount` | Number of views | | `isAnswered` | Whether the question is considered answered | | `acceptedAnswerId` | Accepted answer ID when present | | `creationDate` | Question creation date | | `lastActivityDate` | Last activity date | | `lastEditDate` | Last edit date when present | | `contentLicense` | Content license shown by Stack Exchange | | `answers` | Optional answer summaries when enabled | --- ### How much does it cost to scrape Stack Exchange questions? Pricing is pay-per-event. There is a small `$0.005` run-start charge and a low per-question charge for each saved question row. | Tier | Price per saved question | | --- | ---: | | FREE | `$0.000034022` | | BRONZE | `$0.000029585` | | SILVER | `$0.000023076` | | GOLD | `$0.000017751` | | PLATINUM | `$0.000011834` | | DIAMOND | `$0.00001` | The default prefilled run saves only 20 questions so you can test the actor cheaply. For large monitoring jobs, set `maxItems` to the volume you need and export the dataset after the run finishes. --- ### How to scrape Stack Exchange questions 1. Open the actor on Apify. 2. Enter one or more Stack Exchange site names, for example `stackoverflow` or `serverfault`. 3. Add one or more tags, such as `python`, `react`, or `docker`. 4. Optionally add a keyword query like `timeout error`. 5. Choose a sort order. 6. Set `maxItems`. 7. Run the actor. 8. Download your dataset from the Storage tab. --- ### Input options #### `sites` A list of Stack Exchange site names. Examples: - `stackoverflow` - `serverfault` - `superuser` - `askubuntu` - `datascience` - `math` #### `tags` A list of tags to filter questions. Examples: - `python` - `javascript` - `reactjs` - `docker` - `kubernetes` #### `query` Optional keyword search text. Use this for phrases, errors, product names, or broad topics. #### `sort` Supported values: - `activity` - `votes` - `creation` - `hot` - `week` - `month` - `relevance` #### `fromDate` and `toDate` Optional date filters in ISO format, such as `2026-01-01`. #### `minScore` Minimum score for saved questions. Use `0` to ignore downvoted questions. #### `includeAnswers` Set to `true` to add answer summaries to each question row. --- ### Example input ```json { "sites": ["stackoverflow"], "tags": ["python"], "query": "pandas dataframe", "sort": "activity", "maxItems": 20, "includeAnswers": false, "minScore": 0 } ```` *** ### Example output ```json { "site": "stackoverflow", "questionId": 123456, "title": "How do I filter rows in a pandas DataFrame?", "url": "https://stackoverflow.com/questions/123456/example", "tags": ["python", "pandas", "dataframe"], "ownerDisplayName": "developer123", "ownerUrl": "https://stackoverflow.com/users/123/developer123", "score": 42, "answerCount": 3, "viewCount": 10000, "isAnswered": true, "acceptedAnswerId": 123457, "creationDate": "2026-01-01T12:00:00.000Z", "lastActivityDate": "2026-01-03T09:30:00.000Z", "lastEditDate": "2026-01-02T08:15:00.000Z", "contentLicense": "CC BY-SA 4.0" } ``` *** ### Tips for better results - Use exact product or library tags when they exist. - Use `votes` to find high-signal evergreen questions. - Use `creation` to monitor new questions. - Use `activity` for recently updated discussions. - Use `minScore` to reduce noisy rows. - Keep first test runs small, then scale up after reviewing output. - Use multiple sites when researching topics that span admin, devops, and programming communities. *** ### Common workflows #### Content gap research Run tags like `python`, `pandas`, or `reactjs` and sort by votes. Use top questions to plan tutorials, FAQs, and comparison pages. #### Support trend monitoring Schedule a daily run for your product tag or company keyword. Send new rows to Slack, Airtable, Google Sheets, or a help-center backlog. #### Competitive research Search competitor product names and integration tags. Group questions by score, views, answer count, and recency. #### DevRel topic discovery Monitor language and framework tags to find what developers are struggling with this week. *** ### Integrations You can connect the dataset to: - Google Sheets for editorial planning - Airtable for research queues - Notion for content briefs - Slack for alerting - BigQuery or Snowflake for trend analysis - Zapier or Make for no-code automations - Webhooks for custom pipelines *** ### Use as a public Stack Exchange API workflow Use this actor as a repeatable public Stack Exchange and Stack Overflow API workflow for tag monitoring, keyword research, content-gap analysis, DevRel planning, and support trend tracking. It uses public Stack Exchange data/API behavior, respects platform limits, and does not access private account data. *** ### API usage #### Node.js ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('fetch_cat/stack-exchange-questions-scraper').call({ sites: ['stackoverflow'], tags: ['python'], query: 'pandas dataframe', maxItems: 20 }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items); ``` #### Python ```python from apify_client import ApifyClient import os client = ApifyClient(os.environ['APIFY_TOKEN']) run = client.actor('fetch_cat/stack-exchange-questions-scraper').call(run_input={ 'sites': ['stackoverflow'], 'tags': ['python'], 'query': 'pandas dataframe', 'maxItems': 25, }) items = client.dataset(run['defaultDatasetId']).list_items().items print(items) ``` #### cURL ```bash curl -X POST "https://api.apify.com/v2/acts/fetch_cat~stack-exchange-questions-scraper/runs?token=$APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{"sites":["stackoverflow"],"tags":["python"],"maxItems":20}' ``` *** ### MCP integration Use Apify MCP to run this actor from AI tools. MCP URL: ```text https://mcp.apify.com/?tools=fetch_cat/stack-exchange-questions-scraper ``` Claude Code setup: ```bash claude mcp add apify-stack-exchange-questions https://mcp.apify.com/?tools=fetch_cat/stack-exchange-questions-scraper ``` Claude Desktop JSON config example: ```json { "mcpServers": { "apify-stack-exchange-questions": { "url": "https://mcp.apify.com/?tools=fetch_cat/stack-exchange-questions-scraper" } } } ``` After connecting, ask your AI assistant to run `fetch_cat/stack-exchange-questions-scraper` with the sites, tags, and filters you need. Example prompts showing MCP usage: - "Use the `fetch_cat/stack-exchange-questions-scraper` MCP tool to find recent Stack Overflow questions about Python pandas performance and summarize common pain points." - "Use the Stack Exchange Questions Scraper MCP tool to monitor Docker questions with score above 2 and list topics our docs should cover." - "Run `fetch_cat/stack-exchange-questions-scraper` via MCP to compare activity for Kubernetes and Docker tags over the last month." *** ### Data freshness The actor collects currently available public question metadata at run time. Use scheduled runs for ongoing monitoring. Use date filters to build repeatable daily, weekly, or monthly snapshots. *** ### Limits and behavior Stack Exchange may return fewer questions than requested if the filters are narrow. Some owner fields may be missing for deleted or anonymous accounts. Some questions do not have an accepted answer. Answer summaries require extra requests and may make runs take longer. *** ### Legality and responsible use This actor extracts publicly available question metadata. You are responsible for using the data according to applicable laws, platform terms, and your own compliance requirements. Respect attribution and license requirements for Stack Exchange content. Do not use the output for spam, harassment, or abusive automation. *** ### Troubleshooting #### Why did I get fewer rows than `maxItems`? Your filters may be too narrow, the selected site may have fewer matching questions, or the date range may be restrictive. Try broadening tags, removing `query`, lowering `minScore`, or increasing the date range. #### Why are owner fields empty? Some public question records do not include full owner details, especially when an account was removed or anonymized. #### Which site names should I use? Use Stack Exchange API site names, such as `stackoverflow`, `serverfault`, `superuser`, `askubuntu`, `math`, or `datascience`. *** ### FAQ #### Can I scrape multiple sites at once? Yes. Add several site names to `sites`; the actor will collect matching questions across them until `maxItems` is reached. #### Can I scrape Stack Overflow questions by tag and keyword together? Yes. Use `query` with `tags` to focus on a specific topic inside a Stack Overflow or Stack Exchange tag. #### Can I include answers? Yes. Enable `includeAnswers` to include answer summaries on each question row. #### Can I run this actor on a schedule? Yes. Use Apify schedules to run the same input daily, weekly, or monthly. #### Can I export to CSV or Excel? Yes. Apify datasets can be exported as JSON, CSV, Excel, XML, HTML, and RSS. *** ### Changelog #### 0.1 Initial version with multi-site question collection, tag filters, keyword search, date filters, minimum score filtering, and optional answer summaries. # Actor input Schema ## `sites` (type: `array`): Stack Exchange API site names, such as stackoverflow, serverfault, superuser, askubuntu, math, or datascience. ## `tags` (type: `array`): Question tags to match. Multiple tags are combined by Stack Exchange as a tag intersection. ## `query` (type: `string`): Optional free-text search query. Leave empty to browse questions by site/tag/sort only. ## `sort` (type: `string`): Stack Exchange question sort mode. Query search supports activity, votes, creation, and relevance; unsupported query sorts fall back to activity. ## `maxItems` (type: `integer`): Maximum number of question rows to save across all selected sites. ## `fromDate` (type: `string`): Optional ISO date lower bound, for example 2026-01-01. ## `toDate` (type: `string`): Optional ISO date upper bound, for example 2026-01-31. ## `minScore` (type: `integer`): Only save questions with at least this score. Use 0 to skip downvoted questions, or leave negative to include everything. ## `includeAnswers` (type: `boolean`): Fetch answer summaries for each saved question and include them in an answers array. This uses extra API requests. ## Actor input object example ```json { "sites": [ "stackoverflow" ], "tags": [ "python" ], "query": "", "sort": "activity", "maxItems": 20, "minScore": -999999, "includeAnswers": false } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "sites": [ "stackoverflow" ], "tags": [ "python" ], "maxItems": 20 }; // Run the Actor and wait for it to finish const run = await client.actor("fetch_cat/stack-exchange-questions-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "sites": ["stackoverflow"], "tags": ["python"], "maxItems": 20, } # Run the Actor and wait for it to finish run = client.actor("fetch_cat/stack-exchange-questions-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "sites": [ "stackoverflow" ], "tags": [ "python" ], "maxItems": 20 }' | apify call fetch_cat/stack-exchange-questions-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=fetch_cat/stack-exchange-questions-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Stack Exchange Questions Scraper", "description": "Collect public Stack Overflow and Stack Exchange questions by site, tag, keyword, date, score, and answers for SEO, DevRel, product, and support research.", "version": "0.1", "x-build-id": "ZWiYtEujlhzrYP6ZU" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/fetch_cat~stack-exchange-questions-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-fetch_cat-stack-exchange-questions-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/fetch_cat~stack-exchange-questions-scraper/runs": { "post": { "operationId": "runs-sync-fetch_cat-stack-exchange-questions-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/fetch_cat~stack-exchange-questions-scraper/run-sync": { "post": { "operationId": "run-sync-fetch_cat-stack-exchange-questions-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "sites" ], "properties": { "sites": { "title": "Stack Exchange sites", "type": "array", "description": "Stack Exchange API site names, such as stackoverflow, serverfault, superuser, askubuntu, math, or datascience.", "default": [ "stackoverflow" ], "items": { "type": "string" } }, "tags": { "title": "Tags", "type": "array", "description": "Question tags to match. Multiple tags are combined by Stack Exchange as a tag intersection.", "default": [ "python" ], "items": { "type": "string" } }, "query": { "title": "Keyword query", "type": "string", "description": "Optional free-text search query. Leave empty to browse questions by site/tag/sort only.", "default": "" }, "sort": { "title": "Sort questions by", "enum": [ "activity", "votes", "creation", "hot", "week", "month", "relevance" ], "type": "string", "description": "Stack Exchange question sort mode. Query search supports activity, votes, creation, and relevance; unsupported query sorts fall back to activity.", "default": "activity" }, "maxItems": { "title": "Maximum questions", "minimum": 1, "maximum": 10000, "type": "integer", "description": "Maximum number of question rows to save across all selected sites.", "default": 20 }, "fromDate": { "title": "From date", "type": "string", "description": "Optional ISO date lower bound, for example 2026-01-01." }, "toDate": { "title": "To date", "type": "string", "description": "Optional ISO date upper bound, for example 2026-01-31." }, "minScore": { "title": "Minimum score", "type": "integer", "description": "Only save questions with at least this score. Use 0 to skip downvoted questions, or leave negative to include everything.", "default": -999999 }, "includeAnswers": { "title": "Include answer summaries", "type": "boolean", "description": "Fetch answer summaries for each saved question and include them in an answers array. This uses extra API requests.", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```