# Technical Q\&A Search - Stack Exchange Question Miner (`darknezz/technical-qa-search`) Actor

Search Stack Overflow and 27 sister Stack Exchange sites by keyword or tag. Returns question title, plain-text body excerpt, tags, score, views, answer count, accepted-answer flag, author reputation and URL - optionally with the top answer. No API key.

- **URL**: https://apify.com/darknezz/technical-qa-search.md
- **Developed by:** [Oaida Adrian](https://apify.com/darknezz) (community)
- **Categories:** Developer tools, SEO tools, AI
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$3.00 / 1,000 question extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Technical Q\&A Search — Stack Exchange Question Miner

Search **Stack Overflow** and 27 sister Stack Exchange sites by keyword and/or tag, and get back one structured JSON row per question: title, plain-text body excerpt, tags, score, view count, answer count, accepted-answer flag, author reputation, URL — optionally with the highest-voted answer.

Built for developer-tooling research, content and SEO pipelines, support-deflection analysis, competitive "what breaks for users" studies, RAG corpus building, and any agent that needs real technical Q\&A rather than a marketing page.

### Why this Actor

Quora serves a **captcha interstitial** to datacenter IPs (probed directly, no proxy route fixes it) — that is why the previous Q\&A Actor was retired. The **Stack Exchange API** is the open alternative: it answers `200` with real question items datacenter-direct, **no API key, no Cloudflare**, and it covers the same buyer intent with far better structure (tags, scores, accepted answers, view counts, author reputation).

Feasibility was proven from Apify datacenter IPs before this Actor was written (dc-probe run `U0iaJmkfyXFZZh2bS`):

| Probe target | Result |
|---|---|
| `api.stackexchange.com/2.3/search/advanced` (+ `filter=withbody`) | 200, real questions |
| `api.stackexchange.com/2.3/questions/{ids}/answers` | 200, real answers |
| `api.stackexchange.com/2.3/sites` | 200, 100 sites listed |
| `softwarerecs.stackexchange.com` (search) | 200, real questions |
| `quora.com` (the retired source) | captcha interstitial |

### What it does

1. Runs every **query** against every selected **site** with `search/advanced`, newest state of the public API (`filter=withbody` returns question HTML in the same call — no second request).
2. Applies **client-side quality filters** the API cannot express: `minScore`, `minAnswers`, `acceptedOnly`.
3. Converts each question body from HTML to **readable plain text** and truncates it to a word-boundary excerpt.
4. Optionally fetches the **highest-voted answer** for every emitted question (one extra API call per query, not per question).
5. Reports per-query/per-site fetch-vs-emit counts, the remaining anonymous quota, and any API error in the run's `SUMMARY` key-value record.

### Input

```json
{
  "queries": ["python asyncio", "kubernetes ingress"],
  "sites": ["stackoverflow", "serverfault"],
  "tagged": [],
  "sort": "relevance",
  "minScore": 5,
  "minAnswers": 1,
  "acceptedOnly": false,
  "maxResultsPerQuery": 25,
  "includeBody": true,
  "bodyExcerptChars": 1000,
  "includeTopAnswer": false
}
```

| Option | Type | Description |
|--------|------|-------------|
| `queries` | string\[] | Keyword searches (default `["python asyncio"]`). |
| `sites` | string\[] | Stack Exchange sites by `api_site_parameter` (default `["stackoverflow"]`). Any valid parameter works, not just the curated 28. |
| `tagged` | string\[] | Tag filter, ANDed by the API, e.g. `["python","asyncio"]` (default `[]`). |
| `sort` | string | `relevance` (default), `votes`, `creation`, `activity`. |
| `minScore` | integer | Drop questions scoring below this (default 0). |
| `minAnswers` | integer | Drop questions with fewer answers (default 0). |
| `acceptedOnly` | boolean | Only questions with an accepted answer (default `false`). |
| `maxResultsPerQuery` | integer | Max questions per query per site, also the API page size (default 25, max 100). |
| `includeBody` | boolean | Include the plain-text question body (default `true`). |
| `bodyExcerptChars` | integer | Excerpt length, cut at a word boundary (default 1000). |
| `includeTopAnswer` | boolean | Fetch the highest-voted answer per emitted question (default `false`). |

### Output — one item per question

| Field | Description |
|-------|-------------|
| `questionId` | Stack Exchange question id |
| `site`, `siteName` | `api_site_parameter` and its display name |
| `title` | Question title |
| `url` | Canonical question URL |
| `bodyExcerpt` | Question text, HTML stripped, truncated at a word boundary |
| `bodyLength` | Full plain-text length before truncation |
| `tags` | Tag list |
| `score` | Upvotes minus downvotes |
| `viewCount` | Lifetime views |
| `answerCount` | Number of answers |
| `isAnswered` | API's "has an upvoted-or-accepted answer" flag |
| `hasAcceptedAnswer`, `acceptedAnswerId` | Whether an answer was accepted, and its id |
| `creationDate`, `lastActivityDate` | UTC ISO-8601 timestamps |
| `owner` | `{displayName, reputation, userId, profileUrl}` |
| `contentLicense` | Content licence reported by the API (attribution requirement) |
| `query`, `sort` | Which input produced this row, and under which ordering |
| `topAnswer` | `{answerId, score, isAccepted, bodyExcerpt, creationDate, ownerDisplayName, ownerReputation, url}` or `null` |
| `scrapedAt` | Extraction timestamp (UTC ISO-8601) |

### Worked example

Input:

```json
{ "queries": ["python asyncio"], "sites": ["stackoverflow"], "minScore": 5, "includeTopAnswer": true }
```

Output (trimmed):

```json
{
  "questionId": 27435284,
  "site": "stackoverflow",
  "siteName": "Stack Overflow",
  "title": "multiprocessing vs multithreading vs asyncio",
  "url": "https://stackoverflow.com/questions/27435284/multiprocessing-vs-multithreading-vs-asyncio",
  "bodyExcerpt": "A pod in my cluster is stuck on \"ContainerCreating\" …",
  "tags": ["python", "asyncio", "multiprocessing"],
  "score": 399,
  "viewCount": 1234567,
  "answerCount": 13,
  "isAnswered": true,
  "hasAcceptedAnswer": true,
  "acceptedAnswerId": 27436735,
  "creationDate": "2014-11-20T17:00:41+00:00",
  "owner": {"displayName": "jsbueno", "reputation": 12345, "userId": 7},
  "query": "python asyncio",
  "topAnswer": {"answerId": 48484593, "score": 120, "isAccepted": false}
}
```

1,000 questions cost about **$3.00** in event fees.

### Run it from your code or on a schedule

```python
from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("darknezz/technical-qa-search").call(
    run_input={"queries": ["python asyncio", "kubernetes ingress"],
               "sites": ["stackoverflow"], "minScore": 5},
    max_total_charge_usd=5.00,
)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["score"], item["title"], item["url"])
```

REST equivalent:

```bash
curl -X POST "https://api.apify.com/v2/acts/darknezz~technical-qa-search/runs?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries":["python asyncio","kubernetes ingress"],"sites":["stackoverflow"],"minScore":5}'
```

Schedule a weekly sweep of your product's keywords, or trigger a run per support ticket to surface prior art. `max_total_charge_usd` (Console → *Run options* → *Maximum cost per run*) caps spend: 10 queries × 1 site × 25 results ≈ 250 questions ≈ $0.75.

### Pricing

Pay per event: **$0.003 per question** (`question-extracted` primary event), charged once per emitted question — fetching the top answer adds no per-item charge, only one API call per query. 1,000 questions ≈ **$3.00**; 10,000 ≈ **$30.00**.

### FAQ

**Do I need a Stack Exchange API key?** No. Anonymous requests work; the shared quota is about 300 calls/day per IP, and each query costs one call (plus one if `includeTopAnswer` is on). The run's `SUMMARY` record reports `quotaRemaining` and warns below 30.

**What happens when the quota runs out?** The API replies with a `backoff`/throttle error; the Actor logs it, records it in `SUMMARY.errors`, and finishes the run with whatever it already collected instead of crashing.

**Why do I get fewer results than `maxResultsPerQuery`?** Filters (`minScore`, `minAnswers`, `acceptedOnly`) run after fetch, and the query/page is a sample of matches rather than the whole corpus.

**Can I search multiple pages?** Not in this version: one page per query per site (up to 100 results) keeps runs inside the store's quality window. Use several more specific queries instead — that is also how the API is meant to be used.

**Which sites can I pick?** Anything from `GET /2.3/sites` (about 180). The 28 in the dropdown are the ones that matter for technical work; `softwarerecs` is included for "what tool should I use" questions.

**Is the body text HTML?** No — it is converted to plain text (code blocks fenced) so it drops straight into an embedding pipeline, a spreadsheet or an LLM prompt.

**Can I reuse the content commercially?** Stack Exchange content is CC BY-SA (each row carries `contentLicense`); attribution is your responsibility. The Actor extracts metadata and excerpts — not full answer threads.

**Is this a Quora scraper?** No. Quora blocks datacenter traffic with an interactive captcha, so this Actor deliberately uses the open, keyless Stack Exchange API instead.

### Limitations

- **One page per query per site** (max 100 results), by design; fan out with more queries.
- **Excerpts, not full threads.** The question body is truncated to `bodyExcerptChars`; only the single top answer is fetched, and only when `includeTopAnswer` is on.
- **Deleted, closed or migrated questions** may appear with a link that redirects — the API reports what is live at request time.
- **Anonymous quota** is per-IP and shared with everything else egressing from the same Apify datacenter IP; the Actor surfaces it rather than hiding it.
- **No answer-level deep crawl.** If you need every answer with comments, that is a different, heavier Actor — ask for it.

# Actor input Schema

## `queries` (type: `array`):

Free-text searches, e.g. "python asyncio", "kubernetes ingress", "postgres vacuum bloat". Each query runs against every selected site.

## `sites` (type: `array`):

Stack Exchange sites to search (api\_site\_parameter). Defaults to Stack Overflow. Any valid api\_site\_parameter works, not just the curated list.

## `tagged` (type: `array`):

Optional tag filter (ANDed by the API), e.g. \["python", "asyncio"]. Empty means keyword search only.

## `sort` (type: `string`):

Order the API returns matches in. All results are sorted descending.

## `minScore` (type: `integer`):

Drop questions scoring below this (upvotes minus downvotes). Use 1+ to skip unanswered noise; 5+ for curated lists.

## `minAnswers` (type: `integer`):

Drop questions with fewer than this many answers. 1 keeps only questions someone actually answered.

## `acceptedOnly` (type: `boolean`):

Keep only questions whose asker accepted an answer — the highest-signal technical Q\&A pairs.

## `maxResultsPerQuery` (type: `integer`):

Maximum questions emitted per query per site (also the API page size).

## `includeBody` (type: `boolean`):

Include the question text converted from HTML to plain text.

## `bodyExcerptChars` (type: `integer`):

Maximum characters of the question body excerpt (cut at a word boundary).

## `includeTopAnswer` (type: `boolean`):

Fetch the highest-voted answer for every emitted question (one extra API call per query). Costs nothing extra per item, but uses the anonymous API quota.

## Actor input object example

```json
{
  "queries": [
    "python asyncio",
    "kubernetes ingress"
  ],
  "sites": [
    "stackoverflow"
  ],
  "tagged": [],
  "sort": "relevance",
  "minScore": 0,
  "minAnswers": 0,
  "acceptedOnly": false,
  "maxResultsPerQuery": 25,
  "includeBody": true,
  "bodyExcerptChars": 1000,
  "includeTopAnswer": false
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `title` (type: `string`):

No description

## `site` (type: `string`):

No description

## `url` (type: `string`):

No description

## `score` (type: `string`):

No description

## `tags` (type: `string`):

No description

## `answerCount` (type: `string`):

No description

## `hasAcceptedAnswer` (type: `string`):

No description

## `bodyExcerpt` (type: `string`):

No description

## `scrapedAt` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "python asyncio",
        "kubernetes ingress"
    ],
    "tagged": []
};

// Run the Actor and wait for it to finish
const run = await client.actor("darknezz/technical-qa-search").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": [
        "python asyncio",
        "kubernetes ingress",
    ],
    "tagged": [],
}

# Run the Actor and wait for it to finish
run = client.actor("darknezz/technical-qa-search").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "python asyncio",
    "kubernetes ingress"
  ],
  "tagged": []
}' |
apify call darknezz/technical-qa-search --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,darknezz/technical-qa-search"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/9XrjbHqNAEZaJuUjs/builds/qAsn0vlGDMuIkvN3Y/openapi.json