Stack Overflow Question Scraper

Search Stack Overflow by keywords or tags and get structured question data — title, full question body, score, views, answer counts, accepted-answer status, tags, and author — as JSON, CSV, or Excel.

Built on the official Stack Exchange API: no fragile HTML scraping, no bot walls, no breakage when the site redesigns.

Pay per result — you're only charged for questions actually delivered, never for failed runs or empty searches.

---

What can I use it for?

• AI & LLM datasets — every record includes the full question body (Markdown), ready for fine-tuning, RAG, or trend analysis.
• Developer-tool market research — what are people struggling with in your ecosystem? Sort by votes or recency.
• Content & SEO research — the highest-voted unanswered questions in a tag are a content-idea goldmine.
• Competitive intelligence — track question volume and sentiment around frameworks, APIs, and products.
• Lead generation — questions about problems your product solves, with links to engage.

---

Input

Field	Required	Description
Search query	◐	Free-text search (e.g. react hooks useState). At least one of query or tags is required.
Tags	◐	Only questions having all of these tags (e.g. python, asyncio).
Sort by / Order	–	votes (default), activity, creation, or relevance; descending or ascending.
Max questions	–	1–10,000 (default 100). Each question = one billable result.
Stack Exchange API key	–	Optional, free (stackapps.com). Raises the daily request quota from 300 (shared per IP) to 10,000 — recommended for large or recurring jobs.
Proxy configuration	–	Off by default — the official API is scraper-friendly. Enable only if a shared IP's anonymous quota is exhausted.

Output

One dataset record per question. Example (shortened):

{
  "questionId": 27435284,
  "title": "multiprocessing vs multithreading vs asyncio",
  "url": "https://stackoverflow.com/questions/27435284/...",
  "tags": ["python", "multithreading", "asyncio"],
  "score": 250,
  "viewCount": 150000,
  "answerCount": 8,
  "isAnswered": true,
  "hasAcceptedAnswer": true,
  "acceptedAnswerId": 27436839,
  "body": "I found that in Python 3.4 there are several libraries for ...",
  "createdAt": "2014-12-11T12:58:36Z",
  "lastActivityAt": "2023-11-14T22:13:20Z",
  "owner": {
    "displayName": "user3654650",
    "reputation": 5000,
    "userId": 3654650,
    "profileUrl": "https://stackoverflow.com/users/3654650/..."
  }
}

Field	Description
body	Full question text as Markdown — most competing scrapers return only titles
score / viewCount / answerCount	Community signals for ranking and filtering
hasAcceptedAnswer / acceptedAnswerId	Filter for solved (or unsolved!) questions
owner	Author display name, reputation, profile link (null for deleted users)

---

Why this scraper is reliable

• Official API, not HTML — Stack Overflow redesigns can't break it, and there are no bot walls to fight. Results are exactly what the site's own search returns.
• Quota-aware — honors the API's backoff signals (ignoring them gets IPs banned), reports remaining quota in the run summary, and stops loudly — never silently short — if quota runs out.
• Honest run statuses — a run only says Succeeded when the search completed. API failures mid-run are labeled PARTIAL: with the reason; a search that matches zero questions says so explicitly. Never a silent empty dataset.
• Batch-safe — questions are saved incrementally page by page and runs are checkpointed: an interruption never loses finished work, and a resurrected run resumes without re-charging.
• Deduplicated — question IDs are tracked across pages and restarts; you never pay for the same question twice.

Tips & limits

• Large jobs: bring a (free) API key. Anonymous API quota is 300 requests/day shared per IP address; a key raises it to 10,000/day for your runs alone. One request fetches 100 questions, so a key covers up to 1M questions/day.
• Tags are ANDed: ["python", "pandas"] returns questions tagged with both.
• sort: creation, order: asc walks a tag's history from the very first question — useful for building complete archives.

FAQ

Does it include answers? Records include answer counts and the accepted answer's ID. Full answer text is on the roadmap — tell us in the Issues tab if you need it.

Can it scrape other Stack Exchange sites (Super User, Server Fault…)? This actor targets Stack Overflow. If you need other sites, open an issue — the engine supports them.

Why did my run stop early? Check the SUMMARY record: it reports the exact reason (usually anonymous API quota exhausted — add a free API key) plus how many questions were delivered. You're only charged for what was delivered.

Is this legal? The actor uses the official, documented Stack Exchange API and only accesses public data. Content is licensed CC BY-SA by Stack Exchange; attribution requirements apply to republication.

---

Disclaimer

This is an unofficial tool. It is not affiliated with, endorsed by, sponsored by, or in any way officially connected to Stack Overflow or Stack Exchange, Inc. "Stack Overflow" and related names, marks, and logos are trademarks of their respective owners and are used here for descriptive purposes only.

This actor accesses only publicly available data via the official Stack Exchange API. You are responsible for ensuring your use complies with applicable laws, the Stack Exchange API terms, and content licensing (CC BY-SA). Use it ethically and at your own risk.