Stack Overflow Scraper

Pull questions and answers from Stack Overflow and 170+ other Stack Exchange sites by tag or search term. You get the question text, the accepted answer, the highest-scoring answer, the score, the view count, and whether the question was closed and why.

Filter by tag, search free text, pick any network site, sort by votes or activity.

The site is closed, the API is open

Worth stating plainly: stackoverflow.com returns 403 to a plain HTTP client. across two network exits, the website refused every request while api.stackexchange.com answered 200 on the same run.

So this Actor is not taking a shortcut by using the API. The API is the only working door, and it is a documented, key-free one that returns better structured data than the page ever did.

The quota, stated up front

Stack Exchange allows 300 requests per day per IP without a key, and 10,000 with a free key. One request returns up to 100 questions, so the unauthenticated allowance is roughly 30,000 questions a day. Enough for most jobs, not enough for a bulk harvest.

A key takes about a minute to create at stackapps.com/apps/oauth/register, needs no approval, and goes in the api_key field.

This Actor treats the quota as a real budget rather than something to route around:

• quota_remaining is read from every response, not assumed, and reported in the run summary.
• When fewer than three requests remain, the run stops and says so instead of draining the last of the allowance.
• Throttling that arrives inside a 200 body, which is how Stack Exchange usually signals it, is detected and reported as quota_exhausted rather than parsed as an empty page.

Rows collected before a quota stop are complete and are returned.

What comes back

Per question:

• Identity: question_id, site, title, url, tags
• Signal: score, view_count, answer_count, comment_count
• Resolution: is_answered, has_accepted_answer, accepted_answer_id
• Bodies: question_body, accepted_answer_body, top_answer_body, top_answer_score — all HTML stripped and entity decoded
• Lifecycle: created_at, last_activity_at, closed_at, closed_reason, is_closed
• Asker: asker, asker_reputation, asker_url

Two derived fields worth knowing about:

views_per_score is views divided by net score. A question with 400,000 views and a score of 12 is a problem thousands of people hit that the site never rewarded — which is exactly what documentation and content research is hunting for. Sort descending on it.

unanswered_but_popular flags questions with over 1,000 views and no accepted-standard answer. On a tag you own, that list is a content backlog.

Accepted is not the same as top

accepted_answer_body is the answer the asker chose, matched by id. top_answer_body is the highest scoring answer. They are frequently different, and the gap is informative: a low-scoring accepted answer next to a high-scoring rival usually means the accepted one solved the asker's specific case while the community preferred a better general answer.

Both are returned so you never have to guess which one you are reading.

Input reference

Field	Type	Default
tagged	list of tags, all must match	["python", "asyncio"]
search	free text, uses advanced search	—
site	any network site	stackoverflow
sort	votes, activity, creation, hot, week, month	votes
order	desc, asc	desc
min_score	integer	—
api_key	free key from stackapps.com	—
limit	1-2000	100
retries	1-6	3

Give at least a tag or a search term. An empty input is rejected rather than walking the whole site, because that would burn the daily quota on nothing.

Other network sites are just a parameter: serverfault, superuser, askubuntu, math, unix, dba, security, datascience, and the rest.

Typical uses

• Content and docs research. Take the tags for your product, sort by views_per_score, and you have a ranked list of things people struggle with and nobody answered well.
• Support triage. Watch unanswered_but_popular on your own tags.
• Competitive and ecosystem research. Compare question volume and answer rates across tags to see which tools people are actually adopting.
• Training and evaluation data. Question plus accepted answer, with score and view count as quality signals.

Notes

is_answered is Stack Exchange's own flag and means an answer scored above the site's threshold, which is not the same as the asker accepting one. Both are returned separately for that reason.

Answer bodies are truncated at 4,000 characters. HTML is stripped, so code blocks lose their formatting but keep their text.

Please respect the quota rather than working around it. This API is free, it does not require an account, and it stays that way because people use it as intended.