Stack Overflow Scraper - Questions, Tags & Scores avatar

Stack Overflow Scraper - Questions, Tags & Scores

Pricing

$2.00 / 1,000 question returneds

Go to Apify Store
Stack Overflow Scraper - Questions, Tags & Scores

Stack Overflow Scraper - Questions, Tags & Scores

Search Stack Overflow and Stack Exchange questions by keyword or tag. Each row has the score, answer count, views, author reputation and the body as plain text. Works on five sites. Two tags means AND: only questions with both come back. Answer bodies are not included. $2.00 per 1,000.

Pricing

$2.00 / 1,000 question returneds

Rating

0.0

(0)

Developer

Dami's Studio

Dami's Studio

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

2 days ago

Last modified

Share

Stack Overflow Scraper: questions, tags, scores and the question body

Search Stack Overflow and five sibling Stack Exchange sites by keywords, by tags, or by both. Each row is one question: the title, the score, how many answers it has, the view count, its tag list, who asked it and their reputation, and the whole question body as plain text.

Questions only. Answer bodies are not in the output, so if what you actually need is the accepted answer's code, this is the wrong purchase however cheap it is.

InputKeywords, tags, or both
OutputOne row per question
Ceiling1,000 questions per run
Account neededNone, and no Stack Exchange key
Price$2.00 per 1,000 questions, flat on every plan

๐Ÿ” What Stack Overflow Scraper does

It runs one search and pages through it until it has as many questions as you asked for.

query matches words in titles and bodies. tags filters, and it is an AND: put javascript,promise and you get only questions carrying both, not either. You can use tags on their own with no keywords, which is the cleanest way to pull the top questions in a tag.

sortBy decides the order, and it changes what a small run gives you. votes returns the long-standing classics. creation returns what was asked most recently. activity returns what people are still arguing about. relevance leans on the keyword match.

Six sites are available: Stack Overflow, Server Fault, Super User, Ask Ubuntu, MathOverflow and Software Engineering. One site per run.

๐Ÿ“ฅ What you give it

{
"query": "async await",
"tags": "javascript,promise",
"site": "stackoverflow",
"sortBy": "votes",
"maxItems": 50
}
FieldDefaultWhat it is
querynoneKeywords matched against titles and bodies. The Console shows async await as an example, but that is a prefill, so an API call has to send its own.
tagsnoneComma separated, and every tag must be present on the question. The Console prefills javascript, which again does not travel to an API call.
sitestackoverflowOne of stackoverflow, serverfault, superuser, askubuntu, mathoverflow, softwareengineering.
sortByvotesvotes, relevance, creation or activity. Always highest or newest first.
maxItems501 to 1,000. Paging is automatic, a hundred at a time.
notionConnectornoneOptional. Writes every delivered question into your Notion. Authorise a connector once under Settings, API and Integrations, MCP connectors, then pick it here.
notionParentIdnoneOptional. The Notion data source id to write into. Leave it empty and the pages are created privately in your workspace.
proxyConfigurationoffOptional network setting. Off is right for a normal run.

Send a query, tags, or both. Leaving both empty is not refused, and what comes back then is whatever the site decides to hand over, charged like any other rows. It is the one input mistake here that costs money.

๐Ÿ“ค What you get back

A real row from a recent run, with the body cut short here for length:

{
"ok": true,
"questionId": 37576685,
"title": "Using async/await with a forEach loop",
"url": "https://stackoverflow.com/questions/37576685/using-async-await-with-a-foreach-loop",
"score": 3353,
"answerCount": 35,
"viewCount": 2481345,
"isAnswered": true,
"tags": ["javascript", "node.js", "promise", "async-await", "ecmascript-2017"],
"ownerName": "Saad",
"ownerReputation": 55029,
"createdAt": "2016-06-01T18:55:58.000Z",
"body": "Are there any issues with using async/await in a forEach loop? I'm trying to loop through an array of files and await on the contents of each file. ..."
}
FieldWhat it is
bodyThe question, converted from HTML to plain text. Code blocks survive as text, indentation mostly does not.
isAnsweredWhether an answer has been accepted. Not the same as answerCount being above zero.
ownerName, ownerReputationBoth null when the account was deleted or the post is anonymous.
scoreNet votes, which can be negative.
titlePassed through as the API sends it, so an HTML entity like " can survive into this field. Only body is decoded.
tagsThe question's full tag list, which is usually wider than the tags you filtered on.

๐Ÿงพ Reading the output

Questions carry ok: true. Anything with ok: false carries an errorCode and is not charged.

CodeWhat it means
NO_RESULTSThe search ran and matched nothing. Usually a tag that does not exist, or two tags no single question carries.
RATE_LIMITEDStack Exchange asked the run to stop for the day. Whatever was already collected is in the dataset and complete as far as it goes.
NETWORKStack Exchange was unreachable or answered with something unusable. Re-run it.

A RATE_LIMITED row after some questions is not a failed run. It means the run delivered what it could and stopped rather than spinning.

The Console's Overview table hides tags, isAnswered and body. Download the dataset as JSON, CSV or Excel, or switch the table to all fields.

โ–ถ๏ธ How to run it

  1. Open Stack Overflow Scraper and click Try for free.
  2. Type into Search query, or fill Tags instead, or both.
  3. Pick a Stack Exchange site if you want one of the other five.
  4. Choose a Sort by order and set Max questions, then click Start.
  5. Download the dataset as JSON, CSV, Excel or XML.

๐Ÿ’ฐ How much does it cost?

$2.00 per 1,000 questions. Flat on every Apify plan, no volume tiers.

You pay per question row. Duplicates removed inside a run and every diagnostic row are not charged, and a search that matches nothing does not bill for results.

๐Ÿ’ก What people use it for

  • Finding the questions a product's documentation should have answered, by searching its name and reading what people are stuck on.
  • Comparing tag volume between two libraries before committing to one.
  • Building an evaluation set for a coding model out of plain-text question bodies.
  • Watching a tag weekly with sortBy on creation to see what problems are new.

๐Ÿšง What it does not do

  • No answers, no comments, no accepted-answer text. answerCount and isAnswered tell you they exist, and the row stops there.
  • One site per run. Searching Stack Overflow and Ask Ubuntu together means two runs.
  • No date range input. Sort by creation and filter on createdAt afterwards.
  • Only six of the network's sites, the ones in the list above.
  • Tags are AND, never OR. Three tags usually returns far less than you expect.
  • There is a daily ceiling on how much anyone can pull without a key. When it is reached the run stops and says so instead of failing.
  • Rows are a snapshot. Scores, views and answer counts move.

๐Ÿงญ Which developer data scraper do you need?

If you wantUse
Stack Exchange questions, tags and scoresThis one
Repositories, stars and issues from GitHubGitHub Scraper
npm and PyPI package metadata and download countsPackage Registry Scraper
Developer articles and their tagsDEV Community Scraper

โ“ Questions people ask

Do I need a Stack Exchange API key? No.

Can I search tags without keywords? Yes, and it is the better way to pull a tag's top questions. Leave the query empty and fill tags.

Why did two tags return almost nothing? Because both have to be on the same question. Drop one.

Can I get the answers? Not from this one. Take url from the row and open the question.

Why does my title contain "? The title is passed through as the API sends it. Decode it on your side, or use body, which is already plain text.

Is this legal? Stack Exchange publishes this through a public API meant to be read, and the content is under a Creative Commons licence that asks for attribution. Apify's write-up on scraping and the law is a good starting point, and we are not lawyers.

๐Ÿ†˜ If something breaks

Open the Issues tab on the actor page. Send the query, the tags and the run id. The errorCode on the diagnostic row usually names the problem by itself.