Stack Overflow & Stack Exchange Scraper — Q&A, Users & Tags

Extract complete Stack Overflow and Stack Exchange data — questions, answers, tags, full-text search results, and developer leads — across 360+ Q&A sites (Stack Overflow, Server Fault, Super User, Ask Ubuntu, Mathematics, Data Science, DevOps, Security, DBA, Code Review, Software Engineering, Unix & Linux, and more).

Every record ships the full body (HTML, Markdown and clean plain text), votes, views, owner, timestamps and a direct link. Turn high-reputation contributors into ready-to-contact developer leads with location, personal website, bio, socials and — optionally — emails and phone numbers crawled from their own site.

No login, no cookies, no browser — fast JSON extraction from the official Stack Exchange API with 99%+ reliability.

Why this Stack Exchange scraper?

Most scrapers in this space return only a thin slice (titles + scores) of a single site. This actor pulls the richest dataset in the category from the whole Stack Exchange network and adds a lead-gen layer no other scraper offers:

Data	Typical scrapers	This actor
Question full body (HTML + Markdown + text)	partial	✅
Votes, views, answer count, accepted answer	✅	✅
Answers with full body + accepted flag	❌	✅ opt-in
Comment threads on questions & answers	❌	✅ opt-in
Tags with counts, synonyms, wiki & top users	partial	✅
Full-text advanced search (any site)	partial	✅
Developer profiles: reputation, location, website, bio	❌	✅
Developer emails / phone (from their website)	❌	✅ opt-in
Developer socials (GitHub, X, LinkedIn, …) + lead score	❌	✅
360+ sites, not just Stack Overflow	❌	✅
Monitor mode — only new questions/answers	❌	✅

Use cases

• Developer lead generation — recruiters and dev-tool / DevRel teams find the highest-reputation contributors for a technology (e.g. top kubernetes, rust, or react answerers), score them by reputation and reach them via their website, socials and emails.
• Community & support monitoring — watch your product's tag (e.g. firebase, stripe-payments, next.js) and get alerted to every new question so you can answer first.
• Market & technology research — measure technology demand and trends with tag question counts, search volume and activity over time across the whole network.
• Content & SEO — mine the most-voted, most-viewed questions on a topic to plan documentation, tutorials and knowledge-base articles.
• Training data & RAG — export clean Q&A pairs (question + accepted answer, with content license) as LLM-ready JSON for fine-tuning or retrieval.
• Competitive intelligence — track questions and sentiment around competitor products and libraries.

How to use

1. Sign up for Apify — the free plan is enough to try this actor.
2. Open the Stack Overflow & Stack Exchange Scraper, pick a mode (search / questions / answers / users / tags), set a site and your query or tags (or paste Stack Exchange URLs), then click Start.
3. Watch results stream into the dataset table.
4. Export as JSON, CSV, Excel, XML, or RSS — or pull results programmatically via the Apify API.

Input

{
    "mode": "search",
    "site": "stackoverflow",
    "searchQuery": "fastapi async background task",
    "tags": ["python", "fastapi"],
    "sort": "votes",
    "minAnswers": 1,
    "includeAnswers": true,
    "includeBodyMarkdown": true,
    "maxItems": 100
}

Find the top developers for a technology and enrich them into leads:

{
    "mode": "users",
    "site": "stackoverflow",
    "sort": "reputation",
    "includeUserTopTags": true,
    "enrichUserContacts": true,
    "maxItems": 50
}

• mode — search (full-text), questions (browse by tag/date/votes), answers, users (developer leads), or tags.
• site — any Stack Exchange site's API parameter: stackoverflow, serverfault, superuser, askubuntu, math, datascience, devops, security, dba, codereview, softwareengineering, unix, apple, ux, … (360+ supported).
• searchQuery — free-text query for search mode.
• tags — filter questions by tag(s) (ALL must match), or look up tags directly in tags mode.
• questionIds / userIds / userName — fetch specific posts, specific developers, or search developers by name.
• startUrls — paste question, answer, user, tag or search URLs from any Stack Exchange site; the site is auto-detected.
• sort / order / acceptedOnly / minAnswers / fromDate / toDate — ordering and filters.
• includeAnswers (default false) — also emit each question's answers as separate records.
• includeComments (default false) — attach comment threads to questions/answers.
• includeUserTopTags (default false) — add each developer's top tags by answer score.
• includeTagDetails (default false) — add each tag's wiki excerpt + all-time top answerers & askers.
• enrichUserContacts (default false) — crawl each developer's website for emails, phone and socials, then assign a 0–100 lead score.
• monitorMode / monitorKey — remember IDs across runs and output only new items (for scheduled alerts).
• apiKey (optional) — a free Stack Apps key raises the daily quota from 300 to 10,000 requests. The actor works without one.

Output

One record per item, tagged with a type field. Question (type: "question"):

{
    "type": "question",
    "site": "stackoverflow",
    "questionId": 231767,
    "title": "What does the \"yield\" keyword do in Python?",
    "url": "https://stackoverflow.com/questions/231767/what-does-the-yield-keyword-do-in-python",
    "tags": ["python", "iterator", "generator", "yield"],
    "score": 13135,
    "viewCount": 3494429,
    "answerCount": 51,
    "isAnswered": true,
    "acceptedAnswerId": 231855,
    "creationDate": "2008-10-23T22:21:11.000Z",
    "lastActivityDate": "2026-02-14T09:12:00.000Z",
    "contentLicense": "CC BY-SA 4.0",
    "body": "I want to understand what the yield keyword does…",
    "bodyMarkdown": "I want to understand what the `yield` keyword does…",
    "ownerName": "Alex. S.",
    "owner": { "userId": 19302, "displayName": "Alex. S.", "reputation": 16542, "link": "https://stackoverflow.com/users/19302/alex-s" },
    "scrapedAt": "2026-06-17T21:00:00.000Z"
}

Developer lead (type: "user"):

{
    "type": "user",
    "site": "stackoverflow",
    "userId": 22656,
    "displayName": "Jon Skeet",
    "profileUrl": "https://stackoverflow.com/users/22656/jon-skeet",
    "reputation": 1528055,
    "location": "Reading, United Kingdom",
    "website": "http://csharpindepth.com",
    "aboutMe": "Author of C# in Depth…",
    "badgeCounts": { "gold": 895, "silver": 9318, "bronze": 9357 },
    "answerCount": 35802,
    "questionCount": 56,
    "topTags": [{ "tag": "c#", "answerScore": 250000, "answerCount": 22000 }],
    "emails": ["skeet@example.com"],
    "socialLinks": { "github": "https://github.com/jskeet", "twitter": "https://twitter.com/jonskeet" },
    "leadScore": 86,
    "scrapedAt": "2026-06-17T21:00:00.000Z"
}

Plus optional answer (type: "answer") records with full body, score and accepted flag, and tag (type: "tag") records with count, synonyms, wiki and top users.

Automate & schedule

Run this actor on autopilot and pull results into your own stack:

• Apify API — start runs, fetch datasets, and manage schedules over REST.
• apify-client for JavaScript and apify-client for Python — official SDKs.
• Schedules — run it hourly/daily to monitor new questions on a tag or new answers from a competitor; perfect for support queues and lead pipelines. Pair with monitor mode to emit only items you haven't seen.
• Webhooks — trigger downstream actions (CRM import, Slack alert, ticket creation) the moment a run finishes.

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'MY_APIFY_TOKEN' });

const run = await client.actor('scrapesage/stackexchange-scraper').call({
    mode: 'users',
    site: 'stackoverflow',
    sort: 'reputation',
    enrichUserContacts: true,
    maxItems: 50,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Got ${items.length} developer leads`);

Integrate with any app

Connect the dataset to 5,000+ apps — no code required:

• Make — multi-step automation scenarios.
• Zapier — push new developer leads straight into your CRM.
• Slack — get notified when a monitored tag gets a new question.
• Google Drive / Sheets — auto-export every run to a spreadsheet.
• Airbyte — pipe results into your data warehouse.
• GitHub — trigger runs from commits or releases.

Use with AI assistants (MCP)

The output is clean, LLM-ready JSON. You can call this actor from Claude, ChatGPT, or any agent framework through the Apify MCP server — ask your assistant to "find the top 20 Kubernetes contributors on Stack Overflow and list their websites" and let it run this scraper for you.

Agent-ready: autonomous payments (x402 & Skyfire)

This actor is agent-ready — AI agents can discover it, run it, and pay for it autonomously, with no Apify account and no human in the loop. It uses pay-per-event pricing and limited permissions, so it qualifies for Apify's agentic-payment standards:

• x402 — an open, HTTP-native payment protocol. Agents pay per run in USDC on the Base network directly through the Apify MCP server — no account, no API key.
• Skyfire — agent-to-service payments for fully autonomous AI-agent workflows.

Building an AI agent, MCP tool, or autonomous data pipeline? This scraper is ready to plug in and pay as it goes.

More scrapers from scrapesage

Build a complete developer-intelligence & dev-lead stack with the rest of the cluster:

• GitHub Scraper — repos, contributors, stars and developer leads from GitHub.
• Hugging Face Scraper — AI models, datasets and creator profiles.
• Product Hunt Scraper — launches, makers and product data.
• Chrome Web Store Scraper — extensions, ratings and developer contacts.
• Google Patents Scraper — patents, inventors and assignees.
• YouTube Scraper — channels, videos and creator leads.
• Linktree Scraper — creator bio links, emails and leads.
• Website Contact Scraper — emails, phones and socials from any domain list.

Tips

• Quota: keyless runs share Stack Exchange's 300-requests/day-per-IP limit; the actor rotates a fresh proxy IP per request to spread it. For large or frequent runs, add a free apiKey (10,000/day).
• Whole network: change site to scrape Server Fault, Super User, Ask Ubuntu, Math, Data Science, DevOps, Security, and 350+ more — the same modes work everywhere.
• Monitoring: turn on monitorMode with a unique monitorKey and a daily Schedule to get only newly posted questions on a tag.
• Cost control: answers and comments are opt-in; contact enrichment only runs for developers who actually list a website.

FAQ

Do I need a Stack Exchange API key? No. The actor works keyless out of the box. An optional free key only raises the daily request quota for heavy use.

Which sites are supported? All 360+ sites on the Stack Exchange network — pass the site's API parameter (e.g. stackoverflow, serverfault, math, datascience) or just paste a URL and let the actor detect it.

Where do developer emails come from? Never from Stack Exchange directly. With enrichUserContacts on, the actor visits the developer's own public website (the one they list on their profile) and extracts publicly listed contact details — the same thing a human visitor would see.

Can I export to Google Sheets, CSV, or Excel? Yes — one click in the dataset view, or automatically on every run via the Google Drive integration.

How do I monitor new questions automatically? Turn on monitor mode, then create a Schedule (e.g. daily) and optionally add a webhook or Zapier zap to push new items into Slack or your CRM.

Is scraping Stack Exchange legal? This actor uses the official public Stack Exchange API and collects publicly available data only. Content is licensed under Creative Commons (the contentLicense field tells you which version) — attribute it accordingly. You are responsible for complying with applicable laws (GDPR/CCPA for personal data) and the Stack Exchange terms.

A field is null — why? Some questions have no accepted answer, some developers list no website or location. Fields are null only when the data genuinely doesn't exist, not because the scraper skipped them.

Need help?

Open an issue on the actor's Issues tab, or visit the Apify help center. Feature requests are welcome — this actor is actively maintained.