Steam Community Discussions Scraper

Extract public Steam Community discussion topics, original posts, and replies into structured records for support research, community monitoring, sentiment analysis, and RAG workflows.

Use a Steam app ID for broad game-forum coverage or provide exact category and thread URLs for focused collection. Export the resulting dataset as JSON, CSV, Excel, XML, or RSS, or send it directly to your automation stack.

What does Steam Community Discussions Scraper do?

Steam Community Discussions Scraper turns public, server-rendered Steam forum pages into analysis-ready topic and post records.

It can:

• discover discussions for one or more Steam games
• collect topic titles, IDs, URLs, reply counts, awards, and status flags
• expand discovered threads into original posts and ordered replies
• preserve displayed author names and public profile URLs
• return Steam timestamps in Unix and ISO 8601 formats
• scrape a known category or thread directly
• run in topic-only mode for inexpensive recurring monitoring

The Actor accesses anonymous public pages only. It does not sign in, post, vote, subscribe, or retrieve private content.

Who is it for?

🎮 Game studios and publishers can identify recurring bugs, feature requests, balance complaints, and player questions.

🎧 Community and support teams can route new conversations into triage queues and measure which topics attract the most replies.

📊 Market researchers can compare public player conversations across games and follow shifts around launches or updates.

🤖 Data and AI teams can build sentiment classifiers, searchable archives, RAG collections, and alerting pipelines from structured forum text.

🛡️ Trust and safety analysts can review publicly visible conversation patterns while retaining canonical evidence URLs.

Why use this Actor?

Steam forum pages are designed for people, not repeatable data workflows. This Actor handles discovery, pagination, parsing, normalization, deduplication, and dataset storage for you.

Benefits include:

• Auditable records: every topic and post links to its public Steam thread.
• Stable identifiers: Steam topic and post IDs support downstream deduplication.
• Predictable scope: separate topic and post limits keep output volume controlled.
• Flexible depth: collect summaries frequently, then expand only conversations that matter.
• Automation-ready output: consume results through exports, the Apify API, webhooks, schedules, or integrations.

What data can I extract?

Field	Description
recordType	topic summary or post content record
appId	Steam application ID
categoryId	Discussion category ID when available
topicId	Stable Steam forum topic ID
postId	Stable original-post or reply ID
title	Discussion title
threadUrl	Canonical public thread URL
authorName	Displayed Steam Community author name
authorUrl	Public author profile URL when available
body	Normalized post text
timestamp	ISO 8601 publication timestamp
timestampUnix	Steam Unix timestamp
replyOrder	Position of a post in its conversation
replyCount	Number of replies shown for a topic
awardCount	Number of displayed community awards
isPinned	Whether the topic is pinned
isLocked	Whether the topic is locked
scrapedAt	ISO 8601 collection time

Optional values that Steam does not expose are returned as null; the Actor does not invent missing data.

How to scrape Steam Community discussions

1. Open Steam Community Discussions Scraper in the Apify Store.
2. Click Try for free to open the Actor input page.
3. Enter one or more Steam app IDs, such as 730 for Counter-Strike 2, or add public discussion/thread URLs.
4. Choose whether to enable Extract thread posts. Disable it when topic summaries are sufficient.
5. Set Maximum topics and Maximum posts to control the dataset size and cost.
6. Leave proxy use disabled for a small first run; enable Apify Proxy only if your larger workload needs it.
7. Click Start and wait for the run to finish.
8. Open the Dataset tab to review, filter, export, or integrate the records.

For recurring monitoring, save the input as an Apify task and attach a schedule after confirming the first result set.

Input

The Actor accepts these fields:

Input	Type	Purpose
appIds	string array	Numeric Steam app IDs to monitor
startUrls	request list	Public category indexes or individual thread URLs
expandThreads	boolean	Extract original posts and replies when true
maxTopics	integer	Maximum topic summaries discovered across category pages
maxPosts	integer	Maximum post records extracted across threads
proxyConfiguration	object	Optional Apify Proxy configuration

You may combine app IDs and URLs in one run. Limits apply across the complete run rather than independently to every source.

Input examples

Low-cost topic discovery

{
  "appIds": ["730"],
  "expandThreads": false,
  "maxTopics": 25,
  "maxPosts": 1
}

Collect topics and replies

{
  "appIds": ["570"],
  "expandThreads": true,
  "maxTopics": 20,
  "maxPosts": 100
}

Extract a known public thread

{
  "startUrls": [
    { "url": "https://steamcommunity.com/app/730/discussions/0/" }
  ],
  "expandThreads": true,
  "maxTopics": 10,
  "maxPosts": 50
}

Output example

{
  "recordType": "post",
  "appId": "730",
  "categoryId": "0",
  "topicId": "example-topic-id",
  "postId": "example-post-id",
  "title": "Example discussion title",
  "threadUrl": "https://steamcommunity.com/app/730/discussions/0/example-topic-id/",
  "authorName": "Community member",
  "authorUrl": "https://steamcommunity.com/profiles/example",
  "body": "Public discussion text appears here.",
  "timestamp": "2026-07-12T12:00:00.000Z",
  "timestampUnix": 1783857600,
  "replyOrder": 0,
  "replyCount": null,
  "awardCount": null,
  "isPinned": false,
  "isLocked": false,
  "scrapedAt": "2026-07-12T12:05:00.000Z"
}

Topic rows and post rows share one dataset and are distinguished by recordType.

How much does it cost to scrape Steam Community discussions?

Pricing consists of a $0.005 run start plus a charge for each topic or post saved to the dataset. Per-item prices decrease with the Apify subscription tier:

Tier	Price per saved topic or post
FREE	$0.000035889
BRONZE	$0.000031208
SILVER	$0.000024342
GOLD	$0.000018725
PLATINUM	$0.000012483
DIAMOND	$0.000010000

Concrete examples, excluding any platform compute usage included by your Apify plan:

• 100 records on FREE: $0.005 start + $0.0035889 = about $0.0086.
• 1,000 records on BRONZE: $0.005 start + $0.031208 = about $0.0362.
• 10,000 records on DIAMOND: $0.005 start + $0.10 = about $0.105.

Only records successfully saved are item events. Your exact total depends on output volume and your active Apify tier. Use topic-only mode and conservative limits when validating a new workflow.

Topic-only monitoring workflow

Set expandThreads to false when you need discovery rather than full conversation text.

1. Schedule a modest topic-only task.
2. Store topicId as the stable deduplication key.
3. Compare replyCount, awardCount, isPinned, and isLocked between runs.
4. Send newly discovered or fast-growing threadUrl values to a second deep-extraction task.
5. Alert the relevant support or community team.

This two-stage design reduces unnecessary requests and output events.

Full-thread research workflow

Use exact thread URLs or enable thread expansion when post text and reply order matter.

• Preserve recordType + topicId + postId as a compound key.
• Sort post rows by replyOrder before conversation analysis.
• Keep threadUrl with derived labels so analysts can audit the original source.
• Use timestamp for BI tools and timestampUnix for numeric comparisons.
• Track scrapedAt separately from publication time.

Integrations

Connect the Actor to specific downstream workflows:

• Google Sheets: append newly discovered topics for community managers to review and assign.
• Slack or Microsoft Teams: use an Apify webhook plus Make or Zapier to alert on high-reply or newly pinned discussions.
• Airtable: maintain a support-research backlog keyed by topicId and update reply counts on each scheduled run.
• BigQuery, Snowflake, or PostgreSQL: load normalized records for trend dashboards and cross-game analysis.
• Amazon S3 or Google Cloud Storage: archive periodic exports for reproducible research.
• LLM and RAG pipelines: chunk body, retain threadUrl as citation metadata, and filter retrieval by appId.
• Sentiment services: classify each post, then aggregate labels by game, topic, or release window.

Apify schedules provide recurring runs; webhooks can start the integration only after a run succeeds.

API usage

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/steam-community-discussions-scraper').call({
  appIds: ['730'],
  maxTopics: 20,
  maxPosts: 100,
  expandThreads: true,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();

Python

from apify_client import ApifyClient

client = ApifyClient('YOUR_APIFY_TOKEN')
run = client.actor('automation-lab/steam-community-discussions-scraper').call(
    run_input={
        'appIds': ['730'],
        'maxTopics': 20,
        'maxPosts': 100,
        'expandThreads': True,
    }
)
items = client.dataset(run['defaultDatasetId']).list_items().items

cURL

curl -X POST \
  'https://api.apify.com/v2/acts/automation-lab~steam-community-discussions-scraper/runs?token=YOUR_APIFY_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{"appIds":["730"],"maxTopics":20,"maxPosts":100,"expandThreads":true}'

MCP for Claude

Add the hosted Apify MCP server in Claude Code:

claude mcp add --transport http apify \
  'https://mcp.apify.com/?tools=automation-lab/steam-community-discussions-scraper'

Claude Desktop configuration:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com/?tools=automation-lab/steam-community-discussions-scraper"
    }
  }
}

Example prompts:

• “Find newly active Counter-Strike 2 bug discussions and summarize the recurring issues.”
• “Extract this public Steam thread and return the replies in chronological order.”
• “Compare topic engagement for these Steam app IDs and cite the source threads.”

Data quality and deduplication

Steam IDs are stored as strings to avoid numeric precision loss. Topic records use topicId; post records contain both topicId and postId. For repeat runs, deduplicate with:

recordType + topicId + postId

For topic rows without a post ID, use recordType + topicId. Text is whitespace-normalized while preserving readable content. Canonical URLs make every record traceable to its public source.

Performance tips

• Start with one app ID and the low prefilled limits.
• Disable thread expansion for frequent discovery jobs.
• Use direct thread URLs when the conversations are already known.
• Increase maxTopics and maxPosts gradually after reviewing output.
• Enable Apify Proxy only when sustained workloads require it.
• Schedule larger crawls less frequently and deduplicate downstream.
• Keep output limits aligned with what your integration can process.

Limitations

• Deleted, moderated, private, age-gated, or login-only content is unavailable.
• Steam may change page markup or throttle sustained traffic.
• Display names can change and should not be treated as verified identities.
• Some optional metadata is absent from particular categories or posts.
• Results reflect what the public page exposes at scrape time and are not a complete historical archive.
• One dataset contains both topic and post entity types; use recordType when filtering.

Troubleshooting

A public URL returns no records. What should I check?

Confirm it is a steamcommunity.com/app/.../discussions/... category or thread URL, opens without login, and has not been removed. Then inspect the run log for a response or parsing error.

My scheduled crawl receives throttling responses. What should I do?

Reduce limits or frequency first. For sustained collection, enable Apify Proxy and keep the workload conservative rather than retrying aggressively.

Why does a topic have no reply rows?

The topic may contain only its original post, thread expansion may be disabled, the global post limit may have been reached, or moderated content may expose less text than the index indicates.

Why are some fields null?

Steam does not display every field on every page. The Actor returns null rather than fabricating unavailable values.

Legality and responsible use

The Actor reads public pages without logging in. Use it only for a lawful purpose, respect Steam's terms and reasonable request rates, and process usernames and post text according to applicable privacy and data-protection rules.

Do not use forum data to harass people, infer sensitive traits, or make unsupported consequential decisions. Retain source links and apply human review where context matters.

Related scrapers

These Automation Labs actors support adjacent Steam research workflows:

• Steam Game Reviews Scraper — collect public game reviews and ratings for sentiment analysis.
• Steam Workshop Scraper — extract Workshop mods and community-created item metadata.
• Steam Curator Reviews Scraper — gather curator recommendations and review text.
• Steam Scraper — collect broader Steam store and game metadata.

Choose this Discussions Scraper when the required dataset is forum topics, original posts, and replies rather than store, review, curator, or Workshop records.

FAQ

Does it require a Steam account?

No. It reads only pages available anonymously.

Can I monitor several games in one run?

Yes. Add multiple strings to appIds; the topic and post limits apply across the whole run.

Can I collect only topic summaries?

Yes. Set expandThreads to false.

Can I scrape one exact conversation?

Yes. Add its public thread URL to startUrls and enable thread expansion.

Are timestamps normalized?

Yes. Available Steam timestamps are returned as both Unix values and ISO 8601 strings.

Can I export the output?

Yes. Apify datasets support JSON, CSV, Excel, XML, RSS, and API access.

Support

For reproducible help, include the public app ID or discussion URL, sanitized input, run ID, and a description of the unexpected output. Never share Steam credentials; this Actor does not need them.