Steam Workshop Scraper - Extract Mods, Stats & Community Data

Scrape Steam Workshop data at scale. Extract mod metadata, download statistics, descriptions, dependencies, changelogs, and community discussions from any Steam game's Workshop. Perfect for game developers, modders, researchers, and automation workflows.

🎯 What You Can Scrape

Workshop Items Data

• Mod Information: Titles, authors, Steam IDs, creation & update dates
• Engagement Metrics: Subscriber counts, favorites, views, trending scores
• Content: Full BBCode descriptions, plain text versions, multiple languages
• Media Assets: Preview images, screenshots, video thumbnails
• Relationships: Required mods, dependencies, mod collections
• Tags & Categories: User-defined tags for filtering and discovery

Community Content (Optional)

• Version History: Change notes, update logs, patch notes
• Discussions: Thread titles, posts, timestamps, authors
• Trend Analysis: Track popularity changes over time

💡 Use Cases

For Game Developers

• Monitor popular mods in your game's Workshop
• Identify trending content and community preferences
• Track mod compatibility and dependency chains
• Analyze player engagement with Workshop content

For Modders & Creators

• Research popular mod features and descriptions
• Find complementary mods for dependencies
• Track competitor mods and market trends
• Analyze successful mod marketing strategies

For Researchers & Analysts

• Study gaming community behavior
• Analyze user-generated content trends
• Track mod ecosystem evolution
• Sentiment analysis from discussions

For Automation & Integration

• Auto-update mod databases
• Trigger notifications on new releases
• Generate mod recommendation systems
• Feed data into Discord bots or websites

🚀 Quick Start (3 Minutes)

Step 1: Get Your Steam API Key (Free)

1. Visit steamcommunity.com/dev/apikey
2. Sign in with your Steam account
3. Enter any domain name (e.g., apify.com)
4. Copy your API key

Step 2: Configure & Run

{
  "steamApiKey": "YOUR_KEY_HERE",
  "appId": [550],
  "maxItems": 10
}

That's it! The Actor will automatically:

• Discover trending Workshop items for Left 4 Dead 2 (App ID 550)
• Fetch complete metadata via Steam API
• Enrich with descriptions and media from pages
• Save structured data to your dataset

Step 3: View Results

Access your dataset in multiple views:

• Overview: All fields with clickable Steam URLs
• Popular Items: Sorted by subscriber count
• Dependencies: Mod relationships and requirements

📊 Popular Games Supported

Game	App ID	Workshop Items
Left 4 Dead 2	550	187,000+
RimWorld	294100	45,000+
Cities: Skylines	255710	250,000+
Counter-Strike 2	730	1,000,000+
Garry's Mod	4000	500,000+
Skyrim	72850	28,000+
Ark: Survival	346110	85,000+

Find your game's App ID: Visit store.steampowered.com/app/APPID/ in the game's URL

⚙️ Configuration Options

Basic Settings

Steam API Key (required) Your Steam Web API authentication key

App IDs (required) One or more game Workshop IDs to scrape. Examples:

• [550] - Left 4 Dead 2
• [294100, 255710] - RimWorld + Cities: Skylines

Max Items (default: 1000) Maximum items to scrape per game. Range: 1-50,000

Filtering Options

Search Text Find mods containing specific keywords in title/description

Tags Filter by Workshop tags: ["weapons", "QoL", "graphics"]

Updated Since Only scrape items updated after date: "2025-01-01"

Sort Order

• trend - Currently trending (default)
• top - Most subscribed all-time
• new - Recently published
• updated - Recently updated

Data Enrichment

Include Descriptions (default: true) Scrape full BBCode and text from pages (API provides stripped text only)

Include Media (default: true) Extract preview images, screenshots, video URLs

Include Dependencies (default: true) Find required mods and mod relationships

Include Change Notes (default: false) Scrape version history and changelogs (slower)

Include Discussions (default: false) Extract community threads and posts (significantly slower)

Advanced Options

Proxy Configuration Use Apify's residential proxies to avoid blocking (recommended)

Request Interval Delay between page requests: 1000-10000ms (default: 1200)

Debug Mode Enable verbose logging for troubleshooting

📦 Output Format

Each Workshop item is saved as a structured JSON record:

{
  "appId": 550,
  "publishedFileId": "3578582211",
  "url": "https://steamcommunity.com/sharedfiles/filedetails/?id=3578582211",
  "title": "TW-201-S Morita I (Combat Rifle)",
  "author": {
    "name": "Adorabirb",
    "steamId": "76561198073176857",
    "profileUrl": "https://steamcommunity.com/profiles/76561198073176857"
  },
  "stats": {
    "subscriptions": 69,
    "favorites": 47,
    "views": 430,
    "created": 1759370331,
    "updated": 1759370331
  },
  "tags": ["Weapons", "Rifle"],
  "dependencies": [],
  "media": {
    "preview": "https://...",
    "additionalPreviews": ["https://...", "https://..."]
  },
  "description": {
    "bbcode": "[h1]Features[/h1][list][*]Custom animations...",
    "text": "Features\n- Custom animations...",
    "language": "en"
  },
  "changeNotes": [],
  "discussions": [],
  "hash": "9bf0c7e658befa85f36232f43e93285f55e28665",
  "scrapedAt": "2025-10-02T15:36:18.280Z"
}

Key Fields Explained

• publishedFileId: Unique Workshop item ID (use as primary key)
• hash: Content hash for change detection (changes when item updates)
• stats.updated: Unix timestamp of last modification
• dependencies: Array of required mods
• media.additionalPreviews: All screenshots/videos
• description.bbcode: Raw BBCode for rendering
• scrapedAt: ISO timestamp of when data was collected

💰 Pricing

Pay-Per-Event Model: You only pay for successfully scraped records.

• Event: record-ingested (one per Workshop item)
• Typical Cost: $0.001 - $0.005 per record
• No charges for: Failed requests, API calls, or retries
• Predictable: 1000 items = $1-5 depending on enrichment

Example Costs:

• 100 items with basic data: ~$0.20
• 1,000 items with descriptions: ~$2.00
• 10,000 items with all features: ~$30-50

Special Offer: 0% commission through March 31, 2025! 🎉

🔗 Integration Examples

n8n Workflow

// 1. Trigger: Schedule (daily at 8am)
// 2. Apify: Run Actor
{
  "actorId": "YOUR_ACTOR_ID",
  "input": {
    "steamApiKey": "{{$credentials.steamApiKey}}",
    "appId": [550],
    "maxItems": 100,
    "filters": { "updatedSince": "{{$now.minus({days: 1}).toISO()}}" }
  }
}
// 3. HTTP Request: Get dataset items
// 4. Google Sheets: Upsert by publishedFileId
// 5. Slack: Notify new mods > 100 subscribers

Zapier Integration

1. Trigger: New run finished (Apify webhook)
2. Action: Get dataset items (HTTP)
3. Filter: Only items with stats.subscriptions > 1000
4. Action: Add to Airtable
5. Action: Post to Discord channel

API Integration

# Start a run
curl -X POST https://api.apify.com/v2/acts/YOUR_ACTOR_ID/runs \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"steamApiKey":"KEY","appId":[550],"maxItems":50}'

# Get results
curl https://api.apify.com/v2/datasets/DATASET_ID/items

⚡ Performance & Limits

Speed

• API Discovery: ~1 second per 100 items
• Basic Scraping: ~9 seconds per item average
• With Discussions: ~15-20 seconds per item
• Throughput: 15-20 items/minute

Limits

• Steam API: 100,000 calls/day per key
• Batch Size: 100 items per GetDetails call
• Concurrent Pages: 3 browser contexts
• Max Items: 50,000 per run (soft limit)

Optimization Tips

• Use itemIds to scrape specific items only
• Disable discussions unless needed (5x faster)
• Filter by updatedSince for incremental updates
• Run on schedule instead of scraping all items repeatedly

🛡️ Anti-Blocking & Reliability

Built-in Protection

• ✅ Residential Proxies: Apify's rotating proxy network
• ✅ Rate Limiting: 1.2s delay between requests (configurable)
• ✅ Retry Logic: 3 attempts with exponential backoff
• ✅ Circuit Breaker: Automatic pause on repeated failures
• ✅ Stealth Mode: Browser fingerprint masking
• ✅ User-Agent Rotation: Randomized headers

Compliance

• Respects robots.txt (Workshop paths are allowed)
• API-first approach (minimizes page scraping)
• Configurable delays and concurrency
• Graceful error handling

🐛 Troubleshooting

"403 Forbidden" Error

Cause: Invalid or missing Steam API key Fix: Get a new key from steamcommunity.com/dev/apikey

"No items found"

Causes:

• Wrong App ID
• Overly restrictive filters (tags/search)
• Game has no Workshop content

Fix: Visit steamcommunity.com/app/APPID/workshop/ to verify Workshop exists

"Item page not found" Warnings

Normal behavior: Some items are deleted/hidden by authors Impact: None - Actor continues with remaining items

Slow Performance

Solutions:

• Disable includeDiscussions (default: off)
• Disable includeChangeNotes if not needed
• Reduce maxItems for testing
• Increase requestIntervalMs if getting rate limited

Dataset Schema Not Showing

Verified Fix: Files are correctly placed in .actor/ directory with proper storages.dataset configuration

📚 FAQ

Q: Do I need a Steam account? A: Yes, to get an API key (free, takes 2 minutes)

Q: Can I scrape private Workshop items? A: No, only public items are accessible

Q: How often should I run this? A: Daily or weekly for trends, hourly for time-sensitive monitoring

Q: Can I scrape multiple games at once? A: Yes! Use "appId": [550, 294100, 255710]

Q: What happens if I hit the API limit? A: Actor logs a warning and stops gracefully. Increase delay or split into multiple runs.

Q: Can I get historical data? A: No, only current state. Run regularly to build history.

Q: Is this legal? A: Yes - scrapes public data, respects robots.txt, uses official API where available.

🆘 Support & Feedback

• Email: kontakt@barrierefix.de
• Issues: Report bugs or request features via email
• Updates: Follow actor changelog in Apify Console

🏷️ Keywords

Steam Workshop API, Steam mods scraper, gaming data extraction, Workshop statistics, mod metadata, Steam community data, user-generated content analysis, game Workshop scraper, Steam API integration, mod dependency tracker, Workshop trend analysis, gaming analytics, mod recommendation data, Steam Workshop automation, L4D2 mods, RimWorld Workshop, Cities Skylines mods, Counter-Strike Workshop, Steam mod database, Workshop items export, gaming community insights

---

Built with: Apify SDK v3, Crawlee v3, Playwright, TypeScript, Zod Version: 1.0.0 Last Updated: October 2025