Pricing

from $3.00 / 1,000 results

Discourse Community Scraper

Scrape any public Discourse forum with latest topics, trending discussions, category browsing, tag filtering, full-text search, user profiles, and complete post threads. Works with meta.discourse.org, community forums, and any self-hosted Discourse.

Pricing

from $3.00 / 1,000 results

Rating

0.0

(0)

Developer

Crawler Bros

Actor stats

Bookmarked

Total users

Monthly active users

25 days ago

Last modified

Categories

Developer tools

Automation

Other

You can access the Discourse Community Scraper programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

{
  "openapi": "3.0.1",
  "info": {
    "version": "1.0",
    "x-build-id": "ScYRdnOa1ab9svv6c"
  },
  "servers": [
    {
      "url": "https://api.apify.com/v2"
    }
  ],
  "paths": {
    "/acts/crawlerbros~discourse-community-scraper/run-sync-get-dataset-items": {
      "post": {
        "operationId": "run-sync-get-dataset-items-crawlerbros-discourse-community-scraper",
        "x-openai-isConsequential": false,
        "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
        "tags": [
          "Run Actor"
        ],
        "requestBody": {
          "required": true,
          "content": {
            "application/json": {
              "schema": {
                "$ref": "#/components/schemas/inputSchema"
              }
            }
          }
        },
        "parameters": [
          {
            "name": "token",
            "in": "query",
            "required": true,
            "schema": {
              "type": "string"
            },
            "description": "Enter your Apify token here"
          }
        ],
        "responses": {
          "200": {
            "description": "OK"
          }
        }
      }
    },
    "/acts/crawlerbros~discourse-community-scraper/runs": {
      "post": {
        "operationId": "runs-sync-crawlerbros-discourse-community-scraper",
        "x-openai-isConsequential": false,
        "summary": "Executes an Actor and returns information about the initiated run in response.",
        "tags": [
          "Run Actor"
        ],
        "requestBody": {
          "required": true,
          "content": {
            "application/json": {
              "schema": {
                "$ref": "#/components/schemas/inputSchema"
              }
            }
          }
        },
        "parameters": [
          {
            "name": "token",
            "in": "query",
            "required": true,
            "schema": {
              "type": "string"
            },
            "description": "Enter your Apify token here"
          }
        ],
        "responses": {
          "200": {
            "description": "OK",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/runsResponseSchema"
                }
              }
            }
          }
        }
      }
    },
    "/acts/crawlerbros~discourse-community-scraper/run-sync": {
      "post": {
        "operationId": "run-sync-crawlerbros-discourse-community-scraper",
        "x-openai-isConsequential": false,
        "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
        "tags": [
          "Run Actor"
        ],
        "requestBody": {
          "required": true,
          "content": {
            "application/json": {
              "schema": {
                "$ref": "#/components/schemas/inputSchema"
              }
            }
          }
        },
        "parameters": [
          {
            "name": "token",
            "in": "query",
            "required": true,
            "schema": {
              "type": "string"
            },
            "description": "Enter your Apify token here"
          }
        ],
        "responses": {
          "200": {
            "description": "OK"
          }
        }
      }
    }
  },
  "components": {
    "schemas": {
      "inputSchema": {
        "type": "object",
        "required": [
          "forumUrl"
        ],
        "properties": {
          "forumUrl": {
            "title": "Forum URL",
            "type": "string",
            "description": "Base URL of the Discourse forum to scrape (e.g. https://meta.discourse.org or https://community.myapp.com)."
          },
          "mode": {
            "title": "Mode",
            "enum": [
              "latestTopics",
              "topTopics",
              "byCategory",
              "byTag",
              "searchTopics",
              "byUser",
              "getPosts"
            ],
            "type": "string",
            "description": "What data to scrape from the forum.",
            "default": "latestTopics"
          },
          "period": {
            "title": "Period (topTopics only)",
            "enum": [
              "daily",
              "weekly",
              "monthly",
              "quarterly",
              "yearly",
              "all"
            ],
            "type": "string",
            "description": "Time period for top topics ranking.",
            "default": "weekly"
          },
          "categorySlug": {
            "title": "Category slug (byCategory)",
            "type": "string",
            "description": "Category slug or path (e.g. 'support', 'meta/7'). Find it in the forum URL when browsing a category."
          },
          "tag": {
            "title": "Tag (byTag)",
            "type": "string",
            "description": "Tag name to filter topics by (e.g. 'bug', 'feature-request'). Used in byTag mode."
          },
          "searchQuery": {
            "title": "Search query (searchTopics)",
            "type": "string",
            "description": "Full-text search query for finding topics."
          },
          "username": {
            "title": "Username (byUser)",
            "type": "string",
            "description": "Forum username to get topics created by this user."
          },
          "topicId": {
            "title": "Topic ID (getPosts)",
            "minimum": 1,
            "maximum": 9999999,
            "type": "integer",
            "description": "Numeric topic ID to fetch all posts from (visible in the topic URL). Used in getPosts mode."
          },
          "apiKey": {
            "title": "API key (optional)",
            "type": "string",
            "description": "Discourse API key for accessing private forums. Leave empty for public forums."
          },
          "apiUsername": {
            "title": "API username (required if API key provided)",
            "type": "string",
            "description": "Discourse username associated with the API key."
          },
          "maxItems": {
            "title": "Max items",
            "minimum": 1,
            "maximum": 5000,
            "type": "integer",
            "description": "Maximum number of records to return.",
            "default": 50
          }
        }
      },
      "runsResponseSchema": {
        "type": "object",
        "properties": {
          "data": {
            "type": "object",
            "properties": {
              "id": {
                "type": "string"
              },
              "actId": {
                "type": "string"
              },
              "userId": {
                "type": "string"
              },
              "startedAt": {
                "type": "string",
                "format": "date-time",
                "example": "2025-01-08T00:00:00.000Z"
              },
              "finishedAt": {
                "type": "string",
                "format": "date-time",
                "example": "2025-01-08T00:00:00.000Z"
              },
              "status": {
                "type": "string",
                "example": "READY"
              },
              "meta": {
                "type": "object",
                "properties": {
                  "origin": {
                    "type": "string",
                    "example": "API"
                  },
                  "userAgent": {
                    "type": "string"
                  }
                }
              },
              "stats": {
                "type": "object",
                "properties": {
                  "inputBodyLen": {
                    "type": "integer",
                    "example": 2000
                  },
                  "rebootCount": {
                    "type": "integer",
                    "example": 0
                  },
                  "restartCount": {
                    "type": "integer",
                    "example": 0
                  },
                  "resurrectCount": {
                    "type": "integer",
                    "example": 0
                  },
                  "computeUnits": {
                    "type": "integer",
                    "example": 0
                  }
                }
              },
              "options": {
                "type": "object",
                "properties": {
                  "build": {
                    "type": "string",
                    "example": "latest"
                  },
                  "timeoutSecs": {
                    "type": "integer",
                    "example": 300
                  },
                  "memoryMbytes": {
                    "type": "integer",
                    "example": 1024
                  },
                  "diskMbytes": {
                    "type": "integer",
                    "example": 2048
                  }
                }
              },
              "buildId": {
                "type": "string"
              },
              "defaultKeyValueStoreId": {
                "type": "string"
              },
              "defaultDatasetId": {
                "type": "string"
              },
              "defaultRequestQueueId": {
                "type": "string"
              },
              "buildNumber": {
                "type": "string",
                "example": "1.0.0"
              },
              "containerUrl": {
                "type": "string"
              },
              "usage": {
                "type": "object",
                "properties": {
                  "ACTOR_COMPUTE_UNITS": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATASET_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATASET_WRITES": {
                    "type": "integer",
                    "example": 0
                  },
                  "KEY_VALUE_STORE_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "KEY_VALUE_STORE_WRITES": {
                    "type": "integer",
                    "example": 1
                  },
                  "KEY_VALUE_STORE_LISTS": {
                    "type": "integer",
                    "example": 0
                  },
                  "REQUEST_QUEUE_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "REQUEST_QUEUE_WRITES": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATA_TRANSFER_INTERNAL_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATA_TRANSFER_EXTERNAL_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "PROXY_SERPS": {
                    "type": "integer",
                    "example": 0
                  }
                }
              },
              "usageTotalUsd": {
                "type": "number",
                "example": 0.00005
              },
              "usageUsd": {
                "type": "object",
                "properties": {
                  "ACTOR_COMPUTE_UNITS": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATASET_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATASET_WRITES": {
                    "type": "integer",
                    "example": 0
                  },
                  "KEY_VALUE_STORE_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "KEY_VALUE_STORE_WRITES": {
                    "type": "number",
                    "example": 0.00005
                  },
                  "KEY_VALUE_STORE_LISTS": {
                    "type": "integer",
                    "example": 0
                  },
                  "REQUEST_QUEUE_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "REQUEST_QUEUE_WRITES": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATA_TRANSFER_INTERNAL_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATA_TRANSFER_EXTERNAL_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "PROXY_SERPS": {
                    "type": "integer",
                    "example": 0
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

Discourse Community Scraper OpenAPI definition

OpenAPI is a standard for designing and describing RESTful APIs, allowing developers to define API structure, endpoints, and data formats in a machine-readable way. It simplifies API development, integration, and documentation.

OpenAPI is effective when used with AI agents and GPTs by standardizing how these systems interact with various APIs, for reliable integrations and efficient communication.

By defining machine-readable API specifications, OpenAPI allows AI models like GPTs to understand and use varied data sources, improving accuracy. This accelerates development, reduces errors, and provides context-aware responses, making OpenAPI a core component for AI applications.

You can download the OpenAPI definitions for Discourse Community Scraper from the options below:

OpenAPI.json

If you’d like to learn more about how OpenAPI powers GPTs, read our blog post.

You can also check out our other API clients:

Discourse Community Scraper API in Python

Discourse Community Scraper API in JavaScript

Discourse Community Scraper API through CLI

Discourse Community Scraper API

Discourse Forum Monitor — Mentions & Feedback

bikram07/discourse-forum-monitor

Monitor any Discourse-powered forum for new topics, feature requests, and brand mentions — by latest or keyword search, across one or many forums at once. Keyless official Discourse JSON. Zero-config: latest topics from meta.discourse.org.

Bikram

Discourse Forum Scraper

automation-lab/discourse-scraper

Extract topics, posts, and discussions from any public Discourse forum. Supports latest topics, category filtering, and keyword search. No login required.

Stas Persiianenko

Discourse Scraper: Topics, Posts, Users & Search

perconey/discourse-scraper

Scrape any Discourse forum via the public REST API. Latest / top topics, category topics, full topic + posts, user profiles + activity, full-text search. No browser, no proxies, no auth. Pay only per result item.

Perconey

Discourse Forum Topics Scraper

parseforge/discourse-forum-topics-scraper

Gather social activity from Discourse Forum Topics with profile name, follower count, posts, replies and timestamps. Loved by community managers, brand watchers and trend researchers. Run on demand or on a recurring schedule and feed every row into your favourite analytics or workflow stack.

ParseForge

Hacker News Comments Scraper — Threads & Discussions

openclawmara/hackernews-comments-scraper

Scrape full comment trees from Hacker News stories. Extracts threaded discussions with author, text, points, depth, timestamp. Fetch by story ID, URL, or top/new stories. Ideal for sentiment analysis, tech discourse research, and developer community insights. Uses official Firebase API.

OpenClaw Mara

Google Forums Search

burbn/google-forums-search

Search discussions and forums across the web using Google Forums tab. Find forum posts from Reddit, Quora, Stack Overflow, Apple Support & more. Filter by time, country & language. Perfect for market research, brand monitoring & community insights.

Kevin

Google Forums API

johnvc/google-forums-search-api

Extract forum threads, Q&A, and community discussions from Google Search’s "Forums" tab. Supports 40+ countries, custom localization (GL/HL), and pagination. Get structured JSON data from Reddit, LinkedIn, Quora, and niche forums for sentiment analysis, market research, or AI training.

John

5.0

Flashback.org Scraper

lexis-solutions/flashback-org-scraper

Scrape Flashback.org forum threads and posts for Swedish community insights, capturing content, user profiles, and metadata at scale with flexible search, limits, and proxy support for research and monitoring.

Lexis Solutions

YouTube Community Posts Scraper

lurkapi/youtube-community-posts-scraper

Scrape community posts from YouTube channels. Extract text, images, polls, videos, likes, comments, and engagement data from any channel's Community tab.

LurkAPI

Threads Scraper

magicfingers/threads-scraper

Scrape Threads (by Meta) public data: user profiles, posts/threads, replies, and keyword search. Extracts follower counts, engagement metrics, media URLs, and more.