Discourse Forum Scraper

Scrape topics from any Discourse-powered forum — latest, top, or search — with title, replies, views, likes, tags, and dates. Discourse powers communities like OpenAI, GitLab, Docker, Figma, and thousands of dev and product forums. Built on the public .json endpoints: no API key or setup needed.

Point it at one or many forums, pick a mode, and get a clean, structured feed you can export to CSV, Excel, or JSON — great for community monitoring and trend research.

Features

• Works on any Discourse site — just paste the forum's base URL. The Actor appends .json to its public endpoints.
• Three modes — the live Latest feed, Top topics by period, or full-text Search.
• Rich topic data — posts, replies, views, likes, tags, category, pinned/closed flags, and dates.
• Community monitoring — schedule it on a forum or a search term and track discussions over time.

Use cases

• Community & product monitoring — track what your users discuss on your Discourse forum.
• Competitive research — watch topics and engagement across other product communities.
• Support triage — pull the newest topics and spot rising questions.
• Datasets — build forum text datasets for analysis or LLM pipelines.

Input

Field	Type	Description
forumUrls	array	Base URLs of Discourse forums, e.g. https://meta.discourse.org.
mode	enum	latest, top, or search.
searchQuery	string	Keyword to search for. Used when mode is search.
period	enum	daily, weekly, monthly, yearly, all. Used when mode is top.
maxItems	integer	Max topics per forum. Default 100.

Example

{
  "forumUrls": [{ "url": "https://meta.discourse.org" }],
  "mode": "latest",
  "maxItems": 100
}

Output

{
  "site": "meta.discourse.org",
  "topicId": 12345,
  "title": "How to configure a category default view",
  "slug": "how-to-configure-a-category-default-view",
  "categoryId": 56,
  "tags": ["howto", "categories"],
  "postsCount": 8,
  "replyCount": 7,
  "views": 420,
  "likeCount": 15,
  "pinned": false,
  "closed": false,
  "createdAt": "2026-02-01T10:00:00.000Z",
  "lastPostedAt": "2026-02-03T18:22:00.000Z",
  "url": "https://meta.discourse.org/t/how-to-configure-a-category-default-view/12345",
  "scrapedAt": "2026-08-11T19:00:00.000Z"
}

Download the dataset as JSON, CSV, Excel, or XML, or pull it via the Apify API.

Pricing

Pay-per-event: $0.01 per run start (per GB of run memory) plus $0.50 per 1,000 topics. You only pay for topics actually returned. See the Pricing tab for current rates.

FAQ

Do I need an API key? No. This Actor reads the public Discourse .json endpoints — no key, no auth.

How do I know if a site uses Discourse? If {site}/latest.json returns JSON, it's Discourse. Non-Discourse sites are skipped with a warning (the run still succeeds).

Can I scrape multiple forums at once? Yes — add several base URLs to forumUrls.

How does Top work? Set mode to top and pick a period (daily, weekly, monthly, yearly, or all).

Disclaimer

This is an unofficial Actor. It is not affiliated with or endorsed by Discourse or any forum it scrapes. It uses public .json endpoints and returns publicly available content. You are responsible for how you use the data.