Discourse Community Scraper
Pricing
from $1.00 / 1,000 dataset items
Go to Apify Store
Discourse Community Scraper
Generalised scraper for any Discourse-based community forum (topics, posts, categories, search) via Discourse's JSON API.
Discourse Community Scraper
Pricing
from $1.00 / 1,000 dataset items
Generalised scraper for any Discourse-based community forum (topics, posts, categories, search) via Discourse's JSON API.
Base URL of any Discourse-powered forum, e.g. https://community.n8n.io
What to crawl. 'categories' lists the forum's categories/sections (use this to discover a categorySlug for mode=category). 'latest'/'top' pull the site-wide topic list, 'category' pulls one category, 'search' pulls search results, 'topicUrls' scrapes only the topics you list.
Required when mode=category. E.g. 'questions' or 'questions/12' (slug or slug/id) as it appears in the category URL.
Required when mode=search. Uses Discourse's /search.json endpoint (supports its query filters, e.g. 'order:latest #category').
Required when mode=topicUrls. Direct URLs to Discourse topics.
[]If true, fetch every topic's full post stream (original post + all replies) in addition to the topic metadata. If false, only topic-level metadata is scraped (much faster, fewer requests).
Maximum number of topics to scrape. 0 = unlimited.
Maximum number of posts to fetch per topic (only used when includePosts is true). 0 = unlimited.
Polite delay between outgoing API requests to avoid overloading the forum.
Maximum number of concurrent HTTP requests.