phpBB Forum Scraper

Extract forums, topics, and posts from any phpBB discussion board — the phpBB Forum Scraper discovers a board's forum hierarchy, pulls topic listings (latest activity, a specific forum, or a search query), and walks each topic's pages for the full post stream (author, timestamp, HTML/Markdown/plain-text content). Give it a board like forum.cyclos.org or mantisbt.org/forums and it does the rest. Because it runs on the Apify platform, you also get scheduled runs, an API, proxy rotation, and monitoring out of the box — no server or scraping infrastructure of your own required.

Why use phpBB Forum Scraper?

phpBB powers tens of thousands of community forums, from open-source project support boards to hobbyist communities. Pulling their content by hand doesn't scale. This Actor is useful for:

• Research and archiving — preserve a community's knowledge base before a board goes offline or gets migrated.
• Support and product teams — mine bug reports, feature requests, and troubleshooting threads from a project's user forum.
• Community and market analysis — track sentiment, common questions, and active contributors across niche communities.
• Search and dataset building — feed forum content (as clean Markdown or plain text) into a search index, chatbot knowledge base, or NLP dataset.

How to use phpBB Forum Scraper

1. Open the Actor, set Forum URL to the board's base URL, and pick a Mode (see the five examples below — each is a different way to tell the Actor what to scrape).
2. Toggle Include full posts on to fetch every reply in each topic (off = topic metadata only — titles, views, replies, last-post info — much faster and cheaper).
3. Optionally set maxTopics, maxPostsPerTopic, and a request delay to control run size and stay polite to the target server.
4. Click Start and wait for the run to finish.
5. Open the Dataset tab to view, filter, or download the results.

If the target forum blocks datacenter IPs or restricts access by country, turn on Proxy configuration.

Examples

Every example below was run against a real, live phpBB board.

1. Discover a board's forums (mode: forums)

Run this first on any new board — it lists every forum and sub-forum along with its numeric id, which you'll need for the forum mode below.

{
  "forumUrl": "https://mantisbt.org/forums/",
  "mode": "forums"
}

{
  "type": "forum",
  "forum": "https://mantisbt.org/forums/",
  "forumId": "2",
  "title": "General Discussion",
  "url": "https://mantisbt.org/forums/viewforum.php?f=2",
  "topicCount": 469,
  "postCount": 2462
}

2. Scrape everything in one forum (mode: forum)

Use the forumId you found above to pull every topic (and, with includePosts, every post) from a single forum.

{
  "forumUrl": "https://mantisbt.org/forums/",
  "mode": "forum",
  "forumId": "2",
  "includePosts": true,
  "maxTopics": 50
}

Produces one topic row per thread plus one post row per reply, e.g.:

{
  "type": "post",
  "forum": "https://mantisbt.org/forums/",
  "topicId": "40521",
  "topicTitle": "Microsoft Calendars in MantisBT",
  "postId": "88905",
  "postNumber": 1,
  "username": "Taron",
  "createdAt": "2026-06-11T09:32:04+00:00",
  "contentText": "Hi, We use MantisBT at work to track bugs in our projects...",
  "contentMarkdown": "Hi,\n\nWe use MantisBT at work to track bugs in our projects...",
  "url": "https://mantisbt.org/forums/viewtopic.php?p=88905"
}

3. Follow board activity (mode: latest)

Pulls the board's own "active topics" list — whatever's been posted in recently, without you having to know which forum it's in.

{
  "forumUrl": "https://forum.cyclos.org/",
  "mode": "latest",
  "includePosts": false,
  "maxTopics": 20
}

{
  "type": "topic",
  "forum": "https://forum.cyclos.org/",
  "forumName": "Installation & Configuration issues",
  "topicId": "4311",
  "title": "Webshop order creation by Admin",
  "url": "https://forum.cyclos.org/viewtopic.php?t=4311",
  "pinned": false,
  "closed": false,
  "repliesCount": 1,
  "views": 3073,
  "lastPostedAt": "2026-07-13T14:03:27+00:00",
  "lastPostAuthor": "clement"
}

4. Search the board (mode: search)

Runs a keyword search and scrapes the matching topics — good for pulling everything about one product, error message, or feature across the whole board.

{
  "forumUrl": "https://admidio.org/forum/",
  "mode": "search",
  "searchQuery": "Docker",
  "includePosts": false,
  "maxTopics": 20
}

{
  "type": "topic",
  "forum": "https://admidio.org/forum/",
  "forumName": "Fragen und Probleme zu den Plugins",
  "topicId": "10348",
  "title": "Plugin Statistics Fehler nach Aktualisierung",
  "url": "https://admidio.org/forum/viewtopic.php?t=10348",
  "repliesCount": 1,
  "views": 30884,
  "lastPostAuthor": "josi",
  "lastPostUrl": "https://admidio.org/forum/viewtopic.php?p=36544"
}

Works on non-English boards too — search queries and post content in German, French, Hungarian, etc. all parse correctly.

5. Scrape specific topics (mode: topicUrls)

Already have a list of topic links (from your own research, a sitemap, a previous run)? Skip discovery entirely and scrape exactly those, even across different boards in one run.

{
  "mode": "topicUrls",
  "topicUrls": [
    { "url": "https://forum.cyclos.org/viewtopic.php?t=4308" },
    { "url": "https://mantisbt.org/forums/viewtopic.php?t=40521" }
  ],
  "includePosts": true
}

forumUrl isn't needed in this mode — each topic's board is inferred from its own URL. Note that listing-only stats (views, repliesCount) aren't available here, since phpBB only shows those on a forum's topic list, not on the topic page itself.

6. Market research: mining a support forum for product pain points

Software vendors' own user forums are a source of unfiltered, unprompted feedback — often more honest than a survey. A team evaluating whether to build a managed/hosted offering for Admidio (an open-source membership-management tool) ran a search for "Docker" — its self-hosted deployment method — to see what real users struggle with:

{
  "forumUrl": "https://admidio.org/forum/",
  "mode": "search",
  "searchQuery": "Docker",
  "includePosts": true,
  "maxTopics": 5,
  "maxPostsPerTopic": 3
}

19 rows came back across 5 matching topics. Reading the contentText of each post surfaced a clear, recurring pattern that a single anecdote wouldn't have shown:

• Auto-update breaks installs silently — a user running Watchtower (auto-pulling the latest Docker image) got a broken login page after an unannounced major-version jump, with no database-migration prompt.
• Config env-var syntax confusion — ADMIDIO_ORGANISATION set with : instead of = silently failed to apply, only resolved via a GitHub doc link from another community member.
• Plugin breakage post-upgrade — a plugin threw fatal errors after a Docker-based upgrade, unfixed for months until a follow-up plugin release.

All three threads are Docker-specific upgrade/config friction, not one-off bugs — real signal that "safe Docker upgrade tooling" or "managed hosting" addresses a genuine, currently unmet need in that community. This is the general pattern: run cheap search sweeps (includePosts: false first, to survey volume before committing to a full crawl) across the keywords that matter for your question — a competitor's name, an integration, an error message — across as many community forums as your research needs.

Input

Field	Description
forumUrl	Base URL of the board. Required for every mode except topicUrls.
mode	forums, latest, forum, search, or topicUrls.
forumId	Forum number or viewforum.php URL — required when mode=forum.
searchQuery	Required when mode=search.
topicUrls	Direct topic URLs — required when mode=topicUrls.
includePosts	Fetch each topic's full post stream, not just its listing metadata.
maxTopics	Cap on the number of topics to scrape (0 = unlimited).
maxPostsPerTopic	Cap on posts fetched per topic (0 = unlimited).
maxRequestsPerCrawl	Safety cap on total HTTP requests for the run (0 = unlimited).
maxConcurrency	Parallel HTTP requests.
requestDelaySecs	Delay between requests.
proxyConfiguration	Proxy settings — recommended for large crawls or boards that block/geo-restrict traffic.

Output

Each dataset item is a forum, topic, or post row, distinguished by its type field — see the worked examples above for what each looks like in practice. You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

Data table

Field	Description
type	forum, topic, or post.
title	Forum or topic title (forum/topic items).
topicTitle	Parent topic title (post items).
forumName	Forum a topic belongs to (topic items).
pinned / closed	Sticky/announcement and locked flags (topic items).
repliesCount / postsCount / views	Topic listing stats.
username	Post author's username (post items).
authorPostCount	Author's total post count, as shown on the board.
createdAt	ISO 8601 timestamp.
cookedHtml	Post content with original HTML formatting.
contentMarkdown	Post content converted to Markdown.
contentText	Post content as clean plain text.
url	Direct link to the forum, topic, or post.

How much does it cost to scrape a phpBB forum?

Cost scales with the number of pages fetched. With includePosts off, cost is roughly one request per ~30-50 topics (listing pages only). With it on, cost also scales with pages-per-topic — use maxTopics and maxPostsPerTopic to bound a run. A small board (a few hundred topics) typically costs a fraction of a compute unit on the Apify free tier.

Tips

• Run mode=forums first to get each forum's id, then target it directly with mode=forum.
• Turn includePosts off for a fast, cheap survey of topic activity before committing to a full post-level crawl.
• Increase requestDelaySecs if you see repeated retries in the log — it usually means the target is rate-limiting you.
• If a run comes back empty or with HTTP errors in the log, turn on Proxy configuration; some boards block datacenter IPs outright, and a couple only work with the residential proxy group specifically.

FAQ, disclaimers, and support

This Actor only extracts content that is publicly visible on the target forum. You are responsible for complying with the target site's Terms of Service and applicable law (including copyright and personal-data regulations) when scraping and reusing content. Tested against multiple independent real-world phpBB boards across several themes (prosilver, older subSilver2-derived styles, and translated/localized "Arty" themes) and languages (English, German, Hungarian). Heavily customized or broken themes may still need selector tweaks. phpBB has no native tags, likes, or "top topics" ranking, so those Discourse-style concepts have no phpBB equivalent and aren't part of the output. A board may also disable its own search feature for anonymous visitors — in that case mode=search will correctly return zero results.

Found a bug or need a custom version of this Actor? Open an issue on the Actor's Issues tab.