BiggerPockets Forum Discussions Scraper

Export BiggerPockets forum discussions for recurring real-estate investor topic research. Supply public forum listing or topic URLs to collect titles, canonical links, public authors, timestamps, opening text, counts, and visible first-page replies.

This is an independent tool, not affiliated with or endorsed by BiggerPockets. It does not require a BiggerPockets account and cannot access private or PRO-only discussions.

Who is it for?

Real-estate researchers, rental investors, and content teams can compare questions and themes across public categories. Schedule runs in Apify and compare stable topicId values downstream. This Actor does not itself send alerts or maintain historical snapshots.

Why use this Actor?

Each topic is one dataset row, with opening text and visible replies together. A global deduplication and result limit makes mixed listing/topic inputs predictable. An optional literal keyword filter works on both routes. Explicit coverage markers prevent first-page replies from being mistaken for a complete thread archive.

Getting started

1. Copy a public category URL such as https://www.biggerpockets.com/forums/12-starting-out.
2. Add it to startUrls.
3. Set maxItems and maxPages to bound collection.
4. Run the Actor and inspect the Topics table.
5. Download JSON, CSV, or Excel through Apify. JSON preserves nested reply arrays most naturally.

{
  "startUrls": [{"url": "https://www.biggerpockets.com/forums/12-starting-out"}],
  "maxItems": 25,
  "maxPages": 2
}

Inputs

Input	Default	Behavior
startUrls	Required	1–20 public HTTPS forum listing or topic URLs
maxItems	20	Global accepted unique topic limit, 1–500
maxPages	2	Listing pages per URL, 1–20; not reply pages
keyword	None	Case-insensitive literal substring of title OR opening text

URL queries and fragments are removed. Listing pagination starts at page 1. The Actor processes inputs in order and stops once maxItems is reached. Zero and unlimited limits are unsupported.

keyword trims surrounding whitespace but does not normalize internal whitespace, use word boundaries, apply stemming, or search replies. It is a local filter, not an upstream site search. When supplied, it requires exactly one source URL and maxPages: 1; other combinations fail validation. This bounds the scanned discussion window and avoids unbounded work for rare matches. A rare keyword can return no topics even when matches exist elsewhere.

Extracted data

Field	Meaning
topicId, title, url	Stable topic identity and canonical link
forum, author	Forum name and public author name, username, profile URL
createdAt, text	Source opening timestamp and plain opening text
replyCount, views, votes	Displayed source counters; unavailable counters are null
replies	Visible first-page replies with IDs, links, public authors, dates, text, votes
replyCoverage	Always first-page-visible
hasMoreReplyPages	Source pagination indicates more reply pages
sourceUrl, scrapedAt	Input discovery source and retrieval timestamp

Nested replies are included and deduplicated. The exported array can exceed the source's top-level page size. replyCount is not the number of exported replies. Hidden/blocked replies, signature/contact metadata, profile details, and separately featured replies outside the regular first-page tree are not collected.

Output example

This abbreviated example follows locally verified output; personal identity and discussion content are replaced with illustrative values.

{
  "topicId": "1234567",
  "title": "Example rental investing question",
  "url": "https://www.biggerpockets.com/forums/12/topics/1234567-example",
  "forum": "Starting Out",
  "author": {"name": "Sample Investor", "username": "exampleuser", "url": "https://www.biggerpockets.com/users/exampleuser"},
  "createdAt": "2026-01-15T12:00:00Z",
  "text": "Example question about managing rental properties.",
  "replyCount": 12,
  "views": 250,
  "votes": 3,
  "replies": [],
  "replyCoverage": "first-page-visible",
  "hasMoreReplyPages": false,
  "sourceUrl": "https://www.biggerpockets.com/forums/12-starting-out",
  "scrapedAt": "2026-01-16T12:00:00Z"
}

How much does it cost to export BiggerPockets forum discussions?

Pricing is pay per event: one start event per run and one item event per saved topic. Included replies have no separate event charge. Failed, excluded, filtered, and duplicate topics are not item-charged. A started run can incur the start fee even when its keyword matches nothing.

The run start fee is $0.00005. Topic prices are FREE $0.0023, BRONZE $0.002, SILVER $0.00156, and GOLD / PLATINUM / DIAMOND $0.0012 each. At BRONZE, 1 topic costs about $0.00205, 25 topics $0.05005, and 100 topics $0.20005 including one start. Consult the active Pricing tab before running. Apify Store spend tiers reflect qualifying aggregate monthly Store spend, not a private topic-volume discount. Platform usage and customer charge estimates are not guaranteed invoices. Revenue/payout estimates can change because of refunds, fraud, disputes, taxes, corrections or clawbacks.

Coverage and limitations

Only public anonymous forum pages are supported. No profiles, private messages, paid discussions, search URLs, blog posts, or login inputs. Blog/external-link cards on forum listings are skipped, so 25 source cards may yield fewer than 25 topics. First-page reply collection is intentional; this is not a complete historical thread export.

Direct HTTP is used without proxies or browser execution. Transient timeouts, resets, rate limits and server failures receive at most three attempts with bounded backoff. Authentication, redirects, challenges, invalid URLs, and missing structured topic data fail the run rather than inventing records. Data already saved before a later failure is partial output, not proof of full success.

Integrations

Use Apify schedules for periodic exports, then compare topicId and source counters in a database or spreadsheet. Deliver successful datasets through webhooks, Make, Zapier, or your own API client. Keep failed-run data separate from completed exports. Nested replies are best handled as JSON or flattened downstream.

Scheduled topic monitoring

Create an Apify schedule using the same category inputs and keep each run's dataset separately. Compare topic IDs to detect newly observed discussions and opening text/counters to detect changed observations. A missed topic between scheduled snapshots cannot be recovered automatically; listing order and public visibility can change. This tool does not guarantee exhaustive historical coverage.

API usage

Start a run once and retain its run ID and defaultDatasetId. Check that same run for completion; do not start a second run merely to poll.

curl -X POST \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H 'Content-Type: application/json' \
  'https://api.apify.com/v2/acts/automation-lab~biggerpockets-public-real-estate-forum-posts/runs' \
  -d '{"startUrls":[{"url":"https://www.biggerpockets.com/forums/12-starting-out"}],"maxItems":5,"maxPages":1}'

JavaScript:

import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/biggerpockets-public-real-estate-forum-posts').call({
  startUrls: [{ url: 'https://www.biggerpockets.com/forums/12-starting-out' }],
  maxItems: 5, maxPages: 1,
});
if (run.status !== 'SUCCEEDED') throw new Error(`Run ${run.id}: ${run.status}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems({ limit: 5 });
console.log(items);

Python:

import os
from apify_client import ApifyClient
client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/biggerpockets-public-real-estate-forum-posts').call(
    run_input={'startUrls': [{'url': 'https://www.biggerpockets.com/forums/12-starting-out'}], 'maxItems': 5, 'maxPages': 1})
if run['status'] != 'SUCCEEDED':
    raise RuntimeError(f"Run {run['id']}: {run['status']}")
print(client.dataset(run['defaultDatasetId']).list_items(limit=5).items)

MCP setup

Claude Code:

claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/biggerpockets-public-real-estate-forum-posts"

Claude Desktop, Cursor, and VS Code: use your client's remote HTTP MCP configuration and authenticated Apify connection. Configuration keys vary by client; a server definition has this shape:

{
  "mcpServers": {
    "apify": {
      "type": "http",
      "url": "https://mcp.apify.com?tools=automation-lab/biggerpockets-public-real-estate-forum-posts"
    }
  }
}

Example prompt: “Export five public topics from the BiggerPockets Starting Out forum, then summarize the opening questions.”

Discover actual tool names and argument schemas using scoped tools/list. Actor selection also exposes get-actor-run, get-dataset-items, get-key-value-store-record, and abort-actor-run; discovery is read-only, not permission to execute or abort.

For an authorized call, start once and retain the returned run ID, status, and storage IDs. These are metadata, not dataset rows. call-actor waits at most 45 seconds; configure a transport timeout above your selected wait plus transport margin. If nonterminal, use get-actor-run for that same run with bounded backoff (2, 4, 8, then at most 10 seconds) and a 120-second total deadline. After a client timeout, recover/check the known run rather than restarting. If no run ID was returned, report the outcome as uncertain and do not blindly rerun. At the deadline report pending with the run ID and stop polling. Report terminal failures honestly: any available rows are partial output, not a successful full result.

After success, use get-dataset-items with the returned dataset ID (defaultDatasetId in the run API), explicit limit: 20, offset: 0, and needed fields such as topicId,title,url,text,createdAt, according to the discovered argument schema. Prefer untransformed JSON and advance source offsets using pagination metadata, not transformed row counts. Admit at most 100 source rows and 64 KiB of serialized UTF-8 result content across all pages to model context, whichever comes first. Process pages host-side before model injection, enforcing the byte ceiling even for one oversized row/page; omit or summarize it with disclosure. Without interception, do not promise a hard byte limit or request unbounded results. Stop at source exhaustion or the budget, and report returned rows, the continuation offset, and partial/truncated status when incomplete. Full exports belong in storage or downstream processing outside model context. These consumer limits do not change Actor collection limits or output fields. No numeric token-savings claim is made.

Legality and responsible data handling

Use public data only for legitimate research and comply with applicable law, source terms, and privacy obligations. Public availability does not waive personal-data duties. Do not use this tool for unsolicited contact or sensitive profiling. No AI model processes inputs or scraped output inside this Actor.

The dataset contains public display names/usernames and discussion text; text itself may contain personal information. Signature metadata is omitted. Request bodies, source response bodies, cookies, and credentials are not logged. Output and logs remain in Apify according to your account retention settings; you control deletion of your run storages. This Actor creates no external copy or persistent cross-run cache of successful datasets.

Failed operations send sanitized diagnostic input, exceptions and actor/build/run IDs to our private GlitchTip service for repair; secret fields and URL queries are removed and reports are retained for 30 days. Apify hosts execution/storage, BiggerPockets serves public source data, and private GlitchTip receives failure diagnostics. Credentials remain outside public inputs and source.

FAQ and troubleshooting

Why fewer results than maxItems? It is a ceiling, not a promised quantity. Page limits, deduplication, keyword filtering, and blog cards can reduce results.

Are all replies included? No. Only visible first-page reply trees are included. Check coverage and pagination markers.

Why does a private or redirected URL fail? Anonymous forum scope excludes login and unsupported surfaces. Provide a direct public forum/topic URL.

Can I search the entire site? No. Choose relevant public categories and filter within bounded listing pages.

Where do I ask for help? Open an issue on this Actor's Apify Store page with the run link and a non-sensitive reproduction input. Never post credentials or private content.

Related workflows

This is a standalone BiggerPockets-specific product. It does not require another Actor, a paid source account, or an external enrichment service. General-purpose downstream Apify dataset exports and integrations can consume the result directly.