# BiggerPockets Forum Discussions Scraper (`automation-lab/biggerpockets-public-real-estate-forum-posts`) Actor

Export public BiggerPockets forum topics with titles, authors, dates, opening text, counts and visible first-page replies from listing or topic URLs.

- **URL**: https://apify.com/automation-lab/biggerpockets-public-real-estate-forum-posts.md
- **Developed by:** [Automation Lab](https://apify.com/automation-lab) (community)
- **Categories:** Social media, Real estate
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.20 / 1,000 item extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## BiggerPockets Forum Discussions Scraper

Export BiggerPockets forum discussions for recurring real-estate investor topic research. Supply public forum listing or topic URLs to collect titles, canonical links, public authors, timestamps, opening text, counts, and visible first-page replies.

This is an independent tool, not affiliated with or endorsed by BiggerPockets. It does not require a BiggerPockets account and cannot access private or PRO-only discussions.

### Who is it for?

Real-estate researchers, rental investors, and content teams can compare questions and themes across public categories. Schedule runs in Apify and compare stable `topicId` values downstream. This Actor does not itself send alerts or maintain historical snapshots.

### Why use this Actor?

Each topic is one dataset row, with opening text and visible replies together. A global deduplication and result limit makes mixed listing/topic inputs predictable. An optional literal keyword filter works on both routes. Explicit coverage markers prevent first-page replies from being mistaken for a complete thread archive.

### Getting started

1. Copy a public category URL such as `https://www.biggerpockets.com/forums/12-starting-out`.
2. Add it to `startUrls`.
3. Set `maxItems` and `maxPages` to bound collection.
4. Run the Actor and inspect the Topics table.
5. Download JSON, CSV, or Excel through Apify. JSON preserves nested reply arrays most naturally.

```json
{
  "startUrls": [{"url": "https://www.biggerpockets.com/forums/12-starting-out"}],
  "maxItems": 25,
  "maxPages": 2
}
```

### Inputs

| Input | Default | Behavior |
|---|---|---|
| `startUrls` | Required | 1–20 public HTTPS forum listing or topic URLs |
| `maxItems` | 20 | Global accepted unique topic limit, 1–500 |
| `maxPages` | 2 | Listing pages per URL, 1–20; not reply pages |
| `keyword` | None | Case-insensitive literal substring of title OR opening text |

URL queries and fragments are removed. Listing pagination starts at page 1. The Actor processes inputs in order and stops once `maxItems` is reached. Zero and unlimited limits are unsupported.

`keyword` trims surrounding whitespace but does not normalize internal whitespace, use word boundaries, apply stemming, or search replies. It is a local filter, not an upstream site search. When supplied, it requires exactly one source URL and `maxPages: 1`; other combinations fail validation. This bounds the scanned discussion window and avoids unbounded work for rare matches. A rare keyword can return no topics even when matches exist elsewhere.

### Extracted data

| Field | Meaning |
|---|---|
| `topicId`, `title`, `url` | Stable topic identity and canonical link |
| `forum`, `author` | Forum name and public author name, username, profile URL |
| `createdAt`, `text` | Source opening timestamp and plain opening text |
| `replyCount`, `views`, `votes` | Displayed source counters; unavailable counters are null |
| `replies` | Visible first-page replies with IDs, links, public authors, dates, text, votes |
| `replyCoverage` | Always `first-page-visible` |
| `hasMoreReplyPages` | Source pagination indicates more reply pages |
| `sourceUrl`, `scrapedAt` | Input discovery source and retrieval timestamp |

Nested replies are included and deduplicated. The exported array can exceed the source's top-level page size. `replyCount` is not the number of exported replies. Hidden/blocked replies, signature/contact metadata, profile details, and separately featured replies outside the regular first-page tree are not collected.

### Output example

This abbreviated example follows locally verified output; personal identity and discussion content are replaced with illustrative values.

```json
{
  "topicId": "1234567",
  "title": "Example rental investing question",
  "url": "https://www.biggerpockets.com/forums/12/topics/1234567-example",
  "forum": "Starting Out",
  "author": {"name": "Sample Investor", "username": "exampleuser", "url": "https://www.biggerpockets.com/users/exampleuser"},
  "createdAt": "2026-01-15T12:00:00Z",
  "text": "Example question about managing rental properties.",
  "replyCount": 12,
  "views": 250,
  "votes": 3,
  "replies": [],
  "replyCoverage": "first-page-visible",
  "hasMoreReplyPages": false,
  "sourceUrl": "https://www.biggerpockets.com/forums/12-starting-out",
  "scrapedAt": "2026-01-16T12:00:00Z"
}
```

### How much does it cost to export BiggerPockets forum discussions?

Pricing is pay per event: one `start` event per run and one `item` event per saved topic. Included replies have no separate event charge. Failed, excluded, filtered, and duplicate topics are not item-charged. A started run can incur the start fee even when its keyword matches nothing.

The run start fee is $0.00005. Topic prices are FREE $0.0023, BRONZE $0.002, SILVER $0.00156, and GOLD / PLATINUM / DIAMOND $0.0012 each. At BRONZE, 1 topic costs about $0.00205, 25 topics $0.05005, and 100 topics $0.20005 including one start. Consult the active Pricing tab before running. Apify Store spend tiers reflect qualifying aggregate monthly Store spend, not a private topic-volume discount. Platform usage and customer charge estimates are not guaranteed invoices. Revenue/payout estimates can change because of refunds, fraud, disputes, taxes, corrections or clawbacks.

### Coverage and limitations

Only public anonymous forum pages are supported. No profiles, private messages, paid discussions, search URLs, blog posts, or login inputs. Blog/external-link cards on forum listings are skipped, so 25 source cards may yield fewer than 25 topics. First-page reply collection is intentional; this is not a complete historical thread export.

Direct HTTP is used without proxies or browser execution. Transient timeouts, resets, rate limits and server failures receive at most three attempts with bounded backoff. Authentication, redirects, challenges, invalid URLs, and missing structured topic data fail the run rather than inventing records. Data already saved before a later failure is partial output, not proof of full success.

### Integrations

Use Apify schedules for periodic exports, then compare `topicId` and source counters in a database or spreadsheet. Deliver successful datasets through webhooks, Make, Zapier, or your own API client. Keep failed-run data separate from completed exports. Nested replies are best handled as JSON or flattened downstream.

### Scheduled topic monitoring

Create an Apify schedule using the same category inputs and keep each run's dataset separately. Compare topic IDs to detect newly observed discussions and opening text/counters to detect changed observations. A missed topic between scheduled snapshots cannot be recovered automatically; listing order and public visibility can change. This tool does not guarantee exhaustive historical coverage.

### API usage

Start a run once and retain its run ID and `defaultDatasetId`. Check that same run for completion; do not start a second run merely to poll.

```bash
curl -X POST \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H 'Content-Type: application/json' \
  'https://api.apify.com/v2/acts/automation-lab~biggerpockets-public-real-estate-forum-posts/runs' \
  -d '{"startUrls":[{"url":"https://www.biggerpockets.com/forums/12-starting-out"}],"maxItems":5,"maxPages":1}'
```

JavaScript:

```javascript
import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/biggerpockets-public-real-estate-forum-posts').call({
  startUrls: [{ url: 'https://www.biggerpockets.com/forums/12-starting-out' }],
  maxItems: 5, maxPages: 1,
});
if (run.status !== 'SUCCEEDED') throw new Error(`Run ${run.id}: ${run.status}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems({ limit: 5 });
console.log(items);
```

Python:

```python
import os
from apify_client import ApifyClient
client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/biggerpockets-public-real-estate-forum-posts').call(
    run_input={'startUrls': [{'url': 'https://www.biggerpockets.com/forums/12-starting-out'}], 'maxItems': 5, 'maxPages': 1})
if run['status'] != 'SUCCEEDED':
    raise RuntimeError(f"Run {run['id']}: {run['status']}")
print(client.dataset(run['defaultDatasetId']).list_items(limit=5).items)
```

### MCP setup

Claude Code:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/biggerpockets-public-real-estate-forum-posts"
```

Claude Desktop, Cursor, and VS Code: use your client's remote HTTP MCP configuration and authenticated Apify connection. Configuration keys vary by client; a server definition has this shape:

```json
{
  "mcpServers": {
    "apify": {
      "type": "http",
      "url": "https://mcp.apify.com?tools=automation-lab/biggerpockets-public-real-estate-forum-posts"
    }
  }
}
```

Example prompt: “Export five public topics from the BiggerPockets Starting Out forum, then summarize the opening questions.”

Discover actual tool names and argument schemas using scoped `tools/list`. Actor selection also exposes `get-actor-run`, `get-dataset-items`, `get-key-value-store-record`, and `abort-actor-run`; discovery is read-only, not permission to execute or abort.

For an authorized call, start once and retain the returned run ID, status, and storage IDs. These are metadata, not dataset rows. `call-actor` waits at most 45 seconds; configure a transport timeout above your selected wait plus transport margin. If nonterminal, use `get-actor-run` for that same run with bounded backoff (2, 4, 8, then at most 10 seconds) and a 120-second total deadline. After a client timeout, recover/check the known run rather than restarting. If no run ID was returned, report the outcome as uncertain and do not blindly rerun. At the deadline report pending with the run ID and stop polling. Report terminal failures honestly: any available rows are partial output, not a successful full result.

After success, use `get-dataset-items` with the returned dataset ID (`defaultDatasetId` in the run API), explicit `limit: 20`, `offset: 0`, and needed `fields` such as `topicId,title,url,text,createdAt`, according to the discovered argument schema. Prefer untransformed JSON and advance source offsets using pagination metadata, not transformed row counts. Admit at most 100 source rows and 64 KiB of serialized UTF-8 result content across all pages to model context, whichever comes first. Process pages host-side before model injection, enforcing the byte ceiling even for one oversized row/page; omit or summarize it with disclosure. Without interception, do not promise a hard byte limit or request unbounded results. Stop at source exhaustion or the budget, and report returned rows, the continuation offset, and partial/truncated status when incomplete. Full exports belong in storage or downstream processing outside model context. These consumer limits do not change Actor collection limits or output fields. No numeric token-savings claim is made.

### Legality and responsible data handling

Use public data only for legitimate research and comply with applicable law, source terms, and privacy obligations. Public availability does not waive personal-data duties. Do not use this tool for unsolicited contact or sensitive profiling. No AI model processes inputs or scraped output inside this Actor.

The dataset contains public display names/usernames and discussion text; text itself may contain personal information. Signature metadata is omitted. Request bodies, source response bodies, cookies, and credentials are not logged. Output and logs remain in Apify according to your account retention settings; you control deletion of your run storages. This Actor creates no external copy or persistent cross-run cache of successful datasets.

Failed operations send sanitized diagnostic input, exceptions and actor/build/run IDs to our private GlitchTip service for repair; secret fields and URL queries are removed and reports are retained for 30 days. Apify hosts execution/storage, BiggerPockets serves public source data, and private GlitchTip receives failure diagnostics. Credentials remain outside public inputs and source.

### FAQ and troubleshooting

**Why fewer results than maxItems?** It is a ceiling, not a promised quantity. Page limits, deduplication, keyword filtering, and blog cards can reduce results.

**Are all replies included?** No. Only visible first-page reply trees are included. Check coverage and pagination markers.

**Why does a private or redirected URL fail?** Anonymous forum scope excludes login and unsupported surfaces. Provide a direct public forum/topic URL.

**Can I search the entire site?** No. Choose relevant public categories and filter within bounded listing pages.

**Where do I ask for help?** Open an issue on this Actor's Apify Store page with the run link and a non-sensitive reproduction input. Never post credentials or private content.

### Related workflows

This is a standalone BiggerPockets-specific product. It does not require another Actor, a paid source account, or an external enrichment service. General-purpose downstream Apify dataset exports and integrations can consume the result directly.

# Changelog

This Actor's version history is a separate document: https://apify.com/automation-lab/biggerpockets-public-real-estate-forum-posts/changelog.md

# Actor input Schema

## `startUrls` (type: `array`):

Provide 1–20 HTTPS BiggerPockets forum listings or topic URLs. Listing URLs discover topics; topic URLs fetch one discussion. URL queries and fragments are removed; use maxPages for listing pagination. No login, profiles, PRO-only content, or site search URLs. Only url is consumed; request-list method, payload, headers and userData metadata are ignored. Requests always use anonymous GET.

## `maxItems` (type: `integer`):

Global limit on accepted, unique topic rows across all input URLs, after keyword filtering. Default 20; range 1–500. Zero and unlimited are unsupported. Listing page caps can produce fewer rows.

## `maxPages` (type: `integer`):

Read at most this many listing pages per URL, starting at page 1. Default 2; range 1–20. Stops earlier at source exhaustion or maxItems. Does not paginate topic replies: only visible first-page replies are exported.

## `keyword` (type: `string`):

Optional case-insensitive literal substring in the topic title OR opening text; surrounding whitespace is trimmed. Not whole-word matching, reply filtering, or upstream search. Applies equally to explicit topic URLs and discovered topics; only the bounded listing pages are searched. Requires exactly one source URL and maxPages=1; all other combinations fail validation to bound scanning.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://www.biggerpockets.com/forums/12-starting-out"
    }
  ],
  "maxItems": 20,
  "maxPages": 2
}
```

# Actor output Schema

## `dataset` (type: `string`):

One row per unique accepted topic.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://www.biggerpockets.com/forums/12-starting-out"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/biggerpockets-public-real-estate-forum-posts").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://www.biggerpockets.com/forums/12-starting-out" }] }

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/biggerpockets-public-real-estate-forum-posts").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://www.biggerpockets.com/forums/12-starting-out"
    }
  ]
}' |
apify call automation-lab/biggerpockets-public-real-estate-forum-posts --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,automation-lab/biggerpockets-public-real-estate-forum-posts"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/qk8h0Uico6xfNiV45/builds/3uzvwIxRkv27PbnwU/openapi.json