# Mumsnet Scraper - Threads, Replies, Quotes & Keyword Search (`metaldata/mumsnet-scraper-threads-replies-quotes-keyword-search`) Actor

Extract Mumsnet Talk threads and replies with full quote context, moderation status, and images. Scrape by URL or monitor any topic/brand across all boards via keyword search — built for UK market research and PR monitoring.

- **URL**: https://apify.com/metaldata/mumsnet-scraper-threads-replies-quotes-keyword-search.md
- **Developed by:** [Metal Data](https://apify.com/metaldata) (community)
- **Categories:** E-commerce, Lead generation, Social media
- **Stats:** 3 total users, 3 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Mumsnet Scraper — Threads, Replies, Quotes & Keyword Search

Extract structured, analysis-ready data from Mumsnet Talk — the UK's largest parenting forum, with millions of active monthly discussions across pregnancy, parenting, relationships, style & beauty, and dozens of other boards.

Scrape specific threads by URL, or monitor a topic or brand across all of Mumsnet with keyword search — built for market research, PR/media monitoring, and sentiment analysis at UK-consumer scale.

### Why this Actor

Mumsnet drives real purchasing decisions and public conversation in the UK, but until now there's been no dedicated way to extract its discussions at scale. This Actor was built directly against Mumsnet's real markup — not assumptions — and handles the details that matter for clean data:

- **Quote-aware extraction** — when a reply quotes an earlier post, that quoted text is captured separately (`quoted_author` / `quoted_text`), never mixed into the replier's own words.
- **Moderation-aware** — posts removed by Mumsnet's moderators are flagged (`moderation_status`) instead of silently appearing as missing data.
- **Genuine pagination** — long threads (100+ replies spanning many pages) are followed end-to-end with continuous, gap-free post numbering.
- **Real user images only** — photos users actually upload are captured; Mumsnet's own UI icons and reaction emoji are filtered out.

### Two ways to scrape

**1. Direct URLs** — paste specific thread or board URLs for precise, targeted extraction.

**2. Keyword search / monitoring** — supply search terms and the Actor discovers and scrapes matching threads across Mumsnet automatically. Narrow to one board, a date range, or a minimum/maximum reply count to focus on high-engagement discussions.

### Input options

| Field | What it does |
|---|---|
| `startUrls` | Specific thread or board URLs to scrape |
| `searchKeywords` | Topics/brands to discover and scrape matching threads for |
| `boardCategory` | Restrict search to one board (e.g. `teenagers`, `pregnancy`, `_chat`) |
| `dateFrom` / `dateTo` | Only include posts within a date range |
| `minReplies` / `maxReplies` | Only include threads within a reply-count range |
| `maxThreads` / `maxThreadsPerKeyword` | Safety limits on run size |
| `maxPagesPerThread` | How deep to follow pagination on long threads |
| `proxyConfiguration` | Standard Apify Proxy routing for larger runs |

### Output fields

Each scraped post is a single dataset row:

```json
{
  "thread_id": "5567579",
  "thread_title": "Turning 50 - what can I do to make myself feel better",
  "board_category": "style_and_beauty",
  "post_id": "154200495",
  "post_type": "reply",
  "post_number": 5,
  "author": "PenelopePritstop",
  "post_date": "2026-08-17T10:00:00.000Z",
  "body_text": "I don't go out without make up ever! ...",
  "quoted_author": null,
  "quoted_text": null,
  "images": [],
  "moderation_status": null,
  "thread_url": "https://www.mumsnet.com/talk/style_and_beauty/...",
  "page_number": 1,
  "scraped_at": "2026-08-18T09:32:56.099Z"
}
```

### Use cases

- **PR & media monitoring** — track brand or topic mentions across UK parenting/consumer conversation
- **Market research** — understand real consumer sentiment on products, services, and decisions (schools, childcare products, holidays, health & beauty)
- **Academic & social research** — study parenting discourse, community dynamics, or discussion patterns over time
- **Trend tracking** — scheduled runs with keyword search to monitor an evolving topic

### A note on the data

Mumsnet is a public, open forum — content is visible without an account and is indexed by search engines. This Actor extracts what's already public. It does not access private messages, does not bypass any login wall, and does not scrape anything requiring authentication.

# Actor input Schema

## `startUrls` (type: `array`):

Mumsnet thread URLs (e.g. https://www.mumsnet.com/talk/site\_stuff/4314481-...) or board/topic listing URLs (e.g. https://www.mumsnet.com/talk/parenting) to crawl.

## `searchKeywords` (type: `array`):

Keywords or phrases to search for. The Actor discovers matching Mumsnet threads via Google Search (site:mumsnet.com/talk) and scrapes them, in addition to any Start URLs above. Use this for topic/brand monitoring instead of pasting individual thread URLs.

## `boardCategory` (type: `string`):

Narrow search results to a single Talk board, e.g. 'teenagers', 'am\_i\_being\_unreasonable', 'private-school'. Find the exact slug in that board's URL: mumsnet.com/talk/THIS-PART. Leave blank to search all of Talk.

## `maxThreadsPerKeyword` (type: `integer`):

How many matching threads to pull in for each search keyword.

## `maxThreads` (type: `integer`):

Stop after this many threads (safety limit for your first runs).

## `maxPagesPerThread` (type: `integer`):

Mumsnet paginates long threads. Cap how many pages deep to follow per thread.

## `minReplies` (type: `integer`):

Skip threads with fewer replies than this. Checked once per thread from the page's own reply count — still costs one page load per candidate thread either way, since that count isn't visible before loading the thread.

## `maxReplies` (type: `integer`):

Skip threads with more replies than this. Leave at 0 for no upper limit.

## `dateFrom` (type: `string`):

Posts before this date are excluded from the output. Leave blank for no lower bound.

## `dateTo` (type: `string`):

Posts after this date are excluded from the output. Leave blank for no upper bound.

## `proxyConfiguration` (type: `object`):

Routes requests through Apify Proxy to avoid IP-based rate limiting or blocking, especially on larger runs. Defaults to Apify's automatic datacenter proxy.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://www.mumsnet.com/talk/site_stuff/4314481-How-do-you-link-to-a-thread"
    }
  ],
  "maxThreadsPerKeyword": 10,
  "maxThreads": 20,
  "maxPagesPerThread": 5,
  "minReplies": 0,
  "maxReplies": 0,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://www.mumsnet.com/talk/site_stuff/4314481-How-do-you-link-to-a-thread"
        }
    ],
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("metaldata/mumsnet-scraper-threads-replies-quotes-keyword-search").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [{ "url": "https://www.mumsnet.com/talk/site_stuff/4314481-How-do-you-link-to-a-thread" }],
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("metaldata/mumsnet-scraper-threads-replies-quotes-keyword-search").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://www.mumsnet.com/talk/site_stuff/4314481-How-do-you-link-to-a-thread"
    }
  ],
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call metaldata/mumsnet-scraper-threads-replies-quotes-keyword-search --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,metaldata/mumsnet-scraper-threads-replies-quotes-keyword-search"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/YxciCgaw5bBn78chy/builds/bOVG033wVz7X2Khs1/openapi.json