# Reddit VOC Collector (`research_master/reddit-voc-collector`) Actor

Discover public Reddit VOC candidates or collect reviewed posts and comment trees through Arctic Shift, with raw-response and completeness audits.

- **URL**: https://apify.com/research\_master/reddit-voc-collector.md
- **Developed by:** [Research Master](https://apify.com/research_master) (community)
- **Categories:**
- **Stats:** 2 total users, 1 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.70 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Reddit VOC Collector

An auditable Apify Actor for two separate Reddit research stages using the public [Arctic Shift](https://arctic-shift.photon-reddit.com/) archive:

1. `discover` searches selected public subreddits and writes deduplicated candidate posts.
2. `collect` accepts a separately reviewed post list and writes each returned post plus its public comment records.

The Actor intentionally has no combined mode. A discovery run never starts full comment collection, and its candidates are not automatically treated as approved research evidence.

### Scope and evidence boundary

- Collects only records that Arctic Shift marks as belonging to a public subreddit. Quarantined or mismatched records are rejected.
- Does not authenticate to Reddit, access private communities, bypass controls, or reconstruct `[deleted]` / `[removed]` text.
- Does not infer residence, nationality, language, or demographics from subreddit membership or flair.
- Arctic Shift is a third-party archive. Its availability, freshness, and coverage can differ from Reddit. A successful run proves what the endpoint returned at retrieval time, not a complete history of Reddit.
- Discovery scores are diagnostics for analyst review. They are not relevance labels, population estimates, or market shares.

### Quick start

#### 1. Discover candidates

```json
{
  "mode": "discover",
  "subreddits": ["CampingGear", "overlanding"],
  "queries": ["portable fridge", "12v refrigerator"],
  "includeTerms": ["fridge", "refrigerator"],
  "excludeTerms": ["minecraft"],
  "startDate": "2024-01-01",
  "endDate": "2026-08-31",
  "pageLimit": 50,
  "maxPagesPerQuery": 2,
  "maxCandidates": 200
}
```

Review the candidate Dataset outside the Actor. Keep the post ID, subreddit, all `matched_queries`, and a human-written reason for each selected post.

#### 2. Collect reviewed threads

```json
{
  "mode": "collect",
  "selectedPosts": [
    {
      "id": "abc123",
      "subreddit": "CampingGear",
      "reason": "Detailed first-person power and cooling setup",
      "matchedQueries": ["portable fridge", "12v refrigerator"]
    }
  ],
  "commentLimit": 5000
}
```

Duplicate selected IDs are merged before collection. Their matched queries are preserved and deduplicated. If duplicate entries assign one post ID to different subreddits, validation fails.

### Input rules

The Actor validates input again at runtime instead of relying only on the Console form.

| Mode | Required | Forbidden from the other mode |
| --- | --- | --- |
| `discover` | `subreddits`, `queries` | `selectedPosts`, `commentLimit` |
| `collect` | `selectedPosts` | discovery subreddits, queries, terms, dates, and page limits |

Hard bounds include 25 unique subreddits, 25 unique queries, 20 pages per pair, 1,000 potential discovery requests, 5,000 unique candidates, 100 selected posts, 10,000 comments per post, and 100,000 potential collect-mode Dataset records (`unique selected posts x (commentLimit + 1)`). Unknown input fields, control characters, invalid dates, invalid Reddit IDs, and invalid subreddit names fail before a request starts.

`requestSettings` permits only bounded delay, timeout, retry, and backoff values. The provider origin is fixed in source to `https://arctic-shift.photon-reddit.com`; input cannot supply a URL.

### Default Dataset

Only successful customer-facing records are stored in the default Dataset:

- `candidate` rows in `discover` mode;
- `post` and `comment` rows in `collect` mode.

Every record has a stable `dataset_record_id` in addition to its source IDs, canonical HTTPS Reddit URL, UTC retrieval time, source, and matched queries. Comment rows also preserve `parent_id`, calculated `depth`, and a root-to-comment `thread_path`. Removed/deleted bodies are stored as `null` with `content_status`, not reconstructed. Stable record IDs support downstream reconciliation, but Apify Dataset writes are not advertised as exactly-once.

Post rows expose source-fetch completeness both as compact fields and a `completeness` object. `comments_complete` describes the comment tree returned by the source, not Dataset write confirmation. It is conservative: a count gap or surplus, rejected node, unresolved `more` node, orphan/cycle, reached limit, or missing expected count prevents `comments_complete=true`. Actual confirmed Dataset rows, charge-limit skips, and any indeterminate failed batch are recorded in `POST_AUDITS` and `RUN_AUDIT`.

No page, retry, failure, raw-response, or audit row is written to the default Dataset.

### Key-value store audit records

| Key | Mode | Contents |
| --- | --- | --- |
| `RUN_AUDIT` | both | Final status, counts, sanitized errors, and stage summary |
| `DISCOVERY_AUDIT` | discover | Per-page URL/attempt/count status, caps, errors, and coverage limitations |
| `POST_AUDITS` | collect | Per-post request status, raw keys, comment completeness, and failures |
| `CHECKPOINT` | both | Completed discovery pages or processed/complete/incomplete post IDs |
| `RAW_DISCOVERY_*_MANIFEST` | discover | Manifest for one source-shaped search-page JSON response |
| `RAW_POST_*_MANIFEST` | collect | Manifest for one source-shaped post-lookup JSON response |
| `RAW_COMMENTS_*_MANIFEST` | collect | Manifest for one source-shaped comment-tree JSON response |
| `RAW_*_NNNN.json.gz` | both | Independently gzip-compressed raw JSON byte chunk |

Raw KVS artifacts are created only after public-scope and requested-subreddit/post checks. Private, quarantined, and mismatched records are excluded before storage. Each manifest and chunk key includes a hash of the sanitized content, so reusing a KVS cannot silently overwrite an older raw artifact referenced by a Dataset row. Successful response JSON is preserved in source-shaped form before normalization, except that deleted/removed `body`, `selftext`, related HTML fields, and deleted author markers are set to `null`; `_voc_redacted_fields` and the manifest redaction count make this explicit. The sanitized UTF-8 JSON bytes are split into at most 4 MiB per chunk, then every chunk is gzip-compressed and accompanied by byte counts and SHA-256 hashes in an ordered manifest. This keeps large comment trees below a single-record assumption while preserving byte-exact reconstruction of the compliant raw artifact. HTTP response bodies also have a fixed 128 MiB safety limit. Errors omit headers and response bodies. `RUN_AUDIT.status=partial` means at least one request failed, source completeness was not established, or the Dataset output stopped at a charge limit; if every requested discovery call or selected post fails, the Actor exits non-zero.

The checkpoint is written after every discovery page and selected post and is bound to a deterministic normalized-input fingerprint. It is durable run evidence, but a new Actor run uses new default storage unless the caller explicitly reuses storage; it is not advertised as automatic cross-run resume.

### Output and charging boundary

The source does not define or mutate Store pricing. If default Dataset item pricing is configured later, each successful candidate/post/comment row becomes a billable Dataset result. Audit and raw records remain in KVS and must not be priced as results. Dataset rows are written serially in bounded batches. Before every metered batch, the Actor checks the SDK's remaining default-result charge capacity and never calls `pushData` when that capacity is zero. It uses the returned `chargedCount` as the confirmed prefix; a failed batch is explicitly listed as indeterminate because an HTTP acknowledgement loss cannot prove that the batch was not stored. Verify live pricing separately before publication or any non-trivial run.

### Local verification

```bash
npm install
npm test
npm run check
```

Tests use local mocked HTTP responses and do not call Reddit or Arctic Shift.

### SOURCE\_FILES deployment

`create_actor_payload.mjs` produces private-by-default Apify create and version payloads without reading any credential:

```bash
node create_actor_payload.mjs
```

It writes `/tmp/reddit_voc_actor_create.json` and `/tmp/reddit_voc_actor_version.json`. The create payload sets `isPublic: false`, source type `SOURCE_FILES`, version `0.0`, build tag `latest`, 1 GB memory, and a one-hour timeout. Upload credentials belong only in an Authorization header used by a separately reviewed deployment helper; never add them to these payloads, Actor input, source, or logs.

# Actor input Schema

## `mode` (type: `string`):

Discover emits deduplicated candidate posts. Collect fetches posts and comment trees from a separately reviewed selectedPosts list.

## `subreddits` (type: `array`):

Required only in discover mode. Enter public subreddit names without r/.

## `queries` (type: `array`):

Required only in discover mode. Every query is run inside every selected subreddit.

## `includeTerms` (type: `array`):

Optional discover-mode terms. A candidate is marked likely\_relevant only when at least one term matches; omit to accept any non-excluded candidate.

## `excludeTerms` (type: `array`):

Optional discover-mode terms. Any hit makes likely\_relevant false, even when an include term also matches.

## `startDate` (type: `string`):

Optional discover-mode UTC lower bound in YYYY-MM-DD or RFC 3339 form.

## `endDate` (type: `string`):

Optional discover-mode UTC upper bound in YYYY-MM-DD or RFC 3339 form.

## `pageLimit` (type: `integer`):

Arctic Shift result limit for each page. Runtime default: 100.

## `maxPagesPerQuery` (type: `integer`):

Hard discover-mode pagination cap. Runtime default: 3.

## `maxCandidates` (type: `integer`):

Global hard cap on deduplicated candidate rows. Runtime default: 500.

## `selectedPosts` (type: `array`):

Required only in collect mode. Supply a manually reviewed list from a prior discovery run.

## `commentLimit` (type: `integer`):

Hard collect-mode limit passed to Arctic Shift. Runtime default: 5000.

## `requestSettings` (type: `object`):

Bounded timeout, delay, and retry controls. The Arctic Shift host is fixed and cannot be overridden.

## Actor input object example

```json
{
  "mode": "discover",
  "pageLimit": 100,
  "maxPagesPerQuery": 3,
  "maxCandidates": 500,
  "commentLimit": 5000
}
```

# Actor output Schema

## `results` (type: `string`):

Default Dataset. Discover produces candidate rows; collect produces post and comment rows.

## `runAudit` (type: `string`):

Sanitized run status, counts, failures, and coverage summary.

## `discoveryAudit` (type: `string`):

Per-page discovery coverage, raw response keys, truncation, and errors when mode is discover.

## `postAudits` (type: `string`):

Per-selected-post comment completeness, raw response keys, and failures when mode is collect.

## `checkpoint` (type: `string`):

Latest completed discovery pages or collected post IDs for within-run recovery evidence.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "pageLimit": 100,
    "maxPagesPerQuery": 3,
    "maxCandidates": 500,
    "commentLimit": 5000
};

// Run the Actor and wait for it to finish
const run = await client.actor("research_master/reddit-voc-collector").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "pageLimit": 100,
    "maxPagesPerQuery": 3,
    "maxCandidates": 500,
    "commentLimit": 5000,
}

# Run the Actor and wait for it to finish
run = client.actor("research_master/reddit-voc-collector").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "pageLimit": 100,
  "maxPagesPerQuery": 3,
  "maxCandidates": 500,
  "commentLimit": 5000
}' |
apify call research_master/reddit-voc-collector --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,research_master/reddit-voc-collector"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/NfKh7mf0gesriR960/builds/p88ixWwrZjaLL5pg5/openapi.json