DC Inside Scraper

Collect public posts and comment threads from Korean DC Inside galleries with auditable page and run coverage.

Why choose this Actor

This Actor is designed for research and VOC workflows that need more than a sample of recent post titles.

Capability	This Actor	Deprecated Store alternative	Typical single-page scraper
Major galleries	Yes	No	Sometimes
Minor galleries	Yes	Yes	Sometimes
Mini galleries	Yes	No	Rarely
Search inside selected galleries	Yes	No	Rarely
Direct post and list URLs	Yes	Yes	Varies
Comment pagination and reply depth	Yes	Yes	Often incomplete
Nested and flat analysis outputs	Yes	Yes	Usually one format
Korea residential proxy default	Yes	Proxy support only	Varies
Page and run coverage audits	KVS records	Not documented	No
Default memory	512 MB	4096 MB	Varies

The existing rafazafar/gall-dcinside-scraper Store Actor was marked deprecated and under maintenance when this Actor was created. It targets minor galleries and does not document gallery search or explicit coverage audits.

Recommended uses

• Korean consumer and product VOC from known DC Inside galleries
• Monitoring recent or recommended posts in selected communities
• Collecting full public discussion context for text analysis
• Exporting flat post/comment rows to CSV or BI tools
• Reproducible research that needs page-level failures and collection totals

Do not use it for

• Estimating nationally representative public opinion from DC Inside alone
• Private, deleted, login-only, or age-gated content
• Writing posts, voting, logging in, or collecting account-only data
• Discovering every relevant Gallery from a broad topic; provide the Gallery IDs or URLs first
• Guaranteed historical completeness beyond the page, date, and result limits you set

DC Inside can block data-center or non-Korean traffic. The default proxy setting uses a sticky Korean residential session. Proxy availability and traffic charges depend on the Apify account running the Actor.

Input

Provide at least one of:

• startUrls: DC Inside list or post URLs
• galleryIds: Gallery IDs such as programming, combined with galleryType

The Console form is prefilled with a bounded one-post programming Gallery run so Apify's automated quality test has a valid target and can finish within its five-minute limit. These prefill values do not replace the runtime defaults used by API calls, Schedules, or existing Saved Tasks.

Useful controls:

Field	Default	Purpose
searchQueries	empty	Search title and body inside every Gallery target
sort	latest	Use latest or recommended Gallery feed
dateFrom, dateTo	empty	Keep posts inside a KST date window
maxPosts	20	Global unique-post limit
maxPagesPerTarget	10	Hard page limit for each Gallery feed or search
scrapeComments	true	Fetch public comments and replies
maxCommentsPerPost	100	Per-post comment limit; 0 requests all accessible pages
outputFormat	nested	nested, flat, or postsOnly
includeHtml	false	Preserve post body HTML in addition to plain text
proxyConfiguration	Korean residential	Apify Proxy or custom proxy configuration
maxConcurrency	2	Concurrent post workers
requestDelayMs	700	Delay between requests from a worker

Gallery search example

{
  "galleryIds": ["programming"],
  "galleryType": "major",
  "searchQueries": ["AI"],
  "sort": "latest",
  "dateFrom": "2026-08-01",
  "maxPosts": 20,
  "maxPagesPerTarget": 3,
  "scrapeComments": true,
  "maxCommentsPerPost": 100,
  "outputFormat": "nested"
}

Direct post example

{
  "startUrls": [
    {
      "url": "https://gall.dcinside.com/board/view/?id=programming&no=1847628"
    }
  ],
  "maxPosts": 1,
  "scrapeComments": true,
  "maxCommentsPerPost": 20,
  "outputFormat": "flat"
}

Dataset output

Every Dataset row has a stable record_type:

• post: one successfully parsed public post
• comment: emitted only in flat mode

Important post fields include:

{
  "record_type": "post",
  "source": "dcinside",
  "gallery_id": "programming",
  "gallery_type": "major",
  "post_id": "1847628",
  "url": "https://gall.dcinside.com/board/view/?id=programming&no=1847628",
  "title": "Post title",
  "body_text": "Post body",
  "author_name": "Displayed nickname",
  "author_id": null,
  "author_ip": "Displayed IP fragment",
  "created_at": "2026-08-24T14:30:25+09:00",
  "view_count": 100,
  "recommendation_count": 5,
  "comment_count": 2,
  "media_urls": [],
  "comments": [],
  "comments_collected": 2,
  "comments_complete": true,
  "matched_query": "AI",
  "source_list_url": "https://gall.dcinside.com/board/lists/?id=programming&page=1",
  "scraped_at": "2026-08-24T06:00:00.000Z"
}

In nested mode, comments contains comment IDs, parent IDs, reply depth, public author fields, text, timestamps, and media URLs. In flat mode, the post row has an empty comments array and each comment is a separate Dataset row.

Audit records

Audits are deliberately kept out of the default Dataset:

• PAGE_AUDITS: sanitized request and parser status for list, post, and comment pages
• RUN_AUDIT: input summary, target/page/post counts, comment coverage, failures, and completion status

This separation keeps Dataset exports analysis-ready and prevents audit rows from being confused with business results if result-based pricing is configured later.

Pay-per-event integration

The source code uses two custom event names:

• post: one successfully saved post
• comment: one successfully saved or nested public comment/reply

Post events are charged through the durable post Dataset write. Flat comment events are charged through their Dataset writes; nested comment events are charged only after the containing post has been saved. RUN_AUDIT.charging records attempted and charged counts for both events plus whether the run reached its spending limit.

When enabling pay-per-event pricing in Apify Console, configure both custom events and remove apify-default-dataset-item. Keeping the synthetic Dataset event would charge flat comment rows twice and make nested and flat output prices inconsistent. Page requests, retries, failures, PAGE_AUDITS, and RUN_AUDIT are never custom charge events.

Coverage interpretation

• complete_for_requested_pages: all requested pages finished without recorded failures
• limited_by_max_posts: collection stopped at the requested global post limit
• partial: one or more target, post, or comment requests failed
• failed: no posts were saved and the run recorded failures

A successful HTTP response is not enough: a post is saved only after the expected DC Inside detail selectors are present and parsed.

Responsible collection

Only public page content is collected. Use conservative limits and delays, follow applicable laws and DC Inside terms, and avoid redistributing personal data. Displayed nicknames, user IDs, and partial IP strings are public page fields but may still be personal data in your jurisdiction.

Local development

npm install
npm test
npm run check
apify run

The project uses HTTP requests and Cheerio rather than a browser. Comments are requested from DC Inside's public JSON endpoint, with a mobile endpoint fallback.