LIHKG Scraper (No API, No Login)

Scrape LIHKG (Hong Kong's largest online forum) — latest thread listings per category, and full thread content with all replies — via the public api_v2 JSON endpoint, output as structured data.

This actor is built for English-speaking researchers, analysts, and engineers who need Hong Kong forum sentiment / social listening data. Note: LIHKG itself is a Cantonese/Traditional-Chinese forum, so scraped title / content fields will be in Chinese — this tool structures and exports that data, it does not translate it.

Output Fields

Thread listing / thread content (ThreadRow)

Field	Type	Description
thread_id	string	Thread ID
title	string | null	Thread title
content	string | null	Thread body (null in listing mode)
own_text	string | null	What the author wrote themself, with quote blocks removed; empty string if the post is only a quote, null when content is null. Use this field for sentiment analysis
quoted_text	string | null	Text the author quoted (multiple quotes separated by a blank line; nested quotes stay inside their parent quote); null when there is no quote
quoted_author	string | null	Nickname of the quoted user. LIHKG's API does not provide it, so this is currently always null
author	string | null	Author nickname
category	string | null	Category name
reply_count	number | null	Number of replies
like_count	number | null	Like count
dislike_count	number | null	Dislike ("question mark face") count
created_at	string | null	Post time (ISO 8601)
last_reply_at	string | null	Last reply time (ISO 8601)
thread_url	string	Thread URL
source_mode	string	Scrape mode (thread-list / thread)
source_query	string	Query used (category ID or thread ID)
scraped_at	string	Scrape timestamp (ISO 8601)

Replies (ReplyRow)

Field	Type	Description
reply_id	string | null	Reply ID (post_id)
thread_id	string	Parent thread ID
floor	number | null	Floor number (msg_num)
author	string | null	Author nickname
content	string | null	Reply content (HTML tags stripped; quoted text is glued directly in front of the reply)
own_text	string | null	What the replier wrote themself, with quote blocks removed; empty string if the reply is only a quote, null when content is null. Use this field for sentiment analysis so the quoted person's tone is not attributed to the replier
quoted_text	string | null	Text the replier quoted (multiple quotes separated by a blank line; nested quotes stay inside their parent quote); null when there is no quote
quoted_author	string | null	Nickname of the quoted user. LIHKG's API does not provide it, so this is currently always null
like_count	number | null	Like count
dislike_count	number | null	Dislike count
created_at	string | null	Reply time (ISO 8601)
scraped_at	string	Scrape timestamp (ISO 8601)

AI sentiment label fields (on every row; all null when the feature is off or the row was not labelled)

Field	Type	Description
ai_relevance	number | null	0 to 1: probability that the post is about the tracked topic (aiTopic; in search mode the search keyword by default). null when there is no topic
ai_sentiment	number | null	Continuous sentiment score from -2 (clearly negative) to +2 (clearly positive)
ai_sentiment_label	string | null	negative (ai_sentiment <= -1.25), positive (>= +0.75), otherwise neutral
ai_sentiment_confidence	number | null	0 to 1: confidence in the sentiment judgement
ai_intent	string | null	complaint, praise, question or other
ai_intent_confidence	number | null	0 to 1: confidence in the intent judgement
ai_engine	string | null	The model version that answered (e.g. jev-1.13.0), kept as provenance
ai_status	string | null	ok, failed, skipped_budget (your spending limit was reached), skipped_free_cap (free plan, past 20 rows); null when the feature is off

Modes

Mode	Input	Output	Billing
thread-list	catIds (category ID list, or all)	Thread listing (ThreadRow)	$0.002/row
thread	threadIds / threadUrls	Thread content ($0.008/thread) + all replies ($0.003/reply)	by volume
search	keywords (search queries)	Site-wide search results (ThreadRow)	$0.002/row

Input Fields

Field	Type	Default	Description
mode	enum	thread-list	Mode: thread-list (listing) / thread (full thread) / search (keyword search)
catIds	array	—	Category IDs, e.g. 1 (Chat), 2 (Sports), 5 (Current Affairs). all expands to all 37 postable categories (aggregate boards such as Hot / Latest / Custom are excluded); maxItems is a per-category cap
threadIds	array	—	List of thread IDs
threadUrls	array	—	Thread URLs; the ID is extracted automatically
bulkInput	textarea	—	Bulk paste, one item per line, auto-merged into the matching field (keywords in search mode)
maxItems	integer	50	Max items to scrape (listing: per category; thread: total threads; search: per keyword). Maximum 2000
maxReplies	integer	—	Thread mode: max replies to collect per thread (each reply is billed per row). Takes the first N reply floors per thread (deleted floors can make it slightly fewer than N). Empty = no limit (default: the whole thread). Unrelated to maxItems, which limits the number of threads
keywords	array	—	Search mode: the queries to search for (brand names, product names, property developers). Listing mode: title keyword filter (leave empty for no filtering)
order	enum	desc_reply_time	Search-mode sort: desc_reply_time (latest replies first, for monitoring) / desc_create_time (newest threads first) / score (relevance first, for research)
dateFrom	string	—	Only threads with activity (post or last reply) since this time. Formats: 3d, 12h, 2w (relative) or MM/DD/YY (absolute). Empty = no filter; when set, paging stops early so scheduled monitoring stays cheap
aiLabels	boolean	false	Turn on AI sentiment labels (+$0.002 per successfully labelled row; free plan labels only the first 20 rows per run). See "AI sentiment labels"
aiTopic	string	—	Brand or topic to track, used for ai_relevance. Empty: search mode uses the search keyword; listing and thread modes skip relevance
proxyConfiguration	proxy	Apify datacenter proxy	On by default; without a proxy you share egress IPs with other runs and LIHKG rate-limits you (429). Turn off only for local testing or your own proxyUrls

Input Examples

Thread listing

{
  "mode": "thread-list",
  "catIds": ["1", "5"],
  "maxItems": 100
}

Full thread and replies

{
  "mode": "thread",
  "threadIds": ["4126423"],
  "maxItems": 50
}

Bulk thread URLs

{
  "mode": "thread",
  "threadUrls": [
    "https://lihkg.com/thread/4126423/page/1",
    "https://lihkg.com/thread/4000000/page/1"
  ],
  "maxItems": 50
}

Keyword search

{
  "mode": "search",
  "keywords": ["Taikoo Shing"],
  "order": "desc_reply_time",
  "maxItems": 100
}

All categories

{
  "mode": "thread-list",
  "catIds": ["all"],
  "maxItems": 20
}

Hong Kong brand monitoring

search mode plus dateFrom is keyword monitoring: schedule it daily and it only collects threads created or replied to that day, so the cost stays flat instead of growing with history.

{
  "mode": "search",
  "keywords": ["your brand"],
  "order": "desc_reply_time",
  "dateFrom": "1d"
}

Trigger it daily with an Apify Schedule and you accumulate brand-mention data automatically — no manual reruns, no duplicated old threads.

Need the cross-platform picture? Pair it with the author's other Traditional-Chinese sentiment actors: Threads (threads-feed-scraper), PTT (ptt-scraper, Taiwan forum), LIHKG (this actor, Hong Kong). Other Hong Kong forums (Baby Kingdom, HK01 comments — hk-forums) are coming soon.

AI sentiment labels (optional, +$0.002 per row)

Turn on the aiLabels input and every row gets the eight ai_ fields above, so you do not have to run a separate classification pass.

• What it judges: whether the post is about the brand you track (ai_relevance), its sentiment (ai_sentiment), and whether it is a complaint, praise, a question or something else (ai_intent). Put the brand in aiTopic; in search mode, leaving it empty uses the search keyword, while in listing and thread modes relevance is not asked and ai_relevance is null.
• Thresholds are yours to change: ai_sentiment_label uses default cut-offs (<= -1.25 negative, >= +0.75 positive); you can re-bucket ai_sentiment yourself. We suggest treating ai_relevance >= 0.55 as relevant (0.8 for strict filtering of titles). The confidence fields let you keep only the rows the model is sure about.
• Price: $0.002 per successfully labelled row (the ai-label event), on top of the normal scraping events. Rows that fail (failed), hit your spending limit (skipped_budget) or exceed the free allowance (skipped_free_cap) are not billed for AI; the data row itself is still delivered.
• Free plan: only the first 20 rows of each run are labelled; the rest get ai_status = skipped_free_cap.
• Accuracy: on a test of 300 real LIHKG items (4 brand keywords), on the 150 items not used to tune thresholds, the labels agreed with our reference answers 78% of the time for sentiment, 92% for relevance and 85% for intent. Sarcasm and Cantonese slang are still misjudged, so treat the labels as a fast first filter and use the confidence scores or spot checks for important conclusions.
• Where the data goes: when enabled, the text of labelled posts is sent to TypeSafe (US) for processing. According to its documentation TypeSafe does not train on API data. Leave the option off (the default) if you do not want text sent out.

Notes

• Only public content is scraped — no account or token required
• Usage terms: only public content is collected, with no login. Do not resell or republish full post or reply text, and respect LIHKG's terms of service. If the site operator asks for takedown or to stop access, we will comply and stop.
• Uses the api_v2 JSON endpoint (no browser — fast and low-cost)
• Reply content has HTML tags automatically stripped (<br>, <img>, <span>, etc.)
• Thread mode: maxItems limits the number of threads; all replies within each thread are fetched by default; set maxReplies to cap replies per thread (by floor)
• Billed per event: one-time apify-actor-start fee of $0.005, $0.002 per listing row, $0.008 per thread, $0.003 per reply; with AI sentiment labels on, plus $0.002 per successfully labelled row
• Auto-stops after 5 consecutive pages with no new data — no idle burn
• LIHKG's api_v2 may add an X-LI-DEVICE header check or a challenge in the future, and may block datacenter IPs. This is monitored via smoke tests on Apify Cloud (go/no-go); if blocked, the fallback is Apify residential proxy (proxyConfiguration) — not over-engineered in advance

FAQ

Q: Do I need to log in to view this data?

A: No. This actor only scrapes public thread listings and replies — nothing that requires login.

Q: Can I scrape multiple categories at once?

A: Yes. Add multiple IDs to catIds, or paste them one per line via bulkInput.

Q: How is reply HTML handled?

A: Automatically. The content field is plain text — HTML tags are stripped and HTML entities are decoded (& → &, < → <, etc.).

Q: How is search mode different from the keywords filter in thread-list mode?

A: In search mode the keywords are sent to LIHKG's site-wide search and you get search results across all categories. In thread-list mode the keywords only filter titles inside the category listing you already fetched — they never widen the search.

Q: Why is content null in thread-list mode?

A: The listing API doesn't return thread body content, only titles and metadata. Use thread mode if you need the full text.

Q: Can a long thread get expensive? A: Yes. Thread mode fetches the whole thread by default and bills every reply as one review-item ($0.003 each), and a popular thread can have thousands of replies. To cap the cost per thread, set the optional maxReplies (for example 100): each thread stops paging and billing once that many replies have been collected; the thread's own post is billed separately as one product-detail. Leave it empty for no limit, exactly as before. maxItems limits the number of threads, not replies.

Disclaimer

This tool is for academic research and lawful business analysis only. Users must comply with LIHKG's Terms of Service and applicable laws. Do not use this tool to invade others' privacy, spread false information, or engage in any other unlawful activity. The author accepts no responsibility for any misuse.

---

LIHKG scraper, Hong Kong forum, HK social sentiment, LIHKG API, Hong Kong online discussion, forum thread scraper, HK opinion data, keyword search

---

📚 Related tutorials & tools

Hands-on guides and free tools built on this actor, at AI Social API:

• 📖 Guide: LIHKG API: How to Get LIHKG and HKGolden Forum Data Without an Official API
• 📖 Tutorial: Build a no-code forum sentiment pipeline with Apify + n8n + GPT
• 📊 Data: Asian forum scrapability × cost table — measured on live runs: which forums need only a plain request, which need a residential IP, and what one item costs
• 🛠 Free tools: Forum sentiment quick-check · Apify cost calculator
• 🤝 Rather not run it yourself? I run it for you — scheduled daily, delivered to your Google Sheet or Slack, from US$299/mo.

Maintained by an operator running 30+ live Apify actors.