TapTap Scraper

Scrape TapTap (taptap.cn) — China's mobile game platform. Chart rankings (hot, playing and 20+ genre boards), game detail pages with rating/downloads/developer/tags/screenshots, keyword search across the community, and the hot-search board. A browser warms up first to clear the WAF challenge — no login required.

What this actor does

• Four modes: charts (default), byGameUrl, search, hotSearch
• Charts — 23 ranking boards (热门榜, 热玩榜, 新品榜, 预约榜, 热卖榜, 独家榜 + 17 genre boards) fetched from TapTap's public webapiv2/app-top/v2/hits ranking API. Each record: rank, game name, app id, package name, iOS store id, rating, rating counts (overall + latest version), downloads, player count, purchase count, genre tags, icon/banner/cover, supported platforms, release date, editor's-choice flag, price, boardName
• Board-identity verification — the actor navigates the rendered board page and reads the breadcrumb; if the served board doesn't match the requested chart, it retries up to 3 fresh loads and fails with a typed error instead of silently emitting the wrong board's data. If the page renders nothing (WAF shell on some datacenter IPs), the typed API data is used directly
• Game details — application/ld+json VideoGame blocks on app pages: rating (0–10) + rating count, total downloads, developer + developer page, price/currency, version, file size, genre, languages, release/update dates, icon + screenshots
• Search — type-and-enter search flow hitting the public agg-search API: community posts (title, content preview, author, likes/comments/views, linked game)
• Hot search — the 热搜 board (top keywords with their search URLs) from the webapiv2/search/v4/placeholder API, with a DOM fallback if the API is unreachable
• Warm-up: a real browser clears the JS challenge on the homepage first; ranking data comes from the public JSON API, app data from structured JSON — no captcha solving, no login
• Typed error records for invalid URLs / app ids / wrong boards
• Empty fields are omitted (strip_nulls before every push)

Output fields

Game records (recordType: "game"): gameName, altName, appId, gameUrl, packageName, itunesId, rating, ratingCount, latestRating, latestRatingCount, latestVersionRating, latestVersionRatingCount, downloads, playCount, purchaseCount, developer, developerUrl, publisher, price, priceCurrency, availability, genre, tags[] (charts), softwareVersion, fileSize, operatingSystem, languages[], description, iconUrl, bannerUrl, coverUrl, platforms[] (charts), editorChoice (charts), appleStoreUrl, officialDownloadUrl, screenshots[], datePublished, dateModified, rank (charts), boardName (charts), releaseDate (charts), installUrl, downloadUrl

Moment records (recordType: "moment", search): postTitle, postContent, author, authorUrl, momentId, likesCount, commentsCount, viewsCount, groupName, gameId, sourceUrl

Hot keyword records (recordType: "hotKeyword"): rank, keyword, keywordUrl

Error records (recordType: "error"): input, message

All records carry recordType and scrapedAt; game, moment and hot-keyword records also carry sourceUrl (the canonical game page, post, or keyword-search URL). Error records are diagnostics (input, message) without a sourceUrl.

Input

Field	Type	Default	Description
mode	select	charts	charts / byGameUrl / search / hotSearch
chart	select	download	23 boards: download (热门榜), played, new, reserve, sell, exclusive, action, strategy, idle, single, casual, sandbox, management, unriddle, shooter, multiplayer, acgn, music, scenario, swordsman, otome, independent, roguelike
gameUrls	array	["https://www.taptap.cn/app/168332"]	App page URLs (mode=byGameUrl)
searchQuery	text	原神	Search keyword
minRating	number (0–10)	–	Rating floor
containsKeyword	text	–	Substring on game/post titles, content, keywords
maxItems	int (1–200)	50	Hard cap

Examples

{ "mode": "charts", "chart": "download", "minRating": 8, "maxItems": 20 }
{ "mode": "charts", "chart": "acgn", "maxItems": 10 }
{ "mode": "byGameUrl", "gameUrls": ["https://www.taptap.cn/app/168332", "https://www.taptap.cn/app/2334"] }
{ "mode": "search", "searchQuery": "王者荣耀", "maxItems": 20 }
{ "mode": "hotSearch", "maxItems": 10 }

Data source

TapTap serves a JS WAF challenge to plain HTTP clients; the actor launches headless Chromium, lets the challenge resolve on the homepage, then reads:

• Rankings — the public webapiv2/app-top/v2/hits JSON API (type_name per board), paginated via from/limit=10
• Game details — /app/<id> pages' application/ld+json VideoGame blocks
• Search — the community agg-search API through the type-and-enter flow
• Hot search — the webapiv2/search/v4/placeholder API (the /search page stopped rendering the hot board in the DOM; the actor fetches the API through the warm-up session and falls back to DOM parsing if the API is unreachable)

All surfaces are login-free.

Limitations

• Some TapTap backend servers serve an unrendered app shell (no ranking content) for /top/* pages to datacenter IPs; the actor detects this, relies on the typed ranking API, and reports the page state in the log.
• Some TapTap backend servers intermittently 302 unknown /top/<slug> routes to the hot board; the board-identity check (breadcrumb) catches this and retries. If a board consistently misroutes, the actor emits a typed error record rather than wrong data.
• Search requires the type-and-enter interaction; the actor retries once if the agg-search call doesn't fire, and falls back to parsing the rendered result cards (gated on query relevance). When the API reports no results (its 没找到…推荐 tip), the actor emits 0 records instead of the recommended-content feed.
• screenshots and iconUrl come from TapTap's img-tc.tapimg.com CDN and are directly accessible (verified 200 image/* from a clean shell).
• Chart lists are TapTap's curated boards — no keyword filtering server-side.

Use cases

• Game-market rankings tracking (China mobile)
• Game metadata enrichment (rating, downloads, developer, tags)
• Community sentiment around titles via search
• App-store-style directories

FAQ

What's a gameUrl? The canonical app page, e.g. https://www.taptap.cn/app/168332. byGameUrl accepts URLs with extra paths (/app/<id>/topic) and query strings.

How fresh is the data? Charts update continuously (TapTap says every ~20 minutes for the hot board); app details update on the dateModified field.

Why search returns posts? TapTap's search tab mixes games and community moments; the actor emits both (game and moment records) when present.