RedNote Xiaohongshu Scraper — Posts, Comments, Profiles, Video
Pricing
from $60.00 / 1,000 post scrapeds
Scrape RedNote (Xiaohongshu) posts, comments, profiles and video — six modes in one Actor. Chinese social media and user generated content at the source: Chinese reviews, China consumer opinion, china market research and the creator data behind influencer marketing. No login for posts and profiles.
All notable changes to the RedNote (Xiaohongshu) Scraper Actor.
user_posts note, search links in userUrluser_posts without a cookieString: the run card now says when RedNote reports more posts than the run got. Such a run serves only the posts the creator's anonymous profile page carries, and RedNote decides how many: the two prefill creators' pages carry 6 (RedNote counts 31) and 27 (RedNote counts 54). The note only fired when the page's own "has more" flag was set, which neither prefill creator sets, so the prefill (26 rows in every one of our runs since 23 Sep) said nothing about the short creator. It now also fires when RedNote's own post count (userPageData.posted) is above the posts the page carried, and not when a pull stopped at maxResults, on deltaMode, on a free sample or with a cookie (as before). It names no number of posts to unlock: RedNote's count includes posts that are not public (54 counted against at most ~47 public on one creator). Its old text, "anonymous profile pages serve only their first ~30 posts", was false (6 to 32 per creator measured) and is gone. The note is added only when it fits Apify's 500-character status cut; otherwise it goes to the log. The false "36 measured" (userUrls) and "first ~30" (deltaMode, cookieString, README) texts now say what the code delivers, and the README no longer says creator feeds always need the browser.notesCount is RedNote's own post count, plus exact likedCount and collectedCount. notesCount was always null on the default HTTP path although the page sends the count; the README said the page does not carry it. It is filled on both paths and documented as RedNote's figure, which includes posts that are not public. likedCount and collectedCount are new fields with the exact numbers behind the rounded totalLikes ("1万+" = likes + collects); on the HTTP path totalLikes and every other field keep their values. The content gate that drops a bare nickname-only row decides exactly as before: notesCount is set after it, so a row it dropped is not billed now because of a post count.profile: a count the page did not send is null, not 0. followers, following, totalLikes and notesCount were written as 0 whenever the page left them out (and in lite mode followers); they are null now, as on the HTTP path. A real reported 0 stays 0. Billing is unchanged (the gate reads null and 0 alike).userUrl / userUrls in user_posts (/search_result or /search_result_ai, on xiaohongshu.com or rednote.com) opened a browser three times, blamed a gated profile and armed the 2-hour gate cache, so the next run was skipped as an empty anonymous profile feed (runs XdIuk and SG8r, 8 Oct). It is now never fetched and never gate-cached. A run whose every entry is a search link stops before any fetch and says it is a search link, that keyword search needs a logged-in cookieString in search mode, and that zhorex/weibo-scraper searches Weibo by keyword with no login; in a list with real profiles the link is skipped with a short note, and it is never counted among the profiles a run timeout left unscraped. Other URLs, /search_result/<note id> and profile mode run as before.python -m pytest tests/test_profile_counts_and_search_links.py (one of them opens a local Chromium on two saved profile pages and skips without it).postUrls (post_details, comments, video): one entry per note id. The same post pasted twice — with two different xsec_tokens, as an xhslink.com share link that leads to a post already in the list, or as the exact same URL — was fetched, pushed and charged once per copy. The list now keeps one entry per note id at its first position. When that first copy has no xsec_token and a later copy of the same post has one, the later copy's URL is used in that slot (without a cookieString a tokenless URL cannot be served). A share link pasted twice is followed once. The trending sample used when postUrls is empty is not touched. Known limit: when one post is listed with two different xsec_tokens, only the first is used, so if that token no longer works and the later one would have, the post now comes back refused.userUrl / userUrls (user_posts, profile): one entry per user id. Exact repeats were already dropped; the same creator as /user/profile/<id>, /user/profile/<id>?xsec_token=... or the bare id was scraped twice, and every profile row it returned (and, outside deltaMode, every post row) was delivered and billed twice. The first entry naming an id keeps its position; an entry with no readable id is kept as before. A list left with a single creator still runs as a list, so its rows keep the profileUrl tag they had.searchQuery is one string (one search per run), so it has nothing to deduplicate.userUrls entry when other-form copies were skipped too). A list without a repeat, or whose only repeats are exact userUrl/userUrls copies (already dropped before), runs exactly as before: 56 offline scenarios (prefill, every mode, delta streams over several runs, charge limits, share links, trending samples, mode fallbacks) give the same rows, charges, events, status messages, navigations, key-value stores and logs as the previous commit. No price, event, default or prefill change. Offline tests: python -m pytest tests/test_repeated_inputs.py.deltaMode cookie run that got no post back is no longer told "nothing new". The 2026-10-10 change gave zero-row cookie runs in search and user_posts the logged-out cause, but not delta runs: a scheduled monitor whose cookie RedNote treated as logged out (0 search cards, or 0 posts rendered on every profile) was told "nothing new since the last run... deltaMode working as designed", and the idle hold it armed then said "the previous check found nothing new... posts published during the hold are delivered on the first check after it ends", although no post could arrive until the cookie was replaced. Such a run now says "no posts came back to check for new ones" with the same likely cause and fix as a run without deltaMode; a cookie run whose browser never started says that instead. The idle hold is still armed (it saves re-checking a dead session), but its skip message says what the previous check saw.cookieString is not held by an idle hold armed under another one. The hold now records a 16-character SHA-256 digest of the cookieString the empty check ran with (never the cookie itself), and a run that brings a different, non-empty cookieString checks right away instead of waiting up to ~50 h. A run with the same cookie, or with none, is held exactly as before, and so is any run against a hold saved by an earlier build (it has no digest).delta-check event is billed, as the README describes for every performed delta check; it is not priced, so nothing is billed.search run, "turn deltaMode OFF and this run returns the creator's full recent feed" (a search has no creator feed) now reads "...this run also returns the posts it skipped as already seen", and is left out when the check skipped none (its filterByMinLikes / filterByType removed every card, so a run without deltaMode would return nothing either). Such a search got cards back, so it is never called "no posts came back" or blamed on the cookie: run_search now counts the cards it was served before its filters. user_posts keeps its text.user_posts: a feed emptied by filterByType (e.g. video on a creator who posts image notes) was told the profile may be gated or, on a cookie run, that RedNote treated the cookie as logged out. The status now says how many posts the feed returned that were not of the chosen type, and names filterByMinLikes too when both filters removed posts. With or without a cookie.user_posts: a cookie run whose browser failed to start on every profile was told RedNote treated the cookie as logged out, although no page was loaded. It now says the browser failed to start.user_posts: the "stopped at your maxResults limit" note compared the run's total with maxResults, which applies per profile: 3 creators of 20 posts each at maxResults 50 got the note (60 rows) although every feed ran out at 20. A run with several profiles now gets it only when a profile's pull reached maxResults, and the note says how many did; the charge-limit note ("sized the scrape to N row(s), so there may be more to collect") uses the same per-profile test. One-profile runs and search are unchanged.cookieString no longer waiting out a hold. Offline tests: python -m pytest tests/test_status_truth.py.cookieString and came back with 0 items no longer says "add a cookieString". Measured 3-9 Oct 2026: seven daily searches whose cookieString was read as web_session alone returned 0 rows, and every status told the buyer to add the cookie they had pasted. In search, user_posts, comments and profile the zero-row status of a cookie run now gives as a likely cause that RedNote treated the cookie as logged out, names the cookies the run read (names only, never a value; a paste of more than 4 cookies, or one with no known RedNote name, is given as a count plus which of a1, webId and web_session it held), and asks for the whole Cookie: request header from a fresh login. A cookieString nothing could be read from is reported as such. A cookie search that RedNote answered with its recommendation feed names the cookies too, unless a long searchQuery would push the status past Apify's 500-character cut. Runs without a cookie keep their text byte for byte.cookieString" now copies the whole Cookie: request header. It used to copy web_session alone and say "That's it. One cookie.", which was never verified; the Pro tip that described the whole header is folded into Step 3. The troubleshooting table has a row for a cookie search that returns 0 items.deltaMode and cookieString descriptions and the README said a creator feed is "typically 16-61 posts". A cookieless user_posts run stops at the first ~30 posts the anonymous profile page serves (since 2026-09-21); the texts now say so. The cookieString text's "(the prefill caps it at 40)" went with that sentence: the prefill takes up to 20 posts from each of its two profiles, so 40 caps the prefilled run, not one creator's feed.python -m pytest tests/test_cookie_status_text.py.cookieString now reads the formats buyers actually paste. Only the a=b; c=d header value worked. The README's own DevTools steps copy the Value column of web_session, and that value pasted on its own gave 0 cookies; so did a JSON export from a cookie extension. A leading Cookie: corrupted the first cookie, and one cookie per line became one cookie with a newline in it, which made the browser fail to start. The parser now also reads a leading Cookie:/cookie:, one name=value per line, a JSON array of {name, value} objects (Cookie-Editor, EditThisCookie), and a bare value with no =, taken as web_session. The a=b; c=d format parses exactly as before (offline test against the previous function).web_session, a1, webId...) and counts the others; it never prints a value. It says so when a cookieString gave no cookie at all.= that is not a valid cookie name (a Netscape cookies.txt row, another pasted request header) is dropped instead of becoming a cookie: it could carry a value into the log, and a name with a tab or : can stop the browser from starting. A pasted block of request headers now applies the real a1 from its cookie: line. A bare value ending in = padding is read as web_session; name= with nothing after it still gives no cookie.domain is xiaohongshu.com (or that name no domain), so another site's session cookie is never sent to RedNote.python -m pytest tests/test_cookie_parser.py.user_posts): adding or losing a cookieString re-billed every post. Without a cookie RedNote hides the note ID, so a post was remembered by its cover file; with one it was remembered by its note ID. The first run after the switch (cookie added, cookie expired) found every post "new" and billed it again (offline: 10 of 10 posts). A post with a note ID now also remembers its cover file, and either key counts as seen. A memory saved by an earlier build learns the cover files on its first run with a cookie on this build. A cover shared by two different posts in one run (a placeholder image) is never used as a key.deltaStateKey: about 156,000 note IDs. Memories saved by earlier builds load as they are; nothing is reset.deltaMode are unchanged (checked offline against the previous commit). Offline tests: python -m pytest tests/test_delta_recency.py.cookieString, user_posts had one browser attempt per profile, so a navigation timeout (7 of 66 measured launches) delivered 0 rows. An attempt that raised after the browser was up now gets up to 2 more launches with the same cookie. A logged-in feed that comes back empty or filtered to nothing still costs exactly one launch, and so does a browser that fails to start (the same cookie would fail it again). Cookie launches wait up to 45 s for navigation instead of 25 s. Cookieless runs are unchanged (25 s).create_browser_context left the launched Chromium running when setting up the context or the cookies raised (measured with a real Chromium: a cookieString pasted one cookie per line makes add_cookies fail). It now closes it and re-raises the same error.run_user_posts now stops at that same instant and returns the posts it has; they are pushed, then billed, like any other rows, and the status says the profile hit the per-profile time limit, or names the run's own time limit when that is what ended it. A pull that already had its maxResults when the budget ran out gets no such note. The budget itself is unchanged.python -m pytest tests/ (64 tests, virtual clock, no network).search runs ("Want only NEW posts next time? Turn on deltaMode + a daily Schedule…") and the README banners/lines that sold deltaMode as the cheap "only new" way. The README's scheduling and delta-mode sections now describe both neutrally, with the same facts (plus one correction: each scheduled run writes to its own dataset — the README said runs append to the same one). The partial-failure status note on a non-delta provider search no longer ends with a deltaMode suggestion, and the README now says a cookieless scheduled search still pays the Actor start fee. Nothing else changed: no default, prefill, request, row, charge or delta behaviour was touched, and deltaMode works and is documented as before.Speed-focused release. Cuts in-actor scrape time for multi-URL modes by 50-80%; cuts single-URL search wall-clock by ~46%. No schema changes; one new optional input field added (networkCapture).
networkCapture input (default true). Reads the data as the page loads it, instead of waiting for the page to finish rendering. Falls back to v2.0 extraction automatically when capture finds nothing. Disable only for debugging.post_details, comments, video. Instead of one page per URL, N workers (N = concurrency) each hold one page for an entire chunk of URLs and navigate between them via page.goto(). Saves the ~500-800ms page-create overhead per URL after the first one in each chunk.BrowserPool.page() now yields (page, capture) instead of page. Single-page modes receive both; multi-URL modes get the pool and acquire their own pages internally.wait_for_selector reduced from 15000ms → 8000ms in navigate_and_wait and per-mode waiters. random_delay magnitudes halved across most call sites. wait_for_function for state hydration reduced from 8000ms → 5000ms in post_details, from 10000ms → 6000ms in user_posts and profile.search, user_posts, post_details, comments, video, profile) now consult ResponseCapture first and fall back to Vue state / DOM only when capture finds nothing. The dual-path design preserves v2.0 reliability while making the common case faster.fast_path=N, slow_path=M line so we can audit which path is winning in production.RedNote search is a single endless-scroll feed, so its pages cannot be loaded in parallel and search stays sequential. The practical concurrency win for high-volume search workloads is multiple distinct queries in separate runs. Note documented in src/modes/search.py docstring.
The page's own scripts are what load its data, so blocking scripts leaves a static HTML shell with no extractable data. liteMode already saves on the post-extraction formatting work and skips the comment scroll loop; that's the realistic ceiling without breaking extraction. Note documented in src/resource_blocker.py.
| Mode | v1.0.28 | v2.0 rc1 | v2.1 rc2 | Δ vs v2.0 | Target | Hit? |
|---|---|---|---|---|---|---|
| search 10 items duration | 39.1s | 58.9s | 31.9s | −46% | ≤ 20s | ❌ (closer, but Apify container startup is ~12-15s floor) |
| post_details per URL @ conc=3 | 60s | 28.7s | 5.7s | −80% | ≤ 18s | ✅ beat by 3× |
| search 10 items cost | $0.0132 | $0.0115 | $0.0108 | −6% | ≤ $0.008 | ❌ near-flat (the 30-50% target is gated by container time, not in-actor work) |
| post_details cost per URL @ conc=3 | $0.020 | $0.0097 | $0.0019 | −80% | ≥ 30% cheaper | ✅ beat by 2.7× |
| Search fast-path firing? | n/a | n/a | ✅ fast_path=1, slow_path=0, items=10 | new | n/a | ✅ |
The single-URL targets (search ≤20s, search cost ≤$0.008) were not hit because ~12-15s of the 31.9s wall-clock is Apify's container pull + Python+Playwright init, which the Actor code cannot influence. The realistic ceiling for in-actor-only search time is now ~17s (31.9 − 15 container) vs v2.0's ~44s, a ~60% reduction in actual scraping work. The multi-URL targets (post_details per URL) blew past the goal because chunking + same-page navigation + fast-path stack their wins.
| Mode | Run | Status | Note |
|---|---|---|---|
| search (full, 10 items) | UWM2U5P1TSGzlegM5 | ✅ 10/10, 19-field schema | fast_path=1 |
| search (lite, 5 items) | 6wbIfJsuW6K2Q8irS | ✅ 5/5, exact 8-field set | fast_path=1 |
| search (lite, 5 items, retry) | d0jxYRSkYcCc1crWG | ⚠ 0 items | Run-to-run variance — same input on parallel runs got 5 items elsewhere; unchanged from v2.0 known behavior |
| post_details 3 URLs conc=3 | iVtvQL5zFftKRwD3a | ✅ 3/3 real records, titles populated | Phase B chunking |
| comments | m9rwajgaugRe8ZCgk | ✅ Full-schema record, errorCode=None | — |
| video | H1tzaPVvh6R8sjw0i | ✅ videoUrl extracted, hasVideo=true | — |
| profile | GI14TJS0hkMxRCCBA | ✅ 18-field diagnostic, errorCode=login_required | Schema parity with v2.0; underlying gating unchanged |
| user_posts | kVz88U5mywmw1otoq | ⚠ 0 items | Parity with v2.0 known limitation for gated users |
5 of 6 modes returning useful data on the rc2 baseline. 1 mode (user_posts) consistent with v1.0.28/v2.0 limitation. 0 regressions detected.
latest tag — 1.0.28 (production buyers, untouched)rc1 tag — 2.0.1 (v2.0 baseline, available as rollback target if v2.1 ever flips to latest)rc2 tag — 2.1.2 (this release)Promotion to latest requires explicit greenlight after rc2 bake period.
Performance-focused release. Cuts per-item runtime and bandwidth at no schema cost. Output schema unchanged for existing fields; three new optional input fields added (concurrency, blockResources, liteMode).
concurrency input (default 4, max 10). post_details, comments, and video modes now process input URLs in parallel via a semaphore-bounded asyncio task group. Single-page modes (search, user_posts, profile) ignore the setting since they're inherently single-page.blockResources input (default true). Network-layer filter that aborts requests for images, fonts, media, and known tracker/analytics domains. Extracted data is unchanged — media URLs are still emitted in the output, only the bytes are skipped on the wire. Roughly halves bandwidth per page.liteMode input (default false). Returns a minimal field set per item: for posts {mode, postId, postUrl, xsecToken, type, title, likes, scrapedAt}; for profiles a similarly trimmed subset. The xsec_token in postUrl is preserved so search → post_details pipelines still work on lite output. Skips the scroll-load-more loop in comments mode and the DOM-walk fallbacks in video mode.BrowserPool (src/browser_pool.py). One browser + one shared context per run; pages acquired through an async with pool.page() context manager bounded by the concurrency semaphore. Pages are closed automatically after each unit of work.src/resource_blocker.py). Mode-aware: video mode opts into allow_media=True since some player implementations require the media network request to expose <video>.src.main.py now drives all modes through the BrowserPool rather than allocating a single shared page.post_details.py, comments.py, video.py rewritten around asyncio.gather over per-URL tasks. Each task acquires its own page from the pool, processes one URL, releases. No shared state between tasks.format_post_data / format_profile_data / format_comment_data accept a lite_mode kwarg.cookieString input behavior. Premium-mode extraction still routes through context.add_cookies.RESIDENTIAL).xsec_token propagation through search/user_posts → post_details/comments/video (the v1.0.28 fix).Original v2.0 spec included a stretch goal to load paginated results over plain HTTP after the browser opened the first page, targeting 70-90% cost reduction on search and user_posts. This phase was not pursued in v2.0: pagination stays in the browser.
Search "护肤教程" — 10 items (same input both runs)
| Metric | v1.0.28 (V3QBqmrWFRCnK0adv) | v2.0 (kArYMFGWMjA1OfkFL) | Δ |
|---|---|---|---|
| Memory peak | 1215 MB | 562 MB | −54% |
| Memory avg | 891 MB | 281 MB | −68% |
| Network RX | 6.51 MB | 4.42 MB | −32% |
| Run cost | $0.0132 | $0.0115 | −13% |
| Duration | 39.1s | 58.9s | +51% (within single-sample noise) |
| Items returned | 10 | 10 | — |
post_details — 3 URLs (v2.0 in parallel with concurrency=3 vs v1.0.28 baseline of 1 URL)
| Metric | v1.0.28 (PmvIfvCopAMp2fdKp, 1 URL) | v2.0 (h2lhMwzsHtSLg5xKP, 3 URLs) | Δ per URL |
|---|---|---|---|
| Duration | 60s / 1 URL = 60s/URL | 86s / 3 URLs = 28.7s/URL | −52% time/URL |
| Cost | $0.020 / URL | $0.029 / 3 = $0.0097 / URL | −52% cost/URL |
| Items with full schema | 1 | 3 | same |
The concurrency win is the big delta — multi-URL modes scale near-linearly with concurrency up to the proxy/rate-limit ceiling. Memory ceiling is also dramatically lower, leaving headroom for higher concurrency on the same memory bucket.
| Mode | Run | Status |
|---|---|---|
| search | kArYMFGWMjA1OfkFL | ✅ 10/10 real records, full schema |
| user_posts | zvIDhr46DM8f0BFdB | ⚠ 0 records — known limitation for gated users (unchanged from v1.0.28) |
| post_details | h2lhMwzsHtSLg5xKP | ✅ 3/3 real records, full schema |
| comments | eJzf2pti7p0QMnzUR | ✅ Real comments extracted |
| video | AkQdF9SqjpM3D8hwl | ✅ videoUrl extracted, hasVideo=true |
| profile | aj7eGyqD8l4srUJuM | ✅ Diagnostic record with errorCode (anonymous gated, unchanged) |
| search liteMode | u3JLgJG29kl9rLfa0 | ✅ 5 records, exactly 8 fields: {mode, postId, postUrl, xsecToken, type, title, likes, scrapedAt} |
5 of 6 modes returning useful data, 1 returning diagnostic record consistent with v1.0.28. No regression.
v1.0.28 remains tagged latest until v2.0 (currently on rc1) is promoted. Rollback from v2.0 = re-tag 1.0.28 as latest via Apify console build selector. No dual code paths to maintain.
Honest error categorization + app-share URL expectation-setting.
errorCode: "login_required" (RedNote sometimes drops the connection instead of redirecting to a login page; the buyer-actionable fix is the same).Critical bug fixes for post_details / comments / profile modes.
xsec_token now propagated through search and user_posts output (postUrl + dedicated xsecToken field). Previously stripped despite being captured in Vue state. Unblocks the canonical search → post_details pipeline.userPageData, userInfo, profile, otherInfo, currentUser in order.errorCode instead of empty rows; PPE never charged on failures."Scraped X items, Y failed" instead of misleading totals.See Apify console build history for earlier changes.