# Juejin Scraper (`crawlerbros/juejin-scraper`) Actor

Scrape 稀土掘金 (juejin.cn, ByteDance's developer community) without login - search articles, recommended feed, per-category and per-user articles with view/like/comment counts and tags.

- **URL**: https://apify.com/crawlerbros/juejin-scraper.md
- **Developed by:** [Crawler Bros](https://apify.com/crawlerbros) (community)
- **Categories:** Automation, Developer tools, News
- **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.00 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Juejin Scraper

Scrapes [稀土掘金](https://juejin.cn/) (Juejin, ByteDance's developer community) — **without login** — via the public `api.juejin.cn` endpoints.

### Data source

Articles and users come from Juejin's public web APIs (juejin.cn) — no login, no API key. The `search` mode is geo-restricted to China residential IPs (documented below); use `recommended`, `byCategory` or `byUser`. See the Notes section for further caveats.

### What it does

| Mode | Endpoint (public, no auth) | Output |
|---|---|---|
| `recommended` (default) | `POST recommend_api/v1/article/recommend_all_feed` | Recommended feed |
| `search` | `GET search_api/v1/search?query=&id_type=2&cursor=&limit=` | Paginated article search |
| `byCategory` | `POST recommend_api/v1/article/recommend_cate_feed` | Per-category articles (8 categories) |
| `byUser` | `POST content_api/v1/article/query_list` | All public articles by user id |

All endpoints verified live (HTTP 200 `err_no: 0` with real content) from a datacenter network, no auth headers needed.

> **Search geo-restriction:** Juejin's `search_api/v1/search` returns empty result pages (HTTP 200, `data: []`) from
> non-China datacenter IPs — verified against 8 request variants from Apify infrastructure. It works from
> China-based residential IPs (verified from the authoring network). The actor fail-softs with a clear status
> message (and a typed `EMPTY_ARTICLES` error record if the API returns items that cannot be parsed as articles).
> Use `recommended` / `byCategory` / `byUser` modes instead — they are not geo-restricted and are the actor's default.

> **Tag feed is also geo-gated:** `recommend_api/v1/article/recommend_tag_feed` returns `err_no: 0` with an empty
> `data` from non-CN datacenter IPs (verified with real tag ids), and the tag-list / article-detail routes
> (`tag/recommend_tag_list`, `content_api/v1/article/detail`) are unreachable from non-CN IPs — so `byTag` and
> `byArticleUrl` modes are documented as out of scope rather than shipped as zero-record modes.

### Output fields

**article records (`recordType: "article"`)** — `articleId`, `title`, `briefContent`, `viewCount`, `diggCount`, `collectCount`, `commentCount`, `hotIndex`, `rankIndex`, `authorId`, `authorName`, `authorAvatarUrl`, `authorJobTitle`, `authorCompany`, `authorLevel`, `tags[]`, `tagIds[]`, `categoryId`, `category`, `coverImage`, `isOriginal`, `isHot`, `isEnglish`, `ctime` (unix sec), `mtime`, `rtime`, `readTime`, `contentCount`, `originalType`, `linkUrl`, `sourceUrl`, `scrapedAt`.

**error records (`recordType: "error"`)** — a typed `EMPTY_ARTICLES` record when the search API returns items that cannot be parsed as articles. Invalid user ids and unknown categories fail soft with a clear status message and 0 records — nothing is silently dropped.

Every data record carries `sourceUrl`, `scrapedAt` and `recordType`; error records carry `inputValue` in place of `sourceUrl` (the offending input, not a page URL). Null/empty fields are stripped before push.

### Rehosted media

Juejin article cover images (`*.byteimg.com` with `x-expires`/`x-signature`) are signed and expire after ~8 days —
a customer would hit 403 on a stored URL. Covers are therefore **rehosted into the actor's key-value store** at
scrape time: `coverImage` becomes a permanent Apify KV URL and `coverImageOriginal` keeps the upstream signed URL
for traceability. Author avatars (`passport.byteacctimg.com`) are unsigned and passed through unchanged.

The rehosted KV URL is signed by the actor itself using the platform's `base62(hmac_sha256(secret, key)[:30hex])`
scheme (some SDK builds emit truncated signatures that the platform rejects with 403 from a clean shell) and is
verified to resolve `200 image/*` anonymously.

### Filters

- `minViewCount`, `minDiggCount`, `minCommentCount`
- `containsKeyword` (title substring)

### Notes

- All feeds paginate via the API's opaque `cursor` token; `has_more` drives the stop condition.
- **Category list was refreshed in this cycle:** Juejin migrated its category id space — the previous 12-entry
  enum had 10 stale ids that returned empty feeds. The current enum has 8 categories verified to return real data
  (`前端`, `后端`, `人工智能`, `Android`, `iOS`, `运维`, `代码人生`, `阅读`); `大数据`/`测试`/`云计算`/`开发工具`
  no longer resolve to a working `recommend_cate_feed` id and were dropped.
- `recommend_cate_feed` requires both `cate_id` and `category_id` plus `sort_type: 3` to return data; sort
  types 1 (hot) and 2 (latest) return empty pages from non-CN IPs, so the `sortBy` input only exposes the
  working `3` (recommended) option.
- Auto-escalation: on 403/429 the actor lazily engages the Apify AUTO (datacenter) proxy with exponential backoff.
- Memory: 1024 MB (Tier 1, plain HTTP).

# Actor input Schema

## `mode` (type: `string`):

What to fetch.

## `searchQuery` (type: `string`):

Free-text query (mode=search).

## `category` (type: `string`):

Juejin article category (verified to return data from datacenter IPs).

## `sortBy` (type: `string`):

Sort order for the recommended feed. Only the recommended sort is served by the API from non-CN IPs (sort 1/2 return empty pages).

## `userId` (type: `integer`):

Juejin user id, e.g. 3831141854615655. Either userId or userUrl.

## `userUrl` (type: `string`):

Juejin user profile URL as an alternative to userId.

## `minViewCount` (type: `integer`):

Drop articles with fewer views.

## `minDiggCount` (type: `integer`):

Drop articles with fewer likes.

## `minCommentCount` (type: `integer`):

Drop articles with fewer comments.

## `containsKeyword` (type: `string`):

Only keep articles whose title contains this substring (case-insensitive).

## `maxItems` (type: `integer`):

Hard cap on emitted records.

## Actor input object example

```json
{
  "mode": "recommended",
  "searchQuery": "前端",
  "category": "前端",
  "sortBy": "3",
  "maxItems": 30
}
```

# Actor output Schema

## `items` (type: `string`):

Dataset containing all scraped Juejin article records (recordType: article | error).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "mode": "recommended",
    "searchQuery": "前端",
    "category": "前端",
    "sortBy": "3",
    "maxItems": 30
};

// Run the Actor and wait for it to finish
const run = await client.actor("crawlerbros/juejin-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "mode": "recommended",
    "searchQuery": "前端",
    "category": "前端",
    "sortBy": "3",
    "maxItems": 30,
}

# Run the Actor and wait for it to finish
run = client.actor("crawlerbros/juejin-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "mode": "recommended",
  "searchQuery": "前端",
  "category": "前端",
  "sortBy": "3",
  "maxItems": 30
}' |
apify call crawlerbros/juejin-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,crawlerbros/juejin-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/BiibM7U10AvQoZ2m4/builds/Fe0BxL6qbrAE0dcKg/openapi.json