# China Public Web Intelligence Extractor (`huakelilinjie/china-public-web-intelligence-extractor`) Actor

Extract structured JSON from public Chinese web pages and text-based PDFs. Capture clean text, titles, dates, issuers, metadata, and source URLs for research, monitoring, and AI workflows.

- **URL**: https://apify.com/huakelilinjie/china-public-web-intelligence-extractor.md
- **Developed by:** [林杰 李](https://apify.com/huakelilinjie) (community)
- **Categories:** AI, Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## China Public Web Intelligence Extractor

Turn public Chinese web pages and text-based PDFs into structured, analysis-ready records. The Actor returns clean text, title, publication date, issuing body, Chinese/English keywords, focus-keyword counts, hashes, source metadata, optional Markdown, and an optional English AI summary.

### What it is good for

- Public government policy and regulatory pages
- Company announcements and public research pages
- Text-based PDF reports and notices
- Monitoring inputs for analysts, researchers, and AI agents
- JSON/JSONL/CSV/Excel exports through the Apify dataset API

This Actor uses raw HTTP requests rather than a browser. It is fast and inexpensive, but it does not execute JavaScript and does not perform OCR.

### Safety and compliance boundaries

Only submit content you are legally allowed to access and process. The Actor deliberately rejects localhost, private networks, link-local addresses, internal hostnames, credential-bearing URLs, and non-HTTP protocols. It does not log in, bypass CAPTCHAs, evade paywalls, or defeat access controls. Respect each source's terms, robots policy, copyright, privacy rights, and applicable law.

### Input

- `startUrls`: 1-20 public HTTP(S) pages or PDFs.
- `focusKeywords`: optional terms whose case-insensitive occurrence counts are returned.
- `topKeywords`: number of automatically extracted keywords.
- `includeMarkdown`: include a ready-to-save Markdown field.
- `generateEnglishSummary`: call an OpenAI-compatible API using the encrypted BYOK fields.
- Resource limits cap returned characters, PDF pages, download bytes, and request time.

The optional API key is marked secret in the Apify input schema. It is sent only to the configured public HTTPS endpoint. When an Actor-owner `SUMMARY_API_KEY` secret is configured, the endpoint and model are taken from owner-controlled environment variables, preventing an input URL from receiving the owner's key.

### Output

Each requested URL creates one dataset record. Successful records include `content`, metadata, hashes, keywords, and optional Markdown/summary. Failed records include a stable `errorCode` and readable `errorMessage`; one bad source does not fail the whole batch.

Successful results use the `document-processed` pay-per-event event. Invalid or failed URLs are returned without that custom charge.

### Local development

```powershell
py -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements-dev.txt
.\.venv\Scripts\python.exe -m pytest
.\.venv\Scripts\python.exe -m public_intel.local_cli examples\input.json --output output\results.jsonl
````

If a local transparent proxy returns RFC 2544 `198.18.0.0/15` fake DNS addresses, opt in only for that local command:

```powershell
$env:PUBLIC_FETCH_ALLOW_PROXY_FAKE_IPS = "1"
.\.venv\Scripts\python.exe -m public_intel.local_cli examples\input.json --output output\results.jsonl
Remove-Item Env:PUBLIC_FETCH_ALLOW_PROXY_FAKE_IPS
```

This override is intentionally off by default and must not be enabled in the Apify deployment.

The Apify SDK entrypoint is:

```powershell
.\.venv\Scripts\python.exe -m public_intel
```

When run through the SDK locally, its default storage is written under `storage/`.

### Monetization setup

In the Actor's Apify Console Publication tab, select pay per event and add `document-processed` as the primary custom event. Keep Apify's synthetic `apify-actor-start` event enabled. Start with a low price and confirm real per-run compute/storage costs before increasing it.

### Known limitations

- No JavaScript rendering
- No scanned-PDF OCR
- Metadata quality depends on source markup
- Some sites block data-center IPs or raw HTTP clients
- Automatic keyword extraction is statistical and may need domain-specific tuning

# Actor input Schema

## `startUrls` (type: `array`):

One to 20 public HTTP(S) web pages or PDF documents. Private-network, localhost, credentialed, and non-HTTP URLs are rejected.

## `focusKeywords` (type: `array`):

Optional Chinese or English terms to count in every extracted document.

## `topKeywords` (type: `integer`):

Number of automatically extracted keywords to return.

## `includeMarkdown` (type: `boolean`):

Also return a ready-to-save Markdown representation. JSON and CSV exports are always available from the Apify dataset.

## `generateEnglishSummary` (type: `boolean`):

Generate a concise English summary with an OpenAI-compatible chat-completions API. Extraction still succeeds if the summary service is unavailable.

## `summaryApiKey` (type: `string`):

Encrypted by Apify. Sent only to the configured public HTTPS summary endpoint and never returned in output.

## `summaryApiBaseUrl` (type: `string`):

Public HTTPS OpenAI-compatible base URL, normally ending in /v1.

## `summaryModel` (type: `string`):

Model identifier accepted by the selected summary API.

## `maxContentChars` (type: `integer`):

Maximum cleaned-content length per document.

## `maxPdfPages` (type: `integer`):

Maximum number of text-based PDF pages processed per file. Scanned-image OCR is not included in this version.

## `maxDownloadBytes` (type: `integer`):

Hard per-document download limit after decompression.

## `requestTimeoutSecs` (type: `integer`):

Total timeout for each source request.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://www.gov.cn/zhengce/index.htm"
    }
  ],
  "focusKeywords": [],
  "topKeywords": 12,
  "includeMarkdown": true,
  "generateEnglishSummary": false,
  "summaryApiBaseUrl": "https://api.deepseek.com/v1",
  "summaryModel": "deepseek-chat",
  "maxContentChars": 80000,
  "maxPdfPages": 100,
  "maxDownloadBytes": 20971520,
  "requestTimeoutSecs": 30
}
```

# Actor output Schema

## `results` (type: `string`):

JSON, JSONL, CSV, XML, RSS, and Excel exports are available through the dataset endpoint.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://www.gov.cn/zhengce/index.htm"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("huakelilinjie/china-public-web-intelligence-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://www.gov.cn/zhengce/index.htm" }] }

# Run the Actor and wait for it to finish
run = client.actor("huakelilinjie/china-public-web-intelligence-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://www.gov.cn/zhengce/index.htm"
    }
  ]
}' |
apify call huakelilinjie/china-public-web-intelligence-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=huakelilinjie/china-public-web-intelligence-extractor",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "China Public Web Intelligence Extractor",
        "description": "Extract structured JSON from public Chinese web pages and text-based PDFs. Capture clean text, titles, dates, issuers, metadata, and source URLs for research, monitoring, and AI workflows.",
        "version": "0.1",
        "x-build-id": "TBCQZT2iF5f9qAepk"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/huakelilinjie~china-public-web-intelligence-extractor/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-huakelilinjie-china-public-web-intelligence-extractor",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/huakelilinjie~china-public-web-intelligence-extractor/runs": {
            "post": {
                "operationId": "runs-sync-huakelilinjie-china-public-web-intelligence-extractor",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/huakelilinjie~china-public-web-intelligence-extractor/run-sync": {
            "post": {
                "operationId": "run-sync-huakelilinjie-china-public-web-intelligence-extractor",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "required": [
                    "startUrls"
                ],
                "properties": {
                    "startUrls": {
                        "title": "Public document URLs",
                        "minItems": 1,
                        "maxItems": 20,
                        "type": "array",
                        "description": "One to 20 public HTTP(S) web pages or PDF documents. Private-network, localhost, credentialed, and non-HTTP URLs are rejected.",
                        "items": {
                            "type": "object",
                            "required": [
                                "url"
                            ],
                            "properties": {
                                "url": {
                                    "type": "string",
                                    "title": "URL of a web page",
                                    "format": "uri"
                                }
                            }
                        }
                    },
                    "focusKeywords": {
                        "title": "Focus keywords",
                        "maxItems": 30,
                        "type": "array",
                        "description": "Optional Chinese or English terms to count in every extracted document.",
                        "items": {
                            "type": "string"
                        },
                        "default": []
                    },
                    "topKeywords": {
                        "title": "Generated keyword count",
                        "minimum": 0,
                        "maximum": 50,
                        "type": "integer",
                        "description": "Number of automatically extracted keywords to return.",
                        "default": 12
                    },
                    "includeMarkdown": {
                        "title": "Include Markdown",
                        "type": "boolean",
                        "description": "Also return a ready-to-save Markdown representation. JSON and CSV exports are always available from the Apify dataset.",
                        "default": true
                    },
                    "generateEnglishSummary": {
                        "title": "Generate English summary",
                        "type": "boolean",
                        "description": "Generate a concise English summary with an OpenAI-compatible chat-completions API. Extraction still succeeds if the summary service is unavailable.",
                        "default": false
                    },
                    "summaryApiKey": {
                        "title": "Summary API key",
                        "type": "string",
                        "description": "Encrypted by Apify. Sent only to the configured public HTTPS summary endpoint and never returned in output."
                    },
                    "summaryApiBaseUrl": {
                        "title": "Summary API base URL",
                        "pattern": "^https://.+",
                        "type": "string",
                        "description": "Public HTTPS OpenAI-compatible base URL, normally ending in /v1.",
                        "default": "https://api.deepseek.com/v1"
                    },
                    "summaryModel": {
                        "title": "Summary model",
                        "minLength": 1,
                        "maxLength": 100,
                        "type": "string",
                        "description": "Model identifier accepted by the selected summary API.",
                        "default": "deepseek-chat"
                    },
                    "maxContentChars": {
                        "title": "Maximum returned characters",
                        "minimum": 1000,
                        "maximum": 500000,
                        "type": "integer",
                        "description": "Maximum cleaned-content length per document.",
                        "default": 80000
                    },
                    "maxPdfPages": {
                        "title": "Maximum PDF pages",
                        "minimum": 1,
                        "maximum": 300,
                        "type": "integer",
                        "description": "Maximum number of text-based PDF pages processed per file. Scanned-image OCR is not included in this version.",
                        "default": 100
                    },
                    "maxDownloadBytes": {
                        "title": "Maximum download bytes",
                        "minimum": 102400,
                        "maximum": 52428800,
                        "type": "integer",
                        "description": "Hard per-document download limit after decompression.",
                        "default": 20971520
                    },
                    "requestTimeoutSecs": {
                        "title": "Request timeout (seconds)",
                        "minimum": 5,
                        "maximum": 120,
                        "type": "integer",
                        "description": "Total timeout for each source request.",
                        "default": 30
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```