Spaces:
Paused
Paused
Peng Ding commited on
Commit ·
9e2871e
1
Parent(s): 2424eab
docs: update README_zh.md for v0.4.0
Browse files- README_zh.md +180 -52
README_zh.md
CHANGED
|
@@ -4,113 +4,241 @@
|
|
| 4 |
[](https://github.com/Oaklight/veilrender/releases/latest)
|
| 5 |
[](https://github.com/Oaklight/veilrender/actions/workflows/ci.yml)
|
| 6 |
[](https://hub.docker.com/r/oaklight/veilrender)
|
| 7 |
-
[](https://hub.docker.com/r/oaklight/veilrender)
|
| 8 |
[](https://opensource.org/licenses/MIT)
|
| 9 |
-
[](https://oaklight-veilrender-public.hf.space)
|
| 10 |
|
| 11 |
**中文** | [English](README_en.md)
|
| 12 |
|
| 13 |
-
无头浏览器渲染 API——
|
| 14 |
|
| 15 |
-
|
| 16 |
|
| 17 |
-
|
| 18 |
-
curl -X POST https://oaklight-veilrender-public.hf.space/render \
|
| 19 |
-
-H "Content-Type: application/json" \
|
| 20 |
-
-d '{"url": "https://example.com"}'
|
| 21 |
-
```
|
| 22 |
|
| 23 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
## 快速开始
|
| 26 |
|
| 27 |
-
### Docker
|
| 28 |
|
| 29 |
```bash
|
| 30 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
```
|
| 32 |
|
| 33 |
-
###
|
| 34 |
|
| 35 |
```bash
|
| 36 |
-
pip install
|
| 37 |
-
|
| 38 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
```
|
| 40 |
|
|
|
|
|
|
|
| 41 |
## API
|
| 42 |
|
| 43 |
### GET /health
|
| 44 |
|
| 45 |
-
|
|
|
|
|
|
|
| 46 |
|
| 47 |
### POST /render
|
| 48 |
|
| 49 |
-
渲染
|
| 50 |
|
| 51 |
```bash
|
| 52 |
curl -X POST http://localhost:7860/render \
|
| 53 |
-H "Authorization: Bearer your-secret" \
|
| 54 |
-H "Content-Type: application/json" \
|
| 55 |
-
-d '{"url": "https://example.com"}'
|
| 56 |
```
|
| 57 |
|
| 58 |
-
|
| 59 |
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
"readability": "..."
|
| 66 |
-
},
|
| 67 |
-
"metadata": {
|
| 68 |
-
"title": "Example Domain",
|
| 69 |
-
"url": "https://example.com",
|
| 70 |
-
"status_code": 200
|
| 71 |
-
},
|
| 72 |
-
"links": [{"url": "https://www.iana.org/domains/example", "text": "More information..."}]
|
| 73 |
-
}
|
| 74 |
-
```
|
| 75 |
|
| 76 |
### POST /screenshot
|
| 77 |
|
| 78 |
-
|
| 79 |
|
| 80 |
```bash
|
| 81 |
curl -X POST http://localhost:7860/screenshot \
|
| 82 |
-H "Authorization: Bearer your-secret" \
|
| 83 |
-H "Content-Type: application/json" \
|
| 84 |
-
-d '{"url": "https://example.com"}'
|
| 85 |
-
-o screenshot.png
|
| 86 |
```
|
| 87 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 88 |
## 配置
|
| 89 |
|
| 90 |
-
所有设置通过 `VEILRENDER_`
|
|
|
|
|
|
|
| 91 |
|
| 92 |
| 变量 | 默认值 | 说明 |
|
| 93 |
|------|--------|------|
|
| 94 |
-
| `VEILRENDER_API_TOKEN` | *
|
| 95 |
| `VEILRENDER_PORT` | `7860` | 服务端口 |
|
| 96 |
-
| `VEILRENDER_HOST` | `0.0.0.0` |
|
| 97 |
-
| `VEILRENDER_TIMEOUT` | `30000` |
|
| 98 |
| `VEILRENDER_VIEWPORT_WIDTH` | `1280` | 浏览器视口宽度 |
|
| 99 |
| `VEILRENDER_VIEWPORT_HEIGHT` | `720` | 浏览器视口高度 |
|
| 100 |
-
| `VEILRENDER_MAX_CONCURRENT` | `
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 101 |
|
| 102 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 103 |
|
| 104 |
-
|
| 105 |
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
|
| 112 |
|
| 113 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 114 |
|
| 115 |
## 许可证
|
| 116 |
|
|
|
|
| 4 |
[](https://github.com/Oaklight/veilrender/releases/latest)
|
| 5 |
[](https://github.com/Oaklight/veilrender/actions/workflows/ci.yml)
|
| 6 |
[](https://hub.docker.com/r/oaklight/veilrender)
|
|
|
|
| 7 |
[](https://opensource.org/licenses/MIT)
|
|
|
|
| 8 |
|
| 9 |
**中文** | [English](README_en.md)
|
| 10 |
|
| 11 |
+
具备反检测能力的无头浏览器渲染 API —— 支持通过 Docker 或 pip 自托管部署。
|
| 12 |
|
| 13 |
+
VeilRender 接受一个 URL,使用反检测 Chromium 浏览器返回完整渲染后的页面内容(HTML、Markdown、readability 提取的正文)。专为 fetch 工具在遇到 JavaScript 渲染页面或反爬页面时失败的场景而设计。
|
| 14 |
|
| 15 |
+
## 特性
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
+
- **反检测渲染** —— [CloakBrowser](https://github.com/CloakHQ/CloakBrowser)(71 项 C++ 指纹补丁)+ [Patchright](https://github.com/Kaliiiiiiiiii-Vinyzu/patchright-python)(Playwright 反检测分支)
|
| 18 |
+
- **多种输出格式** —— 原始 HTML、Markdown、readability 提取的正文
|
| 19 |
+
- **截图功能** —— 全页或视口 PNG 截图
|
| 20 |
+
- **水平扩展** —— 网关 + 远程浏览器工作节点池,支持健康检查和自动重连
|
| 21 |
+
- **混合浏览器后端** —— Chromium(CDP)和 Firefox/Camoufox(Playwright 协议)可在同一池中共存
|
| 22 |
+
- **Prometheus 指标** —— `/metrics` 端点提供延迟百分位、每工作节点状态指标
|
| 23 |
+
- **仪表盘** —— `/` 路径展示实时统计,支持国际化(en/zh),含 SVG 容量仪表
|
| 24 |
+
- **广告/追踪器拦截** —— 来自 [StevenBlack/hosts](https://github.com/StevenBlack/hosts) 的 82k 域名黑名单
|
| 25 |
+
- **渲染缓存** —— L1 内存缓存 + L2 S3 兼容存储(R2、Oracle、AWS)
|
| 26 |
+
- **CDP 代理** —— 通过 `/cdp` 直接 WebSocket 访问浏览器
|
| 27 |
+
- **零外部依赖**(除 `patchright` 外)—— HTTP 服务器、S3 客户端、HTML 解析均为内置模块
|
| 28 |
|
| 29 |
## 快速开始
|
| 30 |
|
| 31 |
+
### Docker(推荐)
|
| 32 |
|
| 33 |
```bash
|
| 34 |
+
# 单实例模式,内置 CloakBrowser
|
| 35 |
+
docker run -p 7860:7860 -e VEILRENDER_API_TOKEN=your-secret oaklight/veilrender:latest
|
| 36 |
+
|
| 37 |
+
# 仅网关模式(不含浏览器,336MB)
|
| 38 |
+
docker run -p 7860:7860 oaklight/veilrender:gateway
|
| 39 |
```
|
| 40 |
|
| 41 |
+
### pip
|
| 42 |
|
| 43 |
```bash
|
| 44 |
+
pip install veilrender
|
| 45 |
+
veilrender # 在 :7860 启动,首次运行时自动下载 CloakBrowser 二进制文件
|
| 46 |
+
```
|
| 47 |
+
|
| 48 |
+
### Docker Compose —— 工作节点池
|
| 49 |
+
|
| 50 |
+
```yaml
|
| 51 |
+
# docker-compose.yaml
|
| 52 |
+
services:
|
| 53 |
+
veilrender:
|
| 54 |
+
image: oaklight/veilrender:gateway
|
| 55 |
+
ports: ["7860:7860"]
|
| 56 |
+
environment:
|
| 57 |
+
- VEILRENDER_WORKERS=cdp://browser-worker:9222
|
| 58 |
+
browser-worker:
|
| 59 |
+
image: cloakhq/cloakbrowser
|
| 60 |
+
command: ["cloakserve"]
|
| 61 |
```
|
| 62 |
|
| 63 |
+
更多 compose 示例(包括 Chromium + Camoufox 混合池)请参阅 `deploy/` 目录。
|
| 64 |
+
|
| 65 |
## API
|
| 66 |
|
| 67 |
### GET /health
|
| 68 |
|
| 69 |
+
```json
|
| 70 |
+
{"status": "ok"}
|
| 71 |
+
```
|
| 72 |
|
| 73 |
### POST /render
|
| 74 |
|
| 75 |
+
渲染 URL 并返回页面内容。
|
| 76 |
|
| 77 |
```bash
|
| 78 |
curl -X POST http://localhost:7860/render \
|
| 79 |
-H "Authorization: Bearer your-secret" \
|
| 80 |
-H "Content-Type: application/json" \
|
| 81 |
+
-d '{"url": "https://example.com", "formats": ["html", "readability", "markdown"]}'
|
| 82 |
```
|
| 83 |
|
| 84 |
+
请求体:
|
| 85 |
|
| 86 |
+
| 字段 | 类型 | 默认值 | 说明 |
|
| 87 |
+
|------|------|--------|------|
|
| 88 |
+
| `url` | string | *(必填)* | 要渲染的 URL |
|
| 89 |
+
| `formats` | string[] | `["html"]` | 输出格式:`html`、`readability`、`markdown` |
|
| 90 |
+
| `wait_until` | string | `"load"` | Playwright 等待策略:`load`、`domcontentloaded`、`networkidle` |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 91 |
|
| 92 |
### POST /screenshot
|
| 93 |
|
| 94 |
+
捕获 PNG 截图。
|
| 95 |
|
| 96 |
```bash
|
| 97 |
curl -X POST http://localhost:7860/screenshot \
|
| 98 |
-H "Authorization: Bearer your-secret" \
|
| 99 |
-H "Content-Type: application/json" \
|
| 100 |
+
-d '{"url": "https://example.com"}' -o screenshot.png
|
|
|
|
| 101 |
```
|
| 102 |
|
| 103 |
+
### GET /metrics
|
| 104 |
+
|
| 105 |
+
Prometheus 格式的指标输出,包括运行时间、请求计数器、缓存统计、延迟摘要(p50/p95)以及每工作节点的健康状态指标。
|
| 106 |
+
|
| 107 |
+
### GET /stats
|
| 108 |
+
|
| 109 |
+
仪表盘实时数据的 JSON API(由仪表盘 UI 轮询获取)。
|
| 110 |
+
|
| 111 |
+
### WS /cdp
|
| 112 |
+
|
| 113 |
+
直接连接浏览器的 CDP WebSocket 代理。支持 `?worker=N` 参数指定池中的特定工作节点。
|
| 114 |
+
|
| 115 |
## 配置
|
| 116 |
|
| 117 |
+
所有设置通过 `VEILRENDER_*` 环境变量配置:
|
| 118 |
+
|
| 119 |
+
### 核心配置
|
| 120 |
|
| 121 |
| 变量 | 默认值 | 说明 |
|
| 122 |
|------|--------|------|
|
| 123 |
+
| `VEILRENDER_API_TOKEN` | *(无)* | Bearer 认证令牌。未设置时禁用认证 |
|
| 124 |
| `VEILRENDER_PORT` | `7860` | 服务端口 |
|
| 125 |
+
| `VEILRENDER_HOST` | `0.0.0.0` | 绑定地址 |
|
| 126 |
+
| `VEILRENDER_TIMEOUT` | `30000` | 页面加载超时时间(毫秒) |
|
| 127 |
| `VEILRENDER_VIEWPORT_WIDTH` | `1280` | 浏览器视口宽度 |
|
| 128 |
| `VEILRENDER_VIEWPORT_HEIGHT` | `720` | 浏览器视口高度 |
|
| 129 |
+
| `VEILRENDER_MAX_CONCURRENT` | `5` | 最大并发页面数(单实例模式) |
|
| 130 |
+
|
| 131 |
+
### 工作节点池
|
| 132 |
+
|
| 133 |
+
| 变量 | 默认值 | 说明 |
|
| 134 |
+
|------|--------|------|
|
| 135 |
+
| `VEILRENDER_WORKERS` | *(无)* | 逗号分隔的工作节点端点(启用池模式) |
|
| 136 |
+
| `VEILRENDER_WORKER_MAX_CONCURRENT` | `5` | 每工作节点的页面并发数 |
|
| 137 |
+
| `VEILRENDER_WORKER_HEALTH_INTERVAL` | `10` | 健康检查间隔(秒) |
|
| 138 |
+
|
| 139 |
+
工作节点端点格式:
|
| 140 |
+
- `cdp://host:9222` 或 `http://host:9222` —— Chromium CDP 工作节点
|
| 141 |
+
- `playwright://host:1234/ws-path` —— Firefox/Camoufox Playwright 工作节点
|
| 142 |
+
- `playwrights://host:1234/ws-path` —— TLS Playwright 工作节点
|
| 143 |
+
|
| 144 |
+
### 浏览器二进制文件
|
| 145 |
+
|
| 146 |
+
| 变量 | 默认值 | 说明 |
|
| 147 |
+
|------|--------|------|
|
| 148 |
+
| `CLOAKBROWSER_BINARY` | *(自动)* | 自定义浏览器二进制文件路径 |
|
| 149 |
+
| `CLOAKBROWSER_MIRROR` | *(无)* | GitHub 镜像 URL,适用于中国大陆网络环境(如 `https://ghfast.top`) |
|
| 150 |
+
|
| 151 |
+
二进制文件检测顺序:环境变量 → `~/.cloakbrowser/*/chrome` → 从 GitHub Releases 自动下载。
|
| 152 |
+
|
| 153 |
+
### 缓存与存储
|
| 154 |
|
| 155 |
+
| 变量 | 默认值 | 说明 |
|
| 156 |
+
|------|--------|------|
|
| 157 |
+
| `VEILRENDER_CACHE_ENABLED` | `false` | 启用渲染缓存 |
|
| 158 |
+
| `VEILRENDER_CACHE_TTL` | `86400` | 缓存 TTL(秒) |
|
| 159 |
+
| `VEILRENDER_RESOURCE_FILTER` | `true` | 渲染时拦截广告/追踪器 |
|
| 160 |
+
| `VEILRENDER_S3_ENDPOINT` | *(无)* | L2 缓存的 S3 端点 |
|
| 161 |
+
| `VEILRENDER_S3_ACCESS_KEY` | *(无)* | S3 访问密钥 |
|
| 162 |
+
| `VEILRENDER_S3_SECRET_KEY` | *(无)* | S3 密钥 |
|
| 163 |
+
|
| 164 |
+
## 部署模式
|
| 165 |
+
|
| 166 |
+
### 单实例模式(完整镜像,1.07GB)
|
| 167 |
+
|
| 168 |
+
内置 CloakBrowser —— 最简单的部署方式,无外部依赖。
|
| 169 |
+
|
| 170 |
+
```bash
|
| 171 |
+
docker run -p 7860:7860 -e VEILRENDER_API_TOKEN=secret oaklight/veilrender:latest
|
| 172 |
+
```
|
| 173 |
+
|
| 174 |
+
### 网关 + 工作节点池(网关镜像,336MB)
|
| 175 |
+
|
| 176 |
+
浏览器容器独立扩展。网关通过最少连接数策略路由请求。
|
| 177 |
+
|
| 178 |
+
```yaml
|
| 179 |
+
services:
|
| 180 |
+
veilrender:
|
| 181 |
+
image: oaklight/veilrender:gateway
|
| 182 |
+
environment:
|
| 183 |
+
- VEILRENDER_WORKERS=cdp://worker1:9222,cdp://worker2:9222
|
| 184 |
+
|
| 185 |
+
worker1:
|
| 186 |
+
image: cloakhq/cloakbrowser
|
| 187 |
+
command: ["cloakserve"]
|
| 188 |
+
|
| 189 |
+
worker2:
|
| 190 |
+
image: cloakhq/cloakbrowser
|
| 191 |
+
command: ["cloakserve"]
|
| 192 |
+
```
|
| 193 |
+
|
| 194 |
+
### 混合池(Chromium + Camoufox)
|
| 195 |
|
| 196 |
+
双引擎反检测:CloakBrowser 应对 Chromium 指纹检测站点,Camoufox 应对 Firefox 指纹检测站点。
|
| 197 |
|
| 198 |
+
```yaml
|
| 199 |
+
services:
|
| 200 |
+
veilrender:
|
| 201 |
+
image: oaklight/veilrender:gateway
|
| 202 |
+
environment:
|
| 203 |
+
- VEILRENDER_WORKERS=cdp://chromium:9222,playwright://camoufox:1234/ws
|
| 204 |
|
| 205 |
+
chromium:
|
| 206 |
+
image: cloakhq/cloakbrowser
|
| 207 |
+
command: ["cloakserve"]
|
| 208 |
+
|
| 209 |
+
camoufox:
|
| 210 |
+
build: deploy/Dockerfile.camoufox
|
| 211 |
+
```
|
| 212 |
+
|
| 213 |
+
## Docker 镜像
|
| 214 |
+
|
| 215 |
+
| 标签 | 大小 | 内容 |
|
| 216 |
+
|------|------|------|
|
| 217 |
+
| `latest` / `0.4.0` | 1.07GB | 完整版 —— Patchright + CloakBrowser |
|
| 218 |
+
| `gateway` | 336MB | 仅网关 —— 不含浏览器二进制文件 |
|
| 219 |
+
|
| 220 |
+
本地构建:
|
| 221 |
+
```bash
|
| 222 |
+
make build # 完整镜像
|
| 223 |
+
make build-gateway # 网关镜像
|
| 224 |
+
```
|
| 225 |
+
|
| 226 |
+
## 开发
|
| 227 |
+
|
| 228 |
+
```bash
|
| 229 |
+
# 环境搭建
|
| 230 |
+
pip install -e ".[dev]"
|
| 231 |
+
|
| 232 |
+
# 本地运行
|
| 233 |
+
make dev # 在 :7860 启动
|
| 234 |
+
|
| 235 |
+
# 代码检查与类型检查
|
| 236 |
+
make lint # ruff check --fix && ruff format
|
| 237 |
+
make typecheck # ty check
|
| 238 |
+
|
| 239 |
+
# 开发环境部署
|
| 240 |
+
make deploy-dev SSH_TARGET=oaklight.buttercup
|
| 241 |
+
```
|
| 242 |
|
| 243 |
## 许可证
|
| 244 |
|