longxing commited on
Commit
74f2a5b
·
1 Parent(s): 86418c9

暴露"最近一次回复"的诊断信息,不用翻日志就能判断内容是否发全

Browse files

背景:排查"内容不全"时,一直需要去翻 Space 的 Container logs 找
"Streaming response completed: ... chars=... truncated=..." 这一行,很费事。

改动:
- 新增模块级 LAST_RESPONSE_STATS 与 record_response_stats(),在流式(正常收尾与断流两条
分支)和非流式路径各记录一次:mode / model / chars / chunks / images / fence_open /
truncated / finish_reason / error / at。
- GET /api/session 增加 last_response 字段;管理面板 /api/status 也透出,并在「会话与
__1PSIDTS 轮换」卡片里显示为「最近一次回复」:流式/非流式 · N 字符 · 疑似被截断 · 有错误
(截断或出错时标橙色)。
- 非流式路径补齐与流式一致的截断处理:同样判定、同样追加提示、finish_reason 也置 length。

判断口径:chars 正常且 truncated=false → 服务端发全了,问题在客户端显示;
truncated=true → 上游截断,看 finish_reason 与 error。

tests/test_streaming_fidelity.py 增至 30 项(新增最近回复诊断的 4 项断言),
tests/test_admin_panel.py 增至 23 项。全套 186 项通过。

Files changed (6) hide show
  1. README.md +17 -4
  2. admin.html +18 -0
  3. admin.py +14 -1
  4. main.py +64 -2
  5. tests/test_admin_panel.py +1 -0
  6. tests/test_streaming_fidelity.py +8 -0
README.md CHANGED
@@ -106,12 +106,25 @@ gemini-webapi 自带的后台轮换任务会按 `refresh_interval`(默认 600s
106
  "recycle_count": 0,
107
  "live_1psidts_masked": "1//0g******AbCd",
108
  "cached_1psidts_masked": "1//0g******AbCd",
109
- "env_1psidts_masked": "1//0g******WxYz"
 
 
 
 
 
 
 
 
 
110
  }
111
  ```
112
 
113
  判读要点:
114
 
 
 
 
 
115
  - `rotation_count` 会随进程运行时间缓慢增长,`last_1psidts_change_age_sec` 应该
116
  大致落在 `SESSION_REFRESH_INTERVAL` 附近。**如果这个年龄一直远超轮换周期,说明
117
  轮换没在跑**,去看 `rearm_count`、`account_status` 和 `last_error`。
@@ -141,7 +154,7 @@ gemini-webapi 自带的后台轮换任务会按 `refresh_interval`(默认 600s
141
 
142
  `tests/` 下有七个不联网的校验脚本(用鸭子类型的假 `GeminiClient` 覆盖真实调用点;
143
  模型相关的用例则把真实的 `GeminiClient` 注入 `_model_registry` 后走真库逻辑;
144
- 流式相关用例真起 uvicorn + httpx),合计 181 项断言:
145
 
146
  ```bash
147
  python tests/run_all.py # 一次跑完
@@ -149,10 +162,10 @@ python tests/test_session_manager.py # 35 项:凭据优先级、轮换
149
  python tests/test_integration.py # 21 项:真实 FastAPI lifespan、流式/非流式、失效后自动重试
150
  python tests/test_models.py # 34 项:/v1/models 走账号真实探测、模型解析、对外 id 生成
151
  python tests/test_streaming.py # 11 项:测 TTFB 与分片间隔,验证真流式 + 思考下发
152
- python tests/test_streaming_fidelity.py # 26 项:长文档一字不差、断流不丢缓冲、代码块不被改写、围栏未闭合与散文中途收尾判定截断
153
  python tests/test_upstream_retry.py # 13 项:用真实装饰器复现「两次生成被拼接」,并用真实 GeminiClient 验证 current_retry 透传
154
  python tests/test_system_prompt.py # 19 项:系统指令注入、围栏语言标记、长内容自然断点、可关闭、可追加、环境变量接线
155
- python tests/test_admin_panel.py # 22 项:面板字段与 /api/status、session_manager.status() 对齐
156
  ```
157
 
158
  依赖:`fastapi` + `httpx` + `uvicorn` + `gemini-webapi`,其余按 `pyproject.toml`。
 
106
  "recycle_count": 0,
107
  "live_1psidts_masked": "1//0g******AbCd",
108
  "cached_1psidts_masked": "1//0g******AbCd",
109
+ "env_1psidts_masked": "1//0g******WxYz",
110
+ "last_response": {
111
+ "mode": "stream",
112
+ "model": "gemini-3.8-flash",
113
+ "chars": 1284,
114
+ "chunks": 42,
115
+ "truncated": false,
116
+ "finish_reason": "stop",
117
+ "error": ""
118
+ }
119
  }
120
  ```
121
 
122
  判读要点:
123
 
124
+ - **`last_response` 是最近一次回复的诊断,排查"内容不全"先看它。** `chars` 是服务端实际下发
125
+ 的字符数、`truncated` 是否疑似被截断。`chars` 正常且 `truncated=false` → 服务端发全了,
126
+ 问题在客户端显示;`truncated=true` → 上游截断了,看 `finish_reason` 和 `error`。
127
+ 管理面板的「最近一次回复」也是这个数据。
128
  - `rotation_count` 会随进程运行时间缓慢增长,`last_1psidts_change_age_sec` 应该
129
  大致落在 `SESSION_REFRESH_INTERVAL` 附近。**如果这个年龄一直远超轮换周期,说明
130
  轮换没在跑**,去看 `rearm_count`、`account_status` 和 `last_error`。
 
154
 
155
  `tests/` 下有七个不联网的校验脚本(用鸭子类型的假 `GeminiClient` 覆盖真实调用点;
156
  模型相关的用例则把真实的 `GeminiClient` 注入 `_model_registry` 后走真库逻辑;
157
+ 流式相关用例真起 uvicorn + httpx),合计 186 项断言:
158
 
159
  ```bash
160
  python tests/run_all.py # 一次跑完
 
162
  python tests/test_integration.py # 21 项:真实 FastAPI lifespan、流式/非流式、失效后自动重试
163
  python tests/test_models.py # 34 项:/v1/models 走账号真实探测、模型解析、对外 id 生成
164
  python tests/test_streaming.py # 11 项:测 TTFB 与分片间隔,验证真流式 + 思考下发
165
+ python tests/test_streaming_fidelity.py # 30 项:长文档一字不差、断流不丢缓冲、代码块不被改写、围栏未闭合与散文中途收尾判定截断、最近回复诊断
166
  python tests/test_upstream_retry.py # 13 项:用真实装饰器复现「两次生成被拼接」,并用真实 GeminiClient 验证 current_retry 透传
167
  python tests/test_system_prompt.py # 19 项:系统指令注入、围栏语言标记、长内容自然断点、可关闭、可追加、环境变量接线
168
+ python tests/test_admin_panel.py # 23 项:面板字段与 /api/status、session_manager.status() 对齐
169
  ```
170
 
171
  依赖:`fastapi` + `httpx` + `uvicorn` + `gemini-webapi`,其余按 `pyproject.toml`。
admin.html CHANGED
@@ -491,6 +491,10 @@
491
  <span class="config-label">重建 / 重拉</span>
492
  <span class="config-value" id="sessionRecovery">--</span>
493
  </div>
 
 
 
 
494
  </div>
495
  </div>
496
 
@@ -757,6 +761,20 @@
757
  document.getElementById('sessionRecovery').textContent =
758
  '重建 ' + (session.recycle_count || 0) + ' 次 / 重拉 ' + (session.rearm_count || 0) + ' 次';
759
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
760
  const sessionErrorRow = document.getElementById('sessionErrorRow');
761
  if (session.last_error) {
762
  sessionErrorRow.style.display = '';
 
491
  <span class="config-label">重建 / 重拉</span>
492
  <span class="config-value" id="sessionRecovery">--</span>
493
  </div>
494
+ <div class="config-item">
495
+ <span class="config-label">最近一次回复</span>
496
+ <span class="config-value" id="lastResponse">--</span>
497
+ </div>
498
  </div>
499
  </div>
500
 
 
761
  document.getElementById('sessionRecovery').textContent =
762
  '重建 ' + (session.recycle_count || 0) + ' 次 / 重拉 ' + (session.rearm_count || 0) + ' 次';
763
 
764
+ // 最近一次回复的诊断:用来判断"内容不全"是服务端没发全还是客户端没显示全
765
+ const last = data.last_response || {};
766
+ const lastEl = document.getElementById('lastResponse');
767
+ if (last.chars === undefined) {
768
+ lastEl.textContent = '--';
769
+ lastEl.style.color = '';
770
+ } else {
771
+ const parts = [last.mode === 'stream' ? '流式' : '非流式', last.chars + ' 字符'];
772
+ if (last.truncated) parts.push('疑似被截断');
773
+ if (last.error) parts.push('有错误');
774
+ lastEl.textContent = parts.join(' · ');
775
+ lastEl.style.color = last.truncated || last.error ? '#ff9800' : '#4caf50';
776
+ }
777
+
778
  const sessionErrorRow = document.getElementById('sessionErrorRow');
779
  if (session.last_error) {
780
  sessionErrorRow.style.display = '';
admin.py CHANGED
@@ -205,7 +205,18 @@ async def verify_admin_token(request: Request):
205
  @router.get("/api/status")
206
  async def get_status(token: str = Depends(verify_admin_token)):
207
  """获取服务状态"""
208
- from main import API_KEY, AUTO_DELETE_CHAT, ENABLE_THINKING, HOST, PORT, SECURE_1PSID, SECURE_1PSIDTS, TEMPORARY_CHAT, session_manager
 
 
 
 
 
 
 
 
 
 
 
209
 
210
  # 检查 cookie 是否有效(简单检查是否存在)
211
  cookie_valid = bool(SECURE_1PSID and SECURE_1PSIDTS)
@@ -248,6 +259,8 @@ async def get_status(token: str = Depends(verify_admin_token)):
248
  "rotation_task_alive": session.get("rotation_task_alive", False),
249
  "supervisor_alive": session.get("supervisor_alive", False),
250
  "account_status": session.get("account_status", "unknown"),
 
 
251
  }
252
 
253
 
 
205
  @router.get("/api/status")
206
  async def get_status(token: str = Depends(verify_admin_token)):
207
  """获取服务状态"""
208
+ from main import (
209
+ API_KEY,
210
+ AUTO_DELETE_CHAT,
211
+ ENABLE_THINKING,
212
+ HOST,
213
+ LAST_RESPONSE_STATS,
214
+ PORT,
215
+ SECURE_1PSID,
216
+ SECURE_1PSIDTS,
217
+ TEMPORARY_CHAT,
218
+ session_manager,
219
+ )
220
 
221
  # 检查 cookie 是否有效(简单检查是否存在)
222
  cookie_valid = bool(SECURE_1PSID and SECURE_1PSIDTS)
 
259
  "rotation_task_alive": session.get("rotation_task_alive", False),
260
  "supervisor_alive": session.get("supervisor_alive", False),
261
  "account_status": session.get("account_status", "unknown"),
262
+ # 最近一次回复的诊断:判断"内容不全"是服务端没发全还是客户端没显示全
263
+ "last_response": dict(LAST_RESPONSE_STATS),
264
  }
265
 
266
 
main.py CHANGED
@@ -999,6 +999,17 @@ def _looks_truncated(text: str, fence_open: bool) -> bool:
999
  return stripped[-1] not in _TERMINAL_CHARS
1000
 
1001
 
 
 
 
 
 
 
 
 
 
 
 
1002
  def get_image_signature(url: str) -> str:
1003
  """
1004
  Generate a HMAC-SHA256 signature for the image URL using the persistent SIGNATURE_SECRET.
@@ -1245,6 +1256,17 @@ async def create_chat_completion(
1245
  fence_open,
1246
  suspected,
1247
  )
 
 
 
 
 
 
 
 
 
 
 
1248
  if last_metadata and len(last_metadata) > 0 and not AUTO_DELETE_CHAT:
1249
  asyncio.create_task(background_verify_chat_persistence(gemini_client, last_metadata[0], "stream"))
1250
  except Exception as e:
@@ -1284,6 +1306,17 @@ async def create_chat_completion(
1284
  )
1285
  yield make_chunk({}, finish_reason="length")
1286
  yield "data: [DONE]\n\n"
 
 
 
 
 
 
 
 
 
 
 
1287
  finally:
1288
  # Create background task to delete the chat if AUTO_DELETE_CHAT is enabled
1289
  if AUTO_DELETE_CHAT and captured_cid:
@@ -1335,6 +1368,29 @@ async def create_chat_completion(
1335
  logger.warning("Empty response received from Gemini")
1336
  reply_text = "Server returned an empty response. Please check that Gemini API credentials are valid."
1337
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1338
  result = {
1339
  "id": completion_id,
1340
  "object": "chat.completion",
@@ -1344,7 +1400,7 @@ async def create_chat_completion(
1344
  {
1345
  "index": 0,
1346
  "message": {"role": "assistant", "content": reply_text},
1347
- "finish_reason": "stop",
1348
  }
1349
  ],
1350
  "usage": {
@@ -1494,8 +1550,14 @@ async def session_status():
1494
  2. 「后台是否仍在自动轮换 __Secure-1PSIDTS」—— 看 ``rotation_count``、
1495
  ``last_1psidts_change_age_sec``、``rotation_task_alive``、``supervisor_alive``、
1496
  ``rearm_count``。
 
 
 
1497
  """
1498
- return session_manager.status()
 
 
 
1499
 
1500
 
1501
  if __name__ == "__main__":
 
999
  return stripped[-1] not in _TERMINAL_CHARS
1000
 
1001
 
1002
+ # 最近一次回复的诊断信息:不用翻日志就能回答"服务端到底发了多少字符、是不是被截断了"。
1003
+ LAST_RESPONSE_STATS: Dict[str, object] = {}
1004
+
1005
+
1006
+ def record_response_stats(**kwargs) -> None:
1007
+ """记录最近一次回复的诊断信息(供 /api/session 与管理面板展示)。"""
1008
+ LAST_RESPONSE_STATS.clear()
1009
+ LAST_RESPONSE_STATS.update(kwargs)
1010
+ LAST_RESPONSE_STATS["at"] = int(time.time())
1011
+
1012
+
1013
  def get_image_signature(url: str) -> str:
1014
  """
1015
  Generate a HMAC-SHA256 signature for the image URL using the persistent SIGNATURE_SECRET.
 
1256
  fence_open,
1257
  suspected,
1258
  )
1259
+ record_response_stats(
1260
+ mode="stream",
1261
+ model=request.model,
1262
+ chars=delivered_chars,
1263
+ chunks=chunk_count,
1264
+ images=yielded_images,
1265
+ fence_open=fence_open,
1266
+ truncated=suspected,
1267
+ finish_reason="length" if suspected else "stop",
1268
+ error="",
1269
+ )
1270
  if last_metadata and len(last_metadata) > 0 and not AUTO_DELETE_CHAT:
1271
  asyncio.create_task(background_verify_chat_persistence(gemini_client, last_metadata[0], "stream"))
1272
  except Exception as e:
 
1306
  )
1307
  yield make_chunk({}, finish_reason="length")
1308
  yield "data: [DONE]\n\n"
1309
+ record_response_stats(
1310
+ mode="stream",
1311
+ model=request.model,
1312
+ chars=delivered_chars,
1313
+ chunks=chunk_count,
1314
+ images=yielded_images,
1315
+ fence_open=bool(fence_state.get("in_fence")),
1316
+ truncated=True,
1317
+ finish_reason="length",
1318
+ error=detail,
1319
+ )
1320
  finally:
1321
  # Create background task to delete the chat if AUTO_DELETE_CHAT is enabled
1322
  if AUTO_DELETE_CHAT and captured_cid:
 
1368
  logger.warning("Empty response received from Gemini")
1369
  reply_text = "Server returned an empty response. Please check that Gemini API credentials are valid."
1370
 
1371
+ # 与流式路径保持一致:非流式也要能识别"上游提前收尾"
1372
+ non_stream_truncated = _looks_truncated(reply_text, False)
1373
+ body_chars = len(reply_text)
1374
+ if non_stream_truncated:
1375
+ logger.warning(
1376
+ "Non-streaming response looks truncated (chars=%s). Upstream may have cut it off.",
1377
+ body_chars,
1378
+ )
1379
+ if TRUNCATION_NOTICE:
1380
+ reply_text += "\n\n[回复似乎在句子中间结束,可能被上游截断。可以回复「继续」让模型接着写]"
1381
+
1382
+ record_response_stats(
1383
+ mode="non-stream",
1384
+ model=request.model,
1385
+ chars=body_chars,
1386
+ chunks=1,
1387
+ images=0,
1388
+ fence_open=False,
1389
+ truncated=non_stream_truncated,
1390
+ finish_reason="length" if non_stream_truncated else "stop",
1391
+ error="",
1392
+ )
1393
+
1394
  result = {
1395
  "id": completion_id,
1396
  "object": "chat.completion",
 
1400
  {
1401
  "index": 0,
1402
  "message": {"role": "assistant", "content": reply_text},
1403
+ "finish_reason": "length" if non_stream_truncated else "stop",
1404
  }
1405
  ],
1406
  "usage": {
 
1550
  2. 「后台是否仍在自动轮换 __Secure-1PSIDTS」—— 看 ``rotation_count``、
1551
  ``last_1psidts_change_age_sec``、``rotation_task_alive``、``supervisor_alive``、
1552
  ``rearm_count``。
1553
+
1554
+ 另外返回 ``last_response``:最近一次回复的诊断(下发的字符数、是否疑似被截断、
1555
+ finish_reason 等),用来判断"内容不全"到底是服务端没发全还是客户端没显示全。
1556
  """
1557
+ return {
1558
+ **session_manager.status(),
1559
+ "last_response": dict(LAST_RESPONSE_STATS),
1560
+ }
1561
 
1562
 
1563
  if __name__ == "__main__":
tests/test_admin_panel.py CHANGED
@@ -50,6 +50,7 @@ def main():
50
  "sessionErrorRow",
51
  "sessionError",
52
  "sessionRecovery",
 
53
  ):
54
  check(f"1b 面板已接线 {element_id}", element_id in declared and element_id in referenced)
55
 
 
50
  "sessionErrorRow",
51
  "sessionError",
52
  "sessionRecovery",
53
+ "lastResponse",
54
  ):
55
  check(f"1b 面板已接线 {element_id}", element_id in declared and element_id in referenced)
56
 
tests/test_streaming_fidelity.py CHANGED
@@ -355,6 +355,14 @@ def main():
355
  check("25 短回复不误报", "句子中间结束" not in content_short, repr(content_short))
356
  check("26 且 finish_reason 为 stop", _finish_reason(raw_short) == "stop", str(_finish_reason(raw_short)))
357
 
 
 
 
 
 
 
 
 
358
  server.should_exit = True
359
  time.sleep(0.5)
360
 
 
355
  check("25 短回复不误报", "句子中间结束" not in content_short, repr(content_short))
356
  check("26 且 finish_reason 为 stop", _finish_reason(raw_short) == "stop", str(_finish_reason(raw_short)))
357
 
358
+ # ---------------------------------------------------------------- 最近一次回复的诊断
359
+ stats = dict(app_main.LAST_RESPONSE_STATS)
360
+ print(f" LAST_RESPONSE_STATS = {stats}")
361
+ check("27 记录了最近一次回复的诊断", stats.get("mode") == "stream" and stats.get("chars", 0) > 0, str(stats))
362
+ check("28 记录了是否疑似截断", stats.get("truncated") is False, str(stats.get("truncated")))
363
+ check("29 记录了 finish_reason", stats.get("finish_reason") == "stop", str(stats.get("finish_reason")))
364
+ check("30 记录了模型名与时间戳", stats.get("model") == "gemini-3.8-flash" and isinstance(stats.get("at"), int), str(stats))
365
+
366
  server.should_exit = True
367
  time.sleep(0.5)
368