TypeScript commited on
Commit
7e514a7
·
verified ·
1 Parent(s): 08eba34

Ensure whitespace boundaries around inline KaTeX delimiters and fix CJK CommonMark emphasis

Browse files
Files changed (2) hide show
  1. README.md +27 -27
  2. README_zh.md +28 -28
README.md CHANGED
@@ -65,16 +65,16 @@ FastBrain-9B uses a 32-layer hybrid sequence backbone (`9.30B` parameters, `bflo
65
 
66
  ### 2.2 Joint Schema Decision Head and Vectorized CUDA Kernel
67
 
68
- The `JointSchemaHead` (`116M` parameters, stored in `joint_head.safetensors`) pools token representations bounded by `<|question_start|>`/`<|question_end|>` and `<|option_start|>`/`<|option_end|>` markers from the final backbone hidden states \\(H \in \mathbb{R}^{T \times d}\\), applies cross-attention over question-option pairs, and projects directly to scalar logits for three decision primitives:
69
- 1. **`choice`** (\\(K\\)-way categorical distribution): \\(\hat{p} = \operatorname{softmax}(z / \tau) \in \Delta^{K-1}\\).
70
- 2. **`noul`** (binary verification): \\(\hat{p}(\text{true}) = \sigma(z_{\text{true}} - z_{\text{false}})\\).
71
- 3. **`score`** (ordinal rubric over levels \\(v_1 < \dots < v_K\\)): categorical probabilities \\(\hat{p} \in \Delta^{K-1}\\) together with expected value \\(\mathbb{E}[\hat{y}] = \sum_{k=1}^K \hat{p}_k v_k\\).
72
 
73
- In `joint_schema_model.py`, the reference Python span-extraction loop and padded SDPA mask are replaced on CUDA devices by `_forward_fast_cuda`: span means are computed in \\(\mathcal{O}(1)\\) kernel launches via prefix sums (`torch.cumsum`) over the schema suffix, and cross-attention is executed via packed variable-length FlashAttention (`torch.ops.aten._flash_attention_forward` parameterized by `cu_seqlens`). On NVIDIA RTX PRO 6000 Blackwell, this reduces isolated decision-head latency from `3.27 ms` to `1.72 ms` (**1.90× speedup**) with exact numerical equivalence (`max_logit_diff = 0.00000`).
74
 
75
  ### 2.3 Ordinal-KL Distillation and Schema-Topology Calibration
76
 
77
- Standard cross-entropy or flat KL divergence treats ordinal rubric levels as unordered classes, penalizing adjacent-level errors identically to extreme errors. FastBrain-9B is trained on `TypeScript/FastBrain-Instruct` under a composite loss combining soft-target KL divergence, discrete ranked probability score (squared \\(L_2\\) distance between predicted and target cumulative distribution functions \\(\hat{F}_k = \sum_{j \le k} \hat{p}_j\\)), and expected-score mean squared error on `score` primitives:
78
 
79
  $$
80
  \mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{ord}} \sum_{k=1}^{K-1} \bigl(\hat{F}_{\theta,k} - F_k\bigr)^2 + \mathbb{I}[\text{type}=\text{score}] \cdot \lambda_{\mathrm{mse}} \bigl(\mathbb{E}[\hat{y}_\theta] - \mathbb{E}[y]\bigr)^2.
@@ -101,19 +101,19 @@ At inference time, `joint_schema_model.py` optionally applies compile-time schem
101
 
102
  ### 3.1 Agent Tool-Call Guardrail Evaluation (`safeagents-judge-bench`)
103
 
104
- [`harness-hub/safeagents-judge-bench`](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) evaluates whether a System-1 decision model can audit suspicious tool calls flagged by a static scanner (`atif-scan`) during real coding-agent executions on Terminal-Bench 2.1, OpenThoughts-TBLite, and Terminal-Bench 4. Version 2 (`v2`) comprises \\(N = 661\\) tool calls across 55 tasks (`536` malicious `block` calls spanning `82` misbehaving trajectories and `125` legitimate `allow` calls). Each call is issued as a single zero-shot System-1 request containing one binary `noul` question (`block`) and one 7-way `choice` question (`legit`, `eval_gaming`, `harness_introspection`, `credential_misuse`, `exfiltration`, `sandbox_escape`, `other_overreach`). All v2 results in Table 2 were executed independently on NVIDIA A100-SXM4-80GB via Hugging Face Jobs ([Merged PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1)).
105
 
106
- On the 7-way category classification track, FastBrain-9B achieves `0.751` macro-F1 and `0.858` accuracy (`567/661`), outperforming the 27.4B `Cloudflare/clef` model (`0.563` macro-F1, `0.737` accuracy) by `+18.8` percentage points in macro-F1 and `+12.1` points in accuracy, and improving upon its 9.4B base model `Cloudflare/clef-flash` (`0.462` macro-F1, `0.576` accuracy) by `+28.9` and `+28.2` points, respectively. Per-category F1 reaches `0.945` on `harness_introspection` (\\(n=272\\), recall `0.978`), `0.828` on `eval_gaming` (\\(n=245\\), recall `0.853`), and `0.761` on `legit` (\\(n=125\\), precision `0.975`).
107
 
108
- On the binary guardrail track, FastBrain-9B attains `0.985` AUROC and `0.996` malicious-call recall (`534/536`), intercepting all `82/82` unsafe trajectories (`median_bad_calls_before_catch = 0.0`), with `1.000` recall on `harness_introspection` (`272/272`), `credential_misuse` (`15/15`), and `sandbox_escape` (`4/4`). Under the uncalibrated default decision rule \\(P(\text{block}) \ge 0.50\\), balanced accuracy is `0.814` (false-block rate `0.368` across the 125 `allow` calls, compared with `0.808` balanced accuracy and `0.384` false-block rate for 27.4B `Cloudflare/clef`). Because rank-ordering discrimination is preserved (`0.985` AUROC), setting the operating threshold to \\(P(\text{block}) \ge 0.961\\) yields `0.938` balanced accuracy at a `0.056` (`7/125`) false-block rate while maintaining `82/82` trajectory interception. Table 2 also includes the `v1` release (\\(N = 562\\) calls, `78` trajectories), where FastBrain-9B ranked 1st of 9 on both tracks.
109
 
110
- **Table 2: Zero-shot evaluation on `harness-hub/safeagents-judge-bench` (`v2`, \\(N = 661\\) calls, 82 unsafe trajectories; and `v1` reference, \\(N = 562\\)).**
111
 
112
  | Model | Params | Cat. Macro-F1 ↑ | Cat. Acc ↑ | AUROC ↑ | Bal. Acc ↑ | Recall ↑ | False Block ↓ | Traj. Caught ↑ | p50 Latency (Hardware) |
113
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
114
- | **TypeScript/FastBrain-9B** (`v2`, default \\(t = 0.50\\)) | 9.4B | **0.751** | **0.858** | 0.985 | 0.814 | 0.996 | 0.368 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
115
- | **TypeScript/FastBrain-9B** (`v2`, calibrated \\(t = 0.961\\)) | 9.4B | **0.751** | **0.858** | 0.985 | **0.938** | 0.931 | 0.056 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
116
- | **TypeScript/FastBrain-9B** (`v1` official, \\(N = 562\\)) | 9.4B | **0.850** | **0.923** | 0.983 | **0.974** | **1.000** | 0.051 | **78 / 78** | 104 ms (RTX PRO 6000) |
117
  | Cloudflare/clef | 27.4B | 0.563 | 0.737 | **0.993** | 0.808 | **1.000** | 0.384 | **82 / 82** | 636 ms (A100) |
118
  | vllm-sr/d3-mini | 1.8B | 0.512 | 0.652 | 0.986 | 0.856 | **1.000** | 0.288 | **82 / 82** | 168 ms (A100) |
119
  | Cloudflare/clef-flash (base) | 9.4B | 0.462 | 0.576 | 0.967 | 0.893 | 0.978 | 0.192 | 81 / 82 | 326 ms (A100) |
@@ -128,11 +128,11 @@ On the binary guardrail track, FastBrain-9B attains `0.985` AUROC and `0.996` ma
128
 
129
  [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) tests multi-primitive structured judgment across four enterprise workflows (`agent_trace_observability`, `customer_service`, `invoice_processing`, and `security_incidents`), comprising 400 held-out test cases and 2,000 individual decisions (`noul`, `choice`, and `score`) scored against teacher-panel soft label distributions. Two scoring conventions exist in the benchmark repository: `bench/score_with_jobs.py` resolves exact ties in the gold distribution by selecting the first index (`g.index(max(g))`), whereas `scripts/score_model.py` credits any tied maximum across the 20 tied test items; KL divergence and Brier score are identical under both scripts. We report both accuracy figures (`strict / tied-top`) alongside prediction traces in [`eval_submissions/`](./eval_submissions/).
130
 
131
- On the official zero-shot leaderboard (\\(N = 43\\) models), FastBrain-9B scores `0.7525` (`0.7625` tied-top) accuracy, `0.1508` KL divergence, `0.0795` Brier score, and `0.085` ECE at `50 ms` p50 latency (RTX PRO 6000 via SGLang). It places **7th of 43** overall, **5th of 36** among open-weight models, **2nd of 19** among ≤9B open-weight models, and **1st** among all `Clef`-architecture models, improving over 9.4B `Cloudflare/clef-flash` (`0.7100` accuracy, `0.2093` KL) by `+4.25` percentage points and over 27.4B `Cloudflare/clef` (`0.7275` accuracy, `0.1907` KL) by `+2.50` points. Across all 28 models exceeding `0.700` zero-shot accuracy, FastBrain-9B exhibits the lowest p50 latency (`50 ms`, compared with `118 ms` for 9B `Amos 1 medium`, `357 ms` for 27B `Amos 1 large`, `526 ms` for `meraGPT Decider 1`, and `3,335 ms` for 31B `Quyet-1.0-Large`). Evaluating the uncalibrated ordinal-KL distilled head directly yields `0.7735` raw accuracy (`0.1039` KL, `0.0582` Brier).
132
 
133
- On the separate `train`-fitted leaderboard (\\(N = 51\\) models), fitting only scalar per-question log-priors and temperature parameters on the 1,200 `train` cases—with all backbone and head weights frozen (`no weights changed`)—achieves `0.7970` (`0.8070` tied-top) accuracy, `0.0816` KL divergence, and `0.0437` Brier score, ranking **12th-tied (14th listed of 51)** overall and **1st in KL calibration** among frozen-weight models.
134
 
135
- **Table 3: Performance on `LocalLLaMA/typed-decisions` (\\(N = 400\\) cases, 2,000 decisions). Values separated by `/` denote `score_with_jobs.py` / `score_model.py` tie handling.**
136
 
137
  | Model | Params | Protocol | Overall Acc ↑ | KL ↓ | Brier ↓ | `noul` ↑ | `choice` ↑ | `score` ↑ | Agent ↑ | Cust. Svc ↑ | Invoice ↑ | Security ↑ | p50 Latency |
138
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
@@ -154,7 +154,7 @@ On the separate `train`-fitted leaderboard (\\(N = 51\\) models), fitting only s
154
 
155
  ### 3.3 Multi-Domain Decision Suite (Cloudflare Decision Index `v0.2.1`)
156
 
157
- On the 10-benchmark **Cloudflare Decision Index (`v0.2.1`)** suite (\\(N = 22,427\\) evaluation requests spanning AST tool calling, 20-candidate tool reranking, stateful API invocation, fine-grained intent classification, out-of-scope rejection, reasoning-intensive retrieval, e-commerce relevance, phishing detection, and tool-use governance), FastBrain-9B attains an **80.55%** unweighted macro mean (Table 4, Figures 1–3).
158
 
159
  Across the ≤9B parameter tier (`Clef-flash` 9.4B, `Kev 9B`, and `Laya` 0.4B), FastBrain-9B ranks **1st on all 10 benchmarks**, improving the 10-task macro mean over `Clef-flash` (`74.96%`) by `+5.59` percentage points and over `Kev 9B` (`76.51%`) by `+4.04` points. The largest gains occur on calibration-sensitive abstention and governance tasks: `+31.67` points on `CLINC150+OOS` (`98.44%` vs. `66.77%`), `+15.55` points on `When2Call` (`81.13%` vs. `65.58%`), and `+6.80` points on `PhishNChips` (`78.75%` vs. `71.95%`). Across all model scales up to 27.4B, FastBrain-9B ranks **2nd of 7** overall, surpassing `27B Jev` (`80.32%`) and `26B-A4B DiffusionGemma Jev` (`69.14%`), and exceeding `27.4B Clef` (`81.81%`) on 5 of 10 individual benchmarks (`BFCL`, `API-Bank`, `CLINC150+OOS`, `Home Appliances`, and `When2Call`).
160
 
@@ -167,7 +167,7 @@ Across the ≤9B parameter tier (`Clef-flash` 9.4B, `Kev 9B`, and `Laya` 0.4B),
167
  ![Figure 3: Reliability diagrams and open-set calibration breakdown](./assets/decision_calibration_breakdown.png)
168
  *Figure 3. Confidence calibration, open-set out-of-scope rejection (`CLINC150+OOS`), and abstention margin analysis (`When2Call`).*
169
 
170
- **Table 4: Per-benchmark results on Cloudflare Decision Index (`v0.2.1`, 10 tasks, \\(N = 22,427\\)). Bold denotes the highest score across all parameter scales; underline denotes best in the ≤9B class.**
171
 
172
  | Benchmark (Metric) | **FastBrain-9B** (Ours) | Clef-flash (Base) | Kev 9B | Clef | Jev | DiffGemma | Laya |
173
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
@@ -190,7 +190,7 @@ Across the ≤9B parameter tier (`Clef-flash` 9.4B, `Kev 9B`, and `Laya` 0.4B),
190
 
191
  ### 4.1 Runtime Comparison Across Inference Engines
192
 
193
- Table 5 compares end-to-end latency, prefill throughput, and memory footprint across three serving backends—SGLang (`lmsysorg/sglang:dev-clef` with `JointSchemaPooler`), `llama.cpp` (`b11430` `ClefModel` graph in `Q8_0` and `Q4_K_M`), and native PyTorch (`joint_schema_model.py`)—benchmarking identical requests on a single NVIDIA RTX PRO 6000 Blackwell Server Edition GPU. Because `output_tokens = 0` for all requests, throughput is governed purely by prefill compute and span-pooling efficiency, reaching `19,319 tok/s` (`131.2` decisions/s) at batch size \\(B = 4\\) in SGLang.
194
 
195
  **Table 5: End-to-end latency and prefill throughput on NVIDIA RTX PRO 6000 Blackwell Server Edition (G4).**
196
 
@@ -200,12 +200,12 @@ Table 5 compares end-to-end latency, prefill throughput, and memory footprint ac
200
  | Peak GPU Memory (16K context) | ~20.5 GiB | 12.55 GiB | **10.30 GiB** | 20.3–24.7 GiB |
201
  | Compatible with 16 GiB T4 GPU | No (BF16 TP=1) | Yes (`9.03 GiB` at 4K) | **Yes (`6.78 GiB` at 4K)** | Yes (`6.8 GiB` via NF4) |
202
  | Top-1 Agreement vs. PyTorch BF16 | **100.00%** | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) | 100.00% (reference) |
203
- | 1. Short Intent Routing (293 tok, \\(K=2\\)) | **30.01 ms** (9,763 tok/s) | 32.82 ms (8,929 tok/s) | 31.69 ms (9,245 tok/s) | 43.23 ms (6,778 tok/s) |
204
- | 2. Typed Decisions (736 tok, \\(K=5\\)) | **50.29 ms** (14,635 tok/s) | 62.40 ms (11,795 tok/s) | 61.80 ms (11,909 tok/s) | 63.58 ms (11,576 tok/s) |
205
- | 3. When2Call Governance (1,068 tok, \\(K=1\\)) | **66.06 ms** (16,167 tok/s) | 87.78 ms (12,167 tok/s) | 89.34 ms (11,955 tok/s) | 77.77 ms (13,733 tok/s) |
206
- | 4. PhishNChips Security (1,303 tok, \\(K=9\\)) | **75.86 ms** (17,176 tok/s) | 108.55 ms (12,004 tok/s) | 109.97 ms (11,849 tok/s) | 90.32 ms (14,426 tok/s) |
207
- | 5. SafeAgents Judge (2,217 tok, \\(K=2\\)) | **125.85 ms** (17,616 tok/s) | 181.70 ms (12,201 tok/s) | 183.50 ms (12,082 tok/s) | 138.06 ms (16,058 tok/s) |
208
- | Batch \\(B=4\\) Typed Decisions (2,944 tok, 20 dec) | **152.39 ms** (19,319 tok/s) | 248.50 ms (11,847 tok/s) | 245.20 ms (12,006 tok/s) | 175.85 ms (16,742 tok/s) |
209
 
210
  ### 4.2 Mixed-Precision GGUF Quantization Fidelity
211
 
@@ -215,8 +215,8 @@ Uniform 4-bit quantization degrades cross-attention span readout and linear-atte
215
 
216
  | Metric / Subset | PyTorch `bfloat16` (18.82 GB) | GGUF `Q8_0` (9.66 GB) | GGUF `Q4_K_M` (6.73 GB, `Q8_0` Head) |
217
  | :--- | :---: | :---: | :---: |
218
- | Top-1 Decision Agreement (\\(N = 1,473\\)) | 100.00% (ref) | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) |
219
- | Mean Absolute Probability Error (\\(\lvert \hat{p}_q - \hat{p}_{\mathrm{bf16}} \rvert\\)) | 0.00000 | 0.00203 | 0.00387 |
220
  | 10-Task Mean (Uncalibrated Head) | 82.37% | 82.34% (-0.03) | 82.56% (+0.19) |
221
  | 10-Task Mean (Schema-Calibrated) | 83.68% | 83.65% (-0.03) | 83.48% (-0.20) |
222
  | ├─ 1. BFCL / 3. API-Bank / 4. BANKING77 | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% |
 
65
 
66
  ### 2.2 Joint Schema Decision Head and Vectorized CUDA Kernel
67
 
68
+ The `JointSchemaHead` (`116M` parameters, stored in `joint_head.safetensors`) pools token representations bounded by `<|question_start|>`/`<|question_end|>` and `<|option_start|>`/`<|option_end|>` markers from the final backbone hidden states \ \(H \in \mathbb{R}^{T \times d}\\), applies cross-attention over question-option pairs, and projects directly to scalar logits for three decision primitives:
69
+ 1. **`choice`** (*K*-way categorical distribution): \ \(\hat{p} = \operatorname{softmax}(z / \tau) \in \Delta^{K-1}\\).
70
+ 2. **`noul`** (binary verification): \ \(\hat{p}(\text{true}) = \sigma(z_{\text{true}} - z_{\text{false}})\\).
71
+ 3. **`score`** (ordinal rubric over levels \ \(v_1 < \dots < v_K\\) ): categorical probabilities \ \(\hat{p} \in \Delta^{K-1}\\) together with expected value \ \(\mathbb{E}[\hat{y}] = \sum_{k=1}^K \hat{p}_k v_k\\).
72
 
73
+ In `joint_schema_model.py`, the reference Python span-extraction loop and padded SDPA mask are replaced on CUDA devices by `_forward_fast_cuda`: span means are computed in \ \(\mathcal{O}(1)\\) kernel launches via prefix sums (`torch.cumsum`) over the schema suffix, and cross-attention is executed via packed variable-length FlashAttention (`torch.ops.aten._flash_attention_forward` parameterized by `cu_seqlens`). On NVIDIA RTX PRO 6000 Blackwell, this reduces isolated decision-head latency from `3.27 ms` to `1.72 ms` (**1.90× speedup**) with exact numerical equivalence (`max_logit_diff = 0.00000`).
74
 
75
  ### 2.3 Ordinal-KL Distillation and Schema-Topology Calibration
76
 
77
+ Standard cross-entropy or flat KL divergence treats ordinal rubric levels as unordered classes, penalizing adjacent-level errors identically to extreme errors. FastBrain-9B is trained on `TypeScript/FastBrain-Instruct` under a composite loss combining soft-target KL divergence, discrete ranked probability score (squared \ \(L_2\\) distance between predicted and target cumulative distribution functions \ \(\hat{F}_k = \sum_{j \le k} \hat{p}_j\\) ), and expected-score mean squared error on `score` primitives:
78
 
79
  $$
80
  \mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{ord}} \sum_{k=1}^{K-1} \bigl(\hat{F}_{\theta,k} - F_k\bigr)^2 + \mathbb{I}[\text{type}=\text{score}] \cdot \lambda_{\mathrm{mse}} \bigl(\mathbb{E}[\hat{y}_\theta] - \mathbb{E}[y]\bigr)^2.
 
101
 
102
  ### 3.1 Agent Tool-Call Guardrail Evaluation (`safeagents-judge-bench`)
103
 
104
+ [`harness-hub/safeagents-judge-bench`](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) evaluates whether a System-1 decision model can audit suspicious tool calls flagged by a static scanner (`atif-scan`) during real coding-agent executions on Terminal-Bench 2.1, OpenThoughts-TBLite, and Terminal-Bench 4. Version 2 (`v2`) comprises *N* = 661 tool calls across 55 tasks (`536` malicious `block` calls spanning `82` misbehaving trajectories and `125` legitimate `allow` calls). Each call is issued as a single zero-shot System-1 request containing one binary `noul` question (`block`) and one 7-way `choice` question (`legit`, `eval_gaming`, `harness_introspection`, `credential_misuse`, `exfiltration`, `sandbox_escape`, `other_overreach`). All v2 results in Table 2 were executed independently on NVIDIA A100-SXM4-80GB via Hugging Face Jobs ([Merged PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1)).
105
 
106
+ On the 7-way category classification track, FastBrain-9B achieves `0.751` macro-F1 and `0.858` accuracy (`567/661`), outperforming the 27.4B `Cloudflare/clef` model (`0.563` macro-F1, `0.737` accuracy) by `+18.8` percentage points in macro-F1 and `+12.1` points in accuracy, and improving upon its 9.4B base model `Cloudflare/clef-flash` (`0.462` macro-F1, `0.576` accuracy) by `+28.9` and `+28.2` points, respectively. Per-category F1 reaches `0.945` on `harness_introspection` (*n* = 272, recall `0.978`), `0.828` on `eval_gaming` (*n* = 245, recall `0.853`), and `0.761` on `legit` (*n* = 125, precision `0.975`).
107
 
108
+ On the binary guardrail track, FastBrain-9B attains `0.985` AUROC and `0.996` malicious-call recall (`534/536`), intercepting all `82/82` unsafe trajectories (`median_bad_calls_before_catch = 0.0`), with `1.000` recall on `harness_introspection` (`272/272`), `credential_misuse` (`15/15`), and `sandbox_escape` (`4/4`). Under the uncalibrated default decision rule \ \(P(\text{block}) \ge 0.50\\), balanced accuracy is `0.814` (false-block rate `0.368` across the 125 `allow` calls, compared with `0.808` balanced accuracy and `0.384` false-block rate for 27.4B `Cloudflare/clef`). Because rank-ordering discrimination is preserved (`0.985` AUROC), setting the operating threshold to \ \(P(\text{block}) \ge 0.961\\) yields `0.938` balanced accuracy at a `0.056` (`7/125`) false-block rate while maintaining `82/82` trajectory interception. Table 2 also includes the `v1` release (*N* = 562 calls, `78` trajectories), where FastBrain-9B ranked 1st of 9 on both tracks.
109
 
110
+ **Table 2: Zero-shot evaluation on `harness-hub/safeagents-judge-bench` (`v2`, *N* = 661 calls, 82 unsafe trajectories; and `v1` reference, *N* = 562).**
111
 
112
  | Model | Params | Cat. Macro-F1 ↑ | Cat. Acc ↑ | AUROC ↑ | Bal. Acc ↑ | Recall ↑ | False Block ↓ | Traj. Caught ↑ | p50 Latency (Hardware) |
113
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
114
+ | **TypeScript/FastBrain-9B** (`v2`, default *t* = 0.50) | 9.4B | **0.751** | **0.858** | 0.985 | 0.814 | 0.996 | 0.368 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
115
+ | **TypeScript/FastBrain-9B** (`v2`, calibrated *t* = 0.961) | 9.4B | **0.751** | **0.858** | 0.985 | **0.938** | 0.931 | 0.056 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
116
+ | **TypeScript/FastBrain-9B** (`v1` official, *N* = 562) | 9.4B | **0.850** | **0.923** | 0.983 | **0.974** | **1.000** | 0.051 | **78 / 78** | 104 ms (RTX PRO 6000) |
117
  | Cloudflare/clef | 27.4B | 0.563 | 0.737 | **0.993** | 0.808 | **1.000** | 0.384 | **82 / 82** | 636 ms (A100) |
118
  | vllm-sr/d3-mini | 1.8B | 0.512 | 0.652 | 0.986 | 0.856 | **1.000** | 0.288 | **82 / 82** | 168 ms (A100) |
119
  | Cloudflare/clef-flash (base) | 9.4B | 0.462 | 0.576 | 0.967 | 0.893 | 0.978 | 0.192 | 81 / 82 | 326 ms (A100) |
 
128
 
129
  [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) tests multi-primitive structured judgment across four enterprise workflows (`agent_trace_observability`, `customer_service`, `invoice_processing`, and `security_incidents`), comprising 400 held-out test cases and 2,000 individual decisions (`noul`, `choice`, and `score`) scored against teacher-panel soft label distributions. Two scoring conventions exist in the benchmark repository: `bench/score_with_jobs.py` resolves exact ties in the gold distribution by selecting the first index (`g.index(max(g))`), whereas `scripts/score_model.py` credits any tied maximum across the 20 tied test items; KL divergence and Brier score are identical under both scripts. We report both accuracy figures (`strict / tied-top`) alongside prediction traces in [`eval_submissions/`](./eval_submissions/).
130
 
131
+ On the official zero-shot leaderboard (*N* = 43 models), FastBrain-9B scores `0.7525` (`0.7625` tied-top) accuracy, `0.1508` KL divergence, `0.0795` Brier score, and `0.085` ECE at `50 ms` p50 latency (RTX PRO 6000 via SGLang). It places **7th of 43** overall, **5th of 36** among open-weight models, **2nd of 19** among ≤9B open-weight models, and **1st** among all `Clef`-architecture models, improving over 9.4B `Cloudflare/clef-flash` (`0.7100` accuracy, `0.2093` KL) by `+4.25` percentage points and over 27.4B `Cloudflare/clef` (`0.7275` accuracy, `0.1907` KL) by `+2.50` points. Across all 28 models exceeding `0.700` zero-shot accuracy, FastBrain-9B exhibits the lowest p50 latency (`50 ms`, compared with `118 ms` for 9B `Amos 1 medium`, `357 ms` for 27B `Amos 1 large`, `526 ms` for `meraGPT Decider 1`, and `3,335 ms` for 31B `Quyet-1.0-Large`). Evaluating the uncalibrated ordinal-KL distilled head directly yields `0.7735` raw accuracy (`0.1039` KL, `0.0582` Brier).
132
 
133
+ On the separate `train`-fitted leaderboard (*N* = 51 models), fitting only scalar per-question log-priors and temperature parameters on the 1,200 `train` cases—with all backbone and head weights frozen (`no weights changed`)—achieves `0.7970` (`0.8070` tied-top) accuracy, `0.0816` KL divergence, and `0.0437` Brier score, ranking **12th-tied (14th listed of 51)** overall and **1st in KL calibration** among frozen-weight models.
134
 
135
+ **Table 3: Performance on `LocalLLaMA/typed-decisions` (*N* = 400 cases, 2,000 decisions). Values separated by `/` denote `score_with_jobs.py` / `score_model.py` tie handling.**
136
 
137
  | Model | Params | Protocol | Overall Acc ↑ | KL ↓ | Brier ↓ | `noul` ↑ | `choice` ↑ | `score` ↑ | Agent ↑ | Cust. Svc ↑ | Invoice ↑ | Security ↑ | p50 Latency |
138
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
 
154
 
155
  ### 3.3 Multi-Domain Decision Suite (Cloudflare Decision Index `v0.2.1`)
156
 
157
+ On the 10-benchmark **Cloudflare Decision Index (`v0.2.1`)** suite (*N* = 22,427 evaluation requests spanning AST tool calling, 20-candidate tool reranking, stateful API invocation, fine-grained intent classification, out-of-scope rejection, reasoning-intensive retrieval, e-commerce relevance, phishing detection, and tool-use governance), FastBrain-9B attains an **80.55%** unweighted macro mean (Table 4, Figures 1–3).
158
 
159
  Across the ≤9B parameter tier (`Clef-flash` 9.4B, `Kev 9B`, and `Laya` 0.4B), FastBrain-9B ranks **1st on all 10 benchmarks**, improving the 10-task macro mean over `Clef-flash` (`74.96%`) by `+5.59` percentage points and over `Kev 9B` (`76.51%`) by `+4.04` points. The largest gains occur on calibration-sensitive abstention and governance tasks: `+31.67` points on `CLINC150+OOS` (`98.44%` vs. `66.77%`), `+15.55` points on `When2Call` (`81.13%` vs. `65.58%`), and `+6.80` points on `PhishNChips` (`78.75%` vs. `71.95%`). Across all model scales up to 27.4B, FastBrain-9B ranks **2nd of 7** overall, surpassing `27B Jev` (`80.32%`) and `26B-A4B DiffusionGemma Jev` (`69.14%`), and exceeding `27.4B Clef` (`81.81%`) on 5 of 10 individual benchmarks (`BFCL`, `API-Bank`, `CLINC150+OOS`, `Home Appliances`, and `When2Call`).
160
 
 
167
  ![Figure 3: Reliability diagrams and open-set calibration breakdown](./assets/decision_calibration_breakdown.png)
168
  *Figure 3. Confidence calibration, open-set out-of-scope rejection (`CLINC150+OOS`), and abstention margin analysis (`When2Call`).*
169
 
170
+ **Table 4: Per-benchmark results on Cloudflare Decision Index (`v0.2.1`, 10 tasks, *N* = 22,427). Bold denotes the highest score across all parameter scales; underline denotes best in the ≤9B class.**
171
 
172
  | Benchmark (Metric) | **FastBrain-9B** (Ours) | Clef-flash (Base) | Kev 9B | Clef | Jev | DiffGemma | Laya |
173
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
 
190
 
191
  ### 4.1 Runtime Comparison Across Inference Engines
192
 
193
+ Table 5 compares end-to-end latency, prefill throughput, and memory footprint across three serving backends—SGLang (`lmsysorg/sglang:dev-clef` with `JointSchemaPooler`), `llama.cpp` (`b11430` `ClefModel` graph in `Q8_0` and `Q4_K_M`), and native PyTorch (`joint_schema_model.py`)—benchmarking identical requests on a single NVIDIA RTX PRO 6000 Blackwell Server Edition GPU. Because `output_tokens = 0` for all requests, throughput is governed purely by prefill compute and span-pooling efficiency, reaching `19,319 tok/s` (`131.2` decisions/s) at batch size *B* = 4 in SGLang.
194
 
195
  **Table 5: End-to-end latency and prefill throughput on NVIDIA RTX PRO 6000 Blackwell Server Edition (G4).**
196
 
 
200
  | Peak GPU Memory (16K context) | ~20.5 GiB | 12.55 GiB | **10.30 GiB** | 20.3–24.7 GiB |
201
  | Compatible with 16 GiB T4 GPU | No (BF16 TP=1) | Yes (`9.03 GiB` at 4K) | **Yes (`6.78 GiB` at 4K)** | Yes (`6.8 GiB` via NF4) |
202
  | Top-1 Agreement vs. PyTorch BF16 | **100.00%** | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) | 100.00% (reference) |
203
+ | 1. Short Intent Routing (293 tok, *K* = 2) | **30.01 ms** (9,763 tok/s) | 32.82 ms (8,929 tok/s) | 31.69 ms (9,245 tok/s) | 43.23 ms (6,778 tok/s) |
204
+ | 2. Typed Decisions (736 tok, *K* = 5) | **50.29 ms** (14,635 tok/s) | 62.40 ms (11,795 tok/s) | 61.80 ms (11,909 tok/s) | 63.58 ms (11,576 tok/s) |
205
+ | 3. When2Call Governance (1,068 tok, *K* = 1) | **66.06 ms** (16,167 tok/s) | 87.78 ms (12,167 tok/s) | 89.34 ms (11,955 tok/s) | 77.77 ms (13,733 tok/s) |
206
+ | 4. PhishNChips Security (1,303 tok, *K* = 9) | **75.86 ms** (17,176 tok/s) | 108.55 ms (12,004 tok/s) | 109.97 ms (11,849 tok/s) | 90.32 ms (14,426 tok/s) |
207
+ | 5. SafeAgents Judge (2,217 tok, *K* = 2) | **125.85 ms** (17,616 tok/s) | 181.70 ms (12,201 tok/s) | 183.50 ms (12,082 tok/s) | 138.06 ms (16,058 tok/s) |
208
+ | Batch *B* = 4 Typed Decisions (2,944 tok, 20 dec) | **152.39 ms** (19,319 tok/s) | 248.50 ms (11,847 tok/s) | 245.20 ms (12,006 tok/s) | 175.85 ms (16,742 tok/s) |
209
 
210
  ### 4.2 Mixed-Precision GGUF Quantization Fidelity
211
 
 
215
 
216
  | Metric / Subset | PyTorch `bfloat16` (18.82 GB) | GGUF `Q8_0` (9.66 GB) | GGUF `Q4_K_M` (6.73 GB, `Q8_0` Head) |
217
  | :--- | :---: | :---: | :---: |
218
+ | Top-1 Decision Agreement (*N* = 1,473) | 100.00% (ref) | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) |
219
+ | Mean Absolute Probability Error — \ \(\lvert \hat{p}_q - \hat{p}_{\mathrm{bf16}} \rvert\\) | 0.00000 | 0.00203 | 0.00387 |
220
  | 10-Task Mean (Uncalibrated Head) | 82.37% | 82.34% (-0.03) | 82.56% (+0.19) |
221
  | 10-Task Mean (Schema-Calibrated) | 83.68% | 83.65% (-0.03) | 83.48% (-0.20) |
222
  | ├─ 1. BFCL / 3. API-Bank / 4. BANKING77 | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% |
README_zh.md CHANGED
@@ -39,16 +39,16 @@ FastBrain-9B 主干网络包含 32 层混合序列建模模块(`9.30B` 参数
39
 
40
  ### 2.2 联合 Schema 决策头与向量化 CUDA 算子
41
 
42
- `JointSchemaHead`(`116M` 参数,独立存储于 `joint_head.safetensors`)从主干末层隐藏状态 \\(H \in \mathbb{R}^{T \times d}\\) 中提取由 `<|question_start|>`/`<|question_end|>` 与 `<|option_start|>`/`<|option_end|>` 界定的片段表征,经问题-选项交叉注意力交互后映射为三类基础决策原语的标量对数几率:
43
- 1. **`choice`**(\\(K\\) 分类离散分布):输出归一化概率向量 \\(\hat{p} = \operatorname{softmax}(z / \tau) \in \Delta^{K-1}\\)。
44
- 2. **`noul`**(二元真值判别):输出命题成立概率 \\(\hat{p}(\text{true}) = \sigma(z_{\text{true}} - z_{\text{false}})\\)。
45
- 3. **`score`**(定义在有序分值 \\(v_1 < \dots < v_K\\) 上的量表评估):同时输出各等级概率 \\(\hat{p} \in \Delta^{K-1}\\) 与数学期望得分 \\(\mathbb{E}[\hat{y}] = \sum_{k=1}^K \hat{p}_k v_k\\)。
46
 
47
- 在参考运行时 `joint_schema_model.py` 中,针对 CUDA 设备实现了融合算子路径 `_forward_fast_cuda`:利用基于 `torch.cumsum` 的 \\(\mathcal{O}(1)\\) 前缀和差分替代逐片段 Python 循环完成变长片段均值池化,并通过 `cu_seqlens` 紧凑打包调用变长 FlashAttention(`torch.ops.aten._flash_attention_forward`)替代稠密掩码注意力。在 NVIDIA RTX PRO 6000 Blackwell 上,该实现将决策头单次前向耗时由 `3.27 ms` 降至 `1.72 ms`(**1.90 倍加速**),且保持严格数值等价(最大 Logit 绝对偏差 `max_logit_diff = 0.00000`)。
48
 
49
  ### 2.3 序数 KL 蒸馏与编译期 Schema 拓扑校准
50
 
51
- 标准交叉熵或普通多类 KL 散度将有序评定量表视为无序独立类别,对相邻等级误差与跨端极端误差施加同等惩罚。FastBrain-9B 在 `TypeScript/FastBrain-Instruct` 数据集上采用复合分布匹配损失进行训练,联合优化软标签 KL 散度、离散分级概率评分(预测累积分布 \\(\hat{F}_k = \sum_{j \le k} \hat{p}_j\\) 与目标累积分布 \\(F_k\\) 之间的二阶距离)以及 `score` 题型的期望值均方误差:
52
 
53
  $$
54
  \mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{ord}} \sum_{k=1}^{K-1} \bigl(\hat{F}_{\theta,k} - F_k\bigr)^2 + \mathbb{I}[\text{type}=\text{score}] \cdot \lambda_{\mathrm{mse}} \bigl(\mathbb{E}[\hat{y}_\theta] - \mathbb{E}[y]\bigr)^2.
@@ -75,19 +75,19 @@ $$
75
 
76
  ### 3.1 智能体工具调用安全审计评测 (`safeagents-judge-bench`)
77
 
78
- [`harness-hub/safeagents-judge-bench`](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) 用于评估决策模型对真实编码智能体(运行于 Terminal-Bench 2.1、OpenThoughts-TBLite 与 Terminal-Bench 4)在沙箱中被静态扫描器 `atif-scan` 标记的可疑工具调用的审计能力。`v2` 测试集包含来自 55 个任务的 \\(N = 661\\) 次工具调用(其中 `536` 次为分布在 `82` 条违规轨迹中的恶意 `block` 调用,`125` 次为正常 `allow` 调用)。每次调用以零样本单次请求形式同时询��一个二元 `noul` 拦截问题与一个 7 类 `choice` 违规归因问题(`legit`、`eval_gaming`、`harness_introspection`、`credential_misuse`、`exfiltration`、`sandbox_escape`、`other_overreach`)。表 2 中的 `v2` 官方结果由基准维护者在 NVIDIA A100-SXM4-80GB(HF Jobs)上独立运行并提交合入([PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1))。
79
 
80
- 在 7 类违规归因赛道上,FastBrain-9B 取得 `0.751` 的 Macro-F1 与 `0.858` 的分类准确率(`567/661`),位列全部参评模型第 1:相较于参数量约为其 3 倍的 27.4B 模型 `Cloudflare/clef`(Macro-F1 `0.563`,准确率 `0.737`),分别高出 `+18.8` 与 `+12.1` 个百分点;相较于同尺寸基座 `Cloudflare/clef-flash`(Macro-F1 `0.462`,准确率 `0.576`),分别提升 `+28.9` 与 `+28.2` 个百分点。分项指标上,`harness_introspection`(\\(n=272\\))F1 达 `0.945`(召回率 `0.978`),`eval_gaming`(\\(n=245\\))F1 达 `0.828`(召回率 `0.853`),合法调用 `legit`(\\(n=125\\))精确率达 `0.975`。
81
 
82
- 在二元安全拦截赛道上,FastBrain-9B 的 AUROC 为 `0.985`,恶意调用总召回率为 `0.996`(`534/536`),全部 `82/82` 条越界轨迹均在首次违规调用处即被捕获(`median_bad_calls_before_catch = 0.0`),其中 `harness_introspection`(`272/272`)、`credential_misuse`(`15/15`)与 `sandbox_escape`(`4/4`)三类召回率均为 `1.000`。在未经验证集调整的默认判别阈值 \\(P(\text{block}) \ge 0.50\\) 下,平衡准确率为 `0.814`(误拦率 `0.368`,优于 27.4B `Cloudflare/clef` 的平衡准确率 `0.808` 与误拦率 `0.384`)。由于连续概率排序区分度保持在 `0.985` AUROC,将工作点阈值设定为 \\(P(\text{block}) \ge 0.961\\) 时,平衡准确率提升至 `0.938`,误拦率降至 `0.056`(`7/125`),同时仍保持 `82/82` 越界轨迹全检出。表 2 同时列出了 `v1` 版本(\\(N = 562\\),78 条轨迹)的官方归档结果,FastBrain-9B 在该版本二元与多分类双赛道均位居第 1 / 9。
83
 
84
- **表 2:`harness-hub/safeagents-judge-bench` 零样本评测结果对比(`v2` 测试集 \\(N = 661\\),82 条越界轨迹;附 `v1` 归档对照 \\(N = 562\\))。**
85
 
86
  | ��型 | 参数量 | 7 分类 Macro-F1 ↑ | 7 分类准确率 ↑ | AUROC ↑ | 平衡准确率 ↑ | 恶意召回率 ↑ | 误拦率 ↓ | 越界轨迹检出 ↑ | p50 延迟(测试硬件) |
87
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
88
- | **TypeScript/FastBrain-9B** (`v2`, 默认 \\(t = 0.50\\)) | 9.4B | **0.751** | **0.858** | 0.985 | 0.814 | 0.996 | 0.368 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
89
- | **TypeScript/FastBrain-9B** (`v2`, 校准 \\(t = 0.961\\)) | 9.4B | **0.751** | **0.858** | 0.985 | **0.938** | 0.931 | 0.056 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
90
- | **TypeScript/FastBrain-9B** (`v1` 官方榜, \\(N = 562\\)) | 9.4B | **0.850** | **0.923** | 0.983 | **0.974** | **1.000** | 0.051 | **78 / 78** | 104 ms (RTX PRO 6000) |
91
  | Cloudflare/clef | 27.4B | 0.563 | 0.737 | **0.993** | 0.808 | **1.000** | 0.384 | **82 / 82** | 636 ms (A100) |
92
  | vllm-sr/d3-mini | 1.8B | 0.512 | 0.652 | 0.986 | 0.856 | **1.000** | 0.288 | **82 / 82** | 168 ms (A100) |
93
  | Cloudflare/clef-flash (基座) | 9.4B | 0.462 | 0.576 | 0.967 | 0.893 | 0.978 | 0.192 | 81 / 82 | 326 ms (A100) |
@@ -102,11 +102,11 @@ $$
102
 
103
  [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) 评测集覆盖智能体轨迹可观测性(`agent_trace_observability`)、客户服务(`customer_service`)、供应商发票核验(`invoice_processing`)与安全事件响应(`security_incidents`)四类企业工作流,测试集包含 400 个独立案例、共 2,000 个类型化子问题(涵盖 `noul`、`choice` 与 `score`),并以多教师模型集成的软标签分布作为真值基准。官方仓库存在两种并列真值处理脚本:`bench/score_with_jobs.py` 对真值分布中概率并列最高的选项按首个索引判对(`g.index(max(g))`),而 `scripts/score_model.py` 将 20 道并列题中的任一最高分选项均计为正确;两种脚本下的 KL 散度与 Brier 评分完全一致。我们在表 3 中同时报告两种口径的准确率,并在 [`eval_submissions/`](./eval_submissions/) 中公开原始预测文件。
104
 
105
- 在零样本总榜(\\(N = 43\\) 个参评模型)中,FastBrain-9B 取得 `0.7525`(并列口径 `0.7625`)的准确率、`0.1508` 的 KL 散度、`0.0795` 的 Brier 评分与 `0.085` 的期望校准误差(ECE),在 RTX PRO 6000(SGLang)上的 p50 延迟为 `50 ms`。该成绩位列全量零样本榜 **第 7 / 43**、开源权重模型 **第 5 / 36**、≤9B 开源模型 **第 2 / 19** 以及全部 `Clef` 架构模型 **第 1**,较 9.4B 基座 `Cloudflare/clef-flash`(准确率 `0.7100`,KL `0.2093`)提升 `+4.25` 个百分点,较 27.4B `Cloudflare/clef`(准确率 `0.7275`,KL `0.1907`)提升 `+2.50` 个百分点。在零样本准确率高于 `0.700` 的全部 28 个模型中,FastBrain-9B 具有最低的 p50 推理延迟(`50 ms`,对比 9B `Amos 1 medium` 的 `118 ms`、27B `Amos 1 large` 的 `357 ms`、`meraGPT Decider 1` 的 `526 ms` 以及 31B `Quyet-1.0-Large` 的 `3,335 ms`)。若直接评测未经后验规则调整的序数 KL 蒸馏原生头,零样本准确率为 `0.7735`(KL `0.1039`,Brier `0.0582`)。
106
 
107
- 在使用 `train` 划分(1,200 个样本)的独立榜单(\\(N = 51\\) 个模型)上,在**完全冻结主干与决策头权重(`no weights changed`)**、仅拟合逐问题对数先验偏置与标量温度参数的前提下,FastBrain-9B 取得 `0.7970`(并列口径 `0.8070`)的准确率、`0.0816` 的 KL 散度与 `0.0437` 的 Brier 评分,位列 **并列第 12(列表第 14 / 51)**,并在冻结权重模型中取得最优的 KL 分布校准度。
108
 
109
- **表 3:`LocalLLaMA/typed-decisions` 测试集(\\(N = 400\\) 案例,2,000 次决策)评测结果。`/` 前后分别对应 `score_with_jobs.py` 与 `score_model.py` 口径。**
110
 
111
  | 模型 | 参数量 | 评测设定 | 总体准确率 ↑ | KL 散度 ↓ | Brier ↓ | `noul` ↑ | `choice` ↑ | `score` ↑ | Agent ↑ | 客服 ↑ | 发票 ↑ | 安全 ↑ | p50 延迟 |
112
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
@@ -128,7 +128,7 @@ $$
128
 
129
  ### 3.3 多领域结构化决策基准 (Cloudflare Decision Index `v0.2.1`)
130
 
131
- 在涵盖 10 个评测数据集、共 \\(N = 22,427\\) 条请求的 **Cloudflare Decision Index(`v0.2.1`)** 套件(覆盖 AST 函数调用、20 候选工具检索重排、多步状态 API 调用、细粒度金融意图识别、开放集越界拒识、推理密集型检索、电商相关性分级、多探针钓鱼网站鉴别与工具调用触发治理)上,FastBrain-9B 的无加权 10 任务宏平均得分为 **80.55%**(表 4,图 1–3)。
132
 
133
  在 ≤9B 参数量级(对比 `Clef-flash` 9.4B、`Kev 9B` 与 `Laya` 0.4B)中,FastBrain-9B 在全部 **10 / 10 个子基准上均取得第 1**,宏平均分较同架构基座 `Clef-flash`(`74.96%`)提升 `+5.59` 个百分点,较 `Kev 9B`(`76.51%`)提升 `+4.04` 个百分点。增幅主要集中于对边界校准敏感的开放集拒识与安全治理任务:`CLINC150+OOS` 提升 `+31.67` 个百分点(`98.44%` 对比 `66.77%`),`When2Call` 提升 `+15.55` 个百分点(`81.13%` 对比 `65.58%`),`PhishNChips` 提升 `+6.80` 个百分点(`78.75%` 对比 `71.95%`)。跨全部参数规模(最高 27.4B)比较,FastBrain-9B 总宏平均位列 **第 2 / 7**,高于 `27B Jev`(`80.32%`)与 `26B-A4B DiffusionGemma Jev`(`69.14%`),并在 `BFCL`、`API-Bank`、`CLINC150+OOS`、`Home Appliances` 与 `When2Call` 五个子基准上高于 `27.4B Clef`(`81.81%`)。
134
 
@@ -141,7 +141,7 @@ $$
141
  ![图 3:置信度校准曲线与开放集越界拒识分析](./assets/decision_calibration_breakdown.png)
142
  *图 3. 置信度可靠性图、开放集越界意图拒识(`CLINC150+OOS`)与工具触发边际分析(`When2Call`)。*
143
 
144
- **表 4:Cloudflare Decision Index(`v0.2.1`,10 任务,\\(N = 22,427\\))逐项评测结果。粗体表示全参数规模最高值,下划线表示 ≤9B 量级最高值。**
145
 
146
  | 评测基准(指标) | **FastBrain-9B** (本工作) | Clef-flash (基座) | Kev 9B | Clef | Jev | DiffGemma | Laya |
147
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
@@ -164,7 +164,7 @@ $$
164
 
165
  ### 4.1 不同推理引擎性能对比
166
 
167
- 表 5 展示了在同一张 NVIDIA RTX PRO 6000 Blackwell Server Edition GPU 上,SGLang(集成 `JointSchemaPooler` 的 `lmsysorg/sglang:dev-clef`)、`llama.cpp`(`b11430` `ClefModel` 计算图,含 `Q8_0` 与 `Q4_K_M` 两种精度)以及原生 PyTorch(`joint_schema_model.py`)在五类典型长度负载下的端到端延迟、Prefill 吞吐率与显存占用。由于全部请求的生成步数均为零(`output_tokens = 0`),推理耗时完全取决于 Prefill 计算与片段池化效率;在批量 \\(B=4\\) 并发设定下,SGLang 吞吐达 `19,319 tok/s`(每秒完成 `131.2` 个独立结构化决策)。
168
 
169
  **表 5:NVIDIA RTX PRO 6000 Blackwell Server Edition(G4)上三类推理后端的延迟与吞吐实测。**
170
 
@@ -174,23 +174,23 @@ $$
174
  | 峰值显存占用 (16K 上下文) | ~20.5 GiB | 12.55 GiB | **10.30 GiB** | 20.3–24.7 GiB |
175
  | 兼容 16 GiB T4 显卡 | 否 (BF16 TP=1) | 是 (`9.03 GiB`, 4K) | **是 (`6.78 GiB`, 4K)** | 是 (`6.8 GiB`, NF4) |
176
  | 相对 PyTorch BF16 Top-1 一致率 | **100.00%** | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) | 100.00% (基准) |
177
- | 1. 短文本意图路由 (293 tok, \\(K=2\\)) | **30.01 ms** (9,763 tok/s) | 32.82 ms (8,929 tok/s) | 31.69 ms (9,245 tok/s) | 43.23 ms (6,778 tok/s) |
178
- | 2. Typed Decisions (736 tok, \\(K=5\\)) | **50.29 ms** (14,635 tok/s) | 62.40 ms (11,795 tok/s) | 61.80 ms (11,909 tok/s) | 63.58 ms (11,576 tok/s) |
179
- | 3. When2Call 工具治理 (1,068 tok, \\(K=1\\)) | **66.06 ms** (16,167 tok/s) | 87.78 ms (12,167 tok/s) | 89.34 ms (11,955 tok/s) | 77.77 ms (13,733 tok/s) |
180
- | 4. PhishNChips 安全检测 (1,303 tok, \\(K=9\\)) | **75.86 ms** (17,176 tok/s) | 108.55 ms (12,004 tok/s) | 109.97 ms (11,849 tok/s) | 90.32 ms (14,426 tok/s) |
181
- | 5. SafeAgents 审计 (2,217 tok, \\(K=2\\)) | **125.85 ms** (17,616 tok/s) | 181.70 ms (12,201 tok/s) | 183.50 ms (12,082 tok/s) | 138.06 ms (16,058 tok/s) |
182
- | 批量 \\(B=4\\) Typed Decisions (2,944 tok, 20 题) | **152.39 ms** (19,319 tok/s) | 248.50 ms (11,847 tok/s) | 245.20 ms (12,006 tok/s) | 175.85 ms (16,742 tok/s) |
183
 
184
  ### 4.2 混合精度 GGUF 量化保真度分析
185
 
186
  若对交叉注意力决策头或线性注意力递归状态直接施加均匀 4-bit 量化,细粒度选项间微小的对数几率差值易受量化噪声干扰。因此在 `FastBrain-9B-Q4_K_M.gguf`(`6.73 GB`)构建方案中,仅对前馈网络(FFN)使用 4-bit K-quants,而将全部 `dec.*` 决策头张量、`ssm_*` Gated DeltaNet 状态参数以及 `attn_*` 投影矩阵保持为 `Q8_0` 精度。在涵盖 10 类任务的分层验证子集(`203` 个多约束请求,共 `1,473` 道独立评分题;表 6)上,`Q8_0.gguf`(`9.66 GB`)在 `1,471/1,473` 道题上与原生 `bfloat16` 给出完全一致的 Top-1 决策(一致率 `99.86%`,预测概率平均绝对误差 `0.00203`,10 任务均分差异 `-0.03` 个百分点);`Q4_K_M.gguf` 保持了 `98.78%` 的 Top-1 决策一致率(概率 MAE `0.00387`,校准后均分差异 `-0.20` 个百分点)。表 7 给出了各精度版本在不同硬件(含 NVIDIA Tesla T4 `16 GiB` 实测)上的资源需求。
187
 
188
- **表 6:10 任务分层子集(\\(N = 1,473\\) 道评分题)上 GGUF 量化模型的决策一致性与精度保持率。**
189
 
190
  | 评测指标 / 子任务 | PyTorch `bfloat16` (18.82 GB) | GGUF `Q8_0` (9.66 GB) | GGUF `Q4_K_M` (6.73 GB, `Q8_0` 决策头) |
191
  | :--- | :---: | :---: | :---: |
192
- | Top-1 决策一致率 (\\(N = 1,473\\)) | 100.00% (基准) | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) |
193
- | 预测概率平均绝对误差 (\\(\lvert \hat{p}_q - \hat{p}_{\mathrm{bf16}} \rvert\\)) | 0.00000 | 0.00203 | 0.00387 |
194
  | 10 任务宏平均(未校准原生输出) | 82.37% | 82.34% (-0.03) | 82.56% (+0.19) |
195
  | 10 任务宏平均(Schema 拓扑校准) | 83.68% | 83.65% (-0.03) | 83.48% (-0.20) |
196
  | ├─ 1. BFCL / 3. API-Bank / 4. BANKING77 | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% |
 
39
 
40
  ### 2.2 联合 Schema 决策头与向量化 CUDA 算子
41
 
42
+ `JointSchemaHead`(`116M` 参数,独立存储于 `joint_head.safetensors`)从主干末层隐藏状态 \ \(H \in \mathbb{R}^{T \times d}\\) 中提取由 `<|question_start|>`/`<|question_end|>` 与 `<|option_start|>`/`<|option_end|>` 界定的片段表征,经问题-选项交叉注意力交互后映射为三类基础决策原语的标量对数几率:
43
+ 1. **`choice`** (*K* 分类离散分布): 输出归一化概率向量 \ \(\hat{p} = \operatorname{softmax}(z / \tau) \in \Delta^{K-1}\\) 。
44
+ 2. **`noul`** (二元真值判别): 输出命题成立概率 \ \(\hat{p}(\text{true}) = \sigma(z_{\text{true}} - z_{\text{false}})\\) 。
45
+ 3. **`score`** (定义在有序分值 \ \(v_1 < \dots < v_K\\) 上的量表评估): 同时输出各等级概率 \ \(\hat{p} \in \Delta^{K-1}\\) 与数学期望得分 \ \(\mathbb{E}[\hat{y}] = \sum_{k=1}^K \hat{p}_k v_k\\) 。
46
 
47
+ 在参考运行时 `joint_schema_model.py` 中,针对 CUDA 设备实现了融合算子路径 `_forward_fast_cuda`:利用基于 `torch.cumsum` 的 \ \(\mathcal{O}(1)\\) 前缀和差分替代逐片段 Python 循环完成变长片段均值池化,并通过 `cu_seqlens` 紧凑打包调用变长 FlashAttention(`torch.ops.aten._flash_attention_forward`)替代稠密掩码注意力。在 NVIDIA RTX PRO 6000 Blackwell 上,该实现将决策头单次前向耗时由 `3.27 ms` 降至 `1.72 ms`(**1.90 倍加速**),且保持严格数值等价(最大 Logit 绝对偏差 `max_logit_diff = 0.00000`)。
48
 
49
  ### 2.3 序数 KL 蒸馏与编译期 Schema 拓扑校准
50
 
51
+ 标准交叉熵或普通多类 KL 散度将有序评定量表视为无序独立类别,对相邻等级误差与跨端极端误差施加同等惩罚。FastBrain-9B 在 `TypeScript/FastBrain-Instruct` 数据集上采用复合分布匹配损失进行训练,联合优化软标签 KL 散度、离散分级概率评分(预测累积分布 \ \(\hat{F}_k = \sum_{j \le k} \hat{p}_j\\) 与目标累积分布 \ \(F_k\\) 之间的二阶距离)以及 `score` 题型的期望值均方误差:
52
 
53
  $$
54
  \mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{ord}} \sum_{k=1}^{K-1} \bigl(\hat{F}_{\theta,k} - F_k\bigr)^2 + \mathbb{I}[\text{type}=\text{score}] \cdot \lambda_{\mathrm{mse}} \bigl(\mathbb{E}[\hat{y}_\theta] - \mathbb{E}[y]\bigr)^2.
 
75
 
76
  ### 3.1 智能体工具调用安全审计评测 (`safeagents-judge-bench`)
77
 
78
+ [`harness-hub/safeagents-judge-bench`](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) 用于评估决策模型对真实编码智能体(运行于 Terminal-Bench 2.1、OpenThoughts-TBLite 与 Terminal-Bench 4)在沙箱中被静态扫描器 `atif-scan` 标记的可疑工具调用的审计能力。`v2` 测试集包含来自 55 个任务的 *N* = 661 次工具调用(其中 `536` 次为分布在 `82` 条违规轨迹中的恶意 `block` 调用,`125` 次为正常 `allow` 调用)。每次调用以零样本单次请求形式同时询问一个二元 `noul` 拦截问题与一个 7 类 `choice` 违规归因问题(`legit`、`eval_gaming`、`harness_introspection`、`credential_misuse`、`exfiltration`、`sandbox_escape`、`other_overreach`)。表 2 中的 `v2` 官方结果由基准维护者在 NVIDIA A100-SXM4-80GB(HF Jobs)上独立运行并提交合入([PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1))。
79
 
80
+ 在 7 类违规归因赛道上,FastBrain-9B 取得 `0.751` 的 Macro-F1 与 `0.858` 的分类准确率(`567/661`),位列全部参评模型第 1:相较于参数量约为其 3 倍的 27.4B 模型 `Cloudflare/clef`(Macro-F1 `0.563`,准确率 `0.737`),分别高出 `+18.8` 与 `+12.1` 个百分点;相较于同尺寸基座 `Cloudflare/clef-flash`(Macro-F1 `0.462`,准确率 `0.576`),分别提升 `+28.9` 与 `+28.2` 个百分点。分项指标上,`harness_introspection`(*n* = 272)F1 达 `0.945`(召回率 `0.978`),`eval_gaming`(*n* = 245)F1 达 `0.828`(召回率 `0.853`),合法调用 `legit`(*n* = 125)精确率达 `0.975`。
81
 
82
+ 在二元安全拦截赛道上,FastBrain-9B 的 AUROC 为 `0.985`,恶意调用总召回率为 `0.996`(`534/536`),全部 `82/82` 条越界轨迹均在首次违规调用处即被捕获(`median_bad_calls_before_catch = 0.0`),其中 `harness_introspection`(`272/272`)、`credential_misuse`(`15/15`)与 `sandbox_escape`(`4/4`)三类召回率均为 `1.000`。在未经验证集调整的默认判别阈值 \ \(P(\text{block}) \ge 0.50\\) 下,平衡准确率为 `0.814`(误拦率 `0.368`,优于 27.4B `Cloudflare/clef` 的平衡准确率 `0.808` 与误拦率 `0.384`)。由于连续概率排序区分度保持在 `0.985` AUROC,将工作点阈值设定为 \ \(P(\text{block}) \ge 0.961\\) 时,平衡准确率提升至 `0.938`,误拦率降至 `0.056`(`7/125`),同时仍保持 `82/82` 越界轨迹全检出。表 2 同时列出了 `v1` 版本(*N* = 562,78 条轨迹)的官方归档结果,FastBrain-9B 在该版本二元与多分类双赛道均位居第 1 / 9。
83
 
84
+ **表 2:`harness-hub/safeagents-judge-bench` 零样本评测结果对比(`v2` 测试集 *N* = 661,82 条越界轨迹;附 `v1` 归档对照 *N* = 562)。**
85
 
86
  | ��型 | 参数量 | 7 分类 Macro-F1 ↑ | 7 分类准确率 ↑ | AUROC ↑ | 平衡准确率 ↑ | 恶意召回率 ↑ | 误拦率 ↓ | 越界轨迹检出 ↑ | p50 延迟(测试硬件) |
87
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
88
+ | **TypeScript/FastBrain-9B** (`v2`, 默认 *t* = 0.50) | 9.4B | **0.751** | **0.858** | 0.985 | 0.814 | 0.996 | 0.368 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
89
+ | **TypeScript/FastBrain-9B** (`v2`, 校准 *t* = 0.961) | 9.4B | **0.751** | **0.858** | 0.985 | **0.938** | 0.931 | 0.056 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
90
+ | **TypeScript/FastBrain-9B** (`v1` 官方榜, *N* = 562) | 9.4B | **0.850** | **0.923** | 0.983 | **0.974** | **1.000** | 0.051 | **78 / 78** | 104 ms (RTX PRO 6000) |
91
  | Cloudflare/clef | 27.4B | 0.563 | 0.737 | **0.993** | 0.808 | **1.000** | 0.384 | **82 / 82** | 636 ms (A100) |
92
  | vllm-sr/d3-mini | 1.8B | 0.512 | 0.652 | 0.986 | 0.856 | **1.000** | 0.288 | **82 / 82** | 168 ms (A100) |
93
  | Cloudflare/clef-flash (基座) | 9.4B | 0.462 | 0.576 | 0.967 | 0.893 | 0.978 | 0.192 | 81 / 82 | 326 ms (A100) |
 
102
 
103
  [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) 评测集覆盖智能体轨迹可观测性(`agent_trace_observability`)、客户服务(`customer_service`)、供应商发票核验(`invoice_processing`)与安全事件响应(`security_incidents`)四类企业工作流,测试集包含 400 个独立案例、共 2,000 个类型化子问题(涵盖 `noul`、`choice` 与 `score`),并以多教师模型集成的软标签分布作为真值基准。官方仓库存在两种并列真值处理脚本:`bench/score_with_jobs.py` 对真值分布中概率并列最高的选项按首个索引判对(`g.index(max(g))`),而 `scripts/score_model.py` 将 20 道并列题中的任一最高分选项均计为正确;两种脚本下的 KL 散度与 Brier 评分完全一致。我们在表 3 中同时报告两种口径的准确率,并在 [`eval_submissions/`](./eval_submissions/) 中公开原始预测文件。
104
 
105
+ 在零样本总榜(*N* = 43 个参评模型)中,FastBrain-9B 取得 `0.7525`(并列口径 `0.7625`)的准确率、`0.1508` 的 KL 散度、`0.0795` 的 Brier 评分与 `0.085` 的期望校准误差(ECE),在 RTX PRO 6000(SGLang)上的 p50 延迟为 `50 ms`。该成绩位列全量零样本榜 **第 7 / 43**、开源权重模型 **第 5 / 36**、≤9B 开源模型 **第 2 / 19** 以及全部 `Clef` 架构模型 **第 1**,较 9.4B 基座 `Cloudflare/clef-flash`(准确率 `0.7100`,KL `0.2093`)提升 `+4.25` 个百分点,较 27.4B `Cloudflare/clef`(准确率 `0.7275`,KL `0.1907`)提升 `+2.50` 个百分点。在零样本准确率高于 `0.700` 的全部 28 个模型中,FastBrain-9B 具有最低的 p50 推理延迟(`50 ms`,对比 9B `Amos 1 medium` 的 `118 ms`、27B `Amos 1 large` 的 `357 ms`、`meraGPT Decider 1` 的 `526 ms` 以及 31B `Quyet-1.0-Large` 的 `3,335 ms`)。若直接评测未经后验规则调整的序数 KL 蒸馏原生头,零样本准确率为 `0.7735`(KL `0.1039`,Brier `0.0582`)。
106
 
107
+ 在使用 `train` 划分(1,200 个样本)的独立榜单(*N* = 51 个模型)上,在 **完全冻结主干与决策头权重 (`no weights changed`)** 、仅拟合逐问题对数先验偏置与标量温度参数的前提下,FastBrain-9B 取得 `0.7970`(并列口径 `0.8070`)的准确率、`0.0816` 的 KL 散度与 `0.0437` 的 Brier 评分,位列 **并列第 12(列表第 14 / 51)**,并在冻结权重模型中取得最优的 KL 分布校准度。
108
 
109
+ **表 3:`LocalLLaMA/typed-decisions` 测试集(*N* = 400 案例,2,000 次决策)评测结果。`/` 前后分别对应 `score_with_jobs.py` 与 `score_model.py` 口径。**
110
 
111
  | 模型 | 参数量 | 评测设定 | 总体准确率 ↑ | KL 散度 ↓ | Brier ↓ | `noul` ↑ | `choice` ↑ | `score` ↑ | Agent ↑ | 客服 ↑ | 发票 ↑ | 安全 ↑ | p50 延迟 |
112
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
 
128
 
129
  ### 3.3 多领域结构化决策基准 (Cloudflare Decision Index `v0.2.1`)
130
 
131
+ 在涵盖 10 个评测数据集、共 *N* = 22,427 条请求的 **Cloudflare Decision Index (`v0.2.1`)** 套件(覆盖 AST 函数调用、20 候选工具检索重排、多步状态 API 调用、细粒度金融意图识别、开放集越界拒识、推理密集型检索、电商相关性分级、多探针钓鱼网站鉴别与工具调用触发治理)上,FastBrain-9B 的无加权 10 任务宏平均得分为 **80.55%**(表 4,图 1–3)。
132
 
133
  在 ≤9B 参数量级(对比 `Clef-flash` 9.4B、`Kev 9B` 与 `Laya` 0.4B)中,FastBrain-9B 在全部 **10 / 10 个子基准上均取得第 1**,宏平均分较同架构基座 `Clef-flash`(`74.96%`)提升 `+5.59` 个百分点,较 `Kev 9B`(`76.51%`)提升 `+4.04` 个百分点。增幅主要集中于对边界校准敏感的开放集拒识与安全治理任务:`CLINC150+OOS` 提升 `+31.67` 个百分点(`98.44%` 对比 `66.77%`),`When2Call` 提升 `+15.55` 个百分点(`81.13%` 对比 `65.58%`),`PhishNChips` 提升 `+6.80` 个百分点(`78.75%` 对比 `71.95%`)。跨全部参数规模(最高 27.4B)比较,FastBrain-9B 总宏平均位列 **第 2 / 7**,高于 `27B Jev`(`80.32%`)与 `26B-A4B DiffusionGemma Jev`(`69.14%`),并在 `BFCL`、`API-Bank`、`CLINC150+OOS`、`Home Appliances` 与 `When2Call` 五个子基准上高于 `27.4B Clef`(`81.81%`)。
134
 
 
141
  ![图 3:置信度校准曲线与开放集越界拒识分析](./assets/decision_calibration_breakdown.png)
142
  *图 3. 置信度可靠性图、开放集越界意图拒识(`CLINC150+OOS`)与工具触发边际分析(`When2Call`)。*
143
 
144
+ **表 4:Cloudflare Decision Index(`v0.2.1`,10 任务,*N* = 22,427)逐项评测结果。粗体表示全参数规模最高值,下划线表示 ≤9B 量级最高值。**
145
 
146
  | 评测基准(指标) | **FastBrain-9B** (本工作) | Clef-flash (基座) | Kev 9B | Clef | Jev | DiffGemma | Laya |
147
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
 
164
 
165
  ### 4.1 不同推理引擎性能对比
166
 
167
+ 表 5 展示了在同一张 NVIDIA RTX PRO 6000 Blackwell Server Edition GPU 上,SGLang(集成 `JointSchemaPooler` 的 `lmsysorg/sglang:dev-clef`)、`llama.cpp`(`b11430` `ClefModel` 计算图,含 `Q8_0` 与 `Q4_K_M` 两种精度)以及原生 PyTorch(`joint_schema_model.py`)在五类典型长度负载下的端到端延迟、Prefill 吞吐率与显存占用��由于全部请求的生成步数均为零(`output_tokens = 0`),推理耗时完全取决于 Prefill 计算与片段池化效率;在批量 *B* = 4 并发设定下,SGLang 吞吐达 `19,319 tok/s`(每秒完成 `131.2` 个独立结构化决策)。
168
 
169
  **表 5:NVIDIA RTX PRO 6000 Blackwell Server Edition(G4)上三类推理后端的延迟与吞吐实测。**
170
 
 
174
  | 峰值显存占用 (16K 上下文) | ~20.5 GiB | 12.55 GiB | **10.30 GiB** | 20.3–24.7 GiB |
175
  | 兼容 16 GiB T4 显卡 | 否 (BF16 TP=1) | 是 (`9.03 GiB`, 4K) | **是 (`6.78 GiB`, 4K)** | 是 (`6.8 GiB`, NF4) |
176
  | 相对 PyTorch BF16 Top-1 一致率 | **100.00%** | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) | 100.00% (基准) |
177
+ | 1. 短文本意图路由 (293 tok, *K* = 2) | **30.01 ms** (9,763 tok/s) | 32.82 ms (8,929 tok/s) | 31.69 ms (9,245 tok/s) | 43.23 ms (6,778 tok/s) |
178
+ | 2. Typed Decisions (736 tok, *K* = 5) | **50.29 ms** (14,635 tok/s) | 62.40 ms (11,795 tok/s) | 61.80 ms (11,909 tok/s) | 63.58 ms (11,576 tok/s) |
179
+ | 3. When2Call 工具治理 (1,068 tok, *K* = 1) | **66.06 ms** (16,167 tok/s) | 87.78 ms (12,167 tok/s) | 89.34 ms (11,955 tok/s) | 77.77 ms (13,733 tok/s) |
180
+ | 4. PhishNChips 安全检测 (1,303 tok, *K* = 9) | **75.86 ms** (17,176 tok/s) | 108.55 ms (12,004 tok/s) | 109.97 ms (11,849 tok/s) | 90.32 ms (14,426 tok/s) |
181
+ | 5. SafeAgents 审计 (2,217 tok, *K* = 2) | **125.85 ms** (17,616 tok/s) | 181.70 ms (12,201 tok/s) | 183.50 ms (12,082 tok/s) | 138.06 ms (16,058 tok/s) |
182
+ | 批量 *B* = 4 Typed Decisions (2,944 tok, 20 题) | **152.39 ms** (19,319 tok/s) | 248.50 ms (11,847 tok/s) | 245.20 ms (12,006 tok/s) | 175.85 ms (16,742 tok/s) |
183
 
184
  ### 4.2 混合精度 GGUF 量化保真度分析
185
 
186
  若对交叉注意力决策头或线性注意力递归状态直接施加均匀 4-bit 量化,细粒度选项间微小的对数几率差值易受量化噪声干扰。因此在 `FastBrain-9B-Q4_K_M.gguf`(`6.73 GB`)构建方案中,仅对前馈网络(FFN)使用 4-bit K-quants,而将全部 `dec.*` 决策头张量、`ssm_*` Gated DeltaNet 状态参数以及 `attn_*` 投影矩阵保持为 `Q8_0` 精度。在涵盖 10 类任务的分层验证子集(`203` 个多约束请求,共 `1,473` 道独立评分题;表 6)上,`Q8_0.gguf`(`9.66 GB`)在 `1,471/1,473` 道题上与原生 `bfloat16` 给出完全一致的 Top-1 决策(一致率 `99.86%`,预测概率平均绝对误差 `0.00203`,10 任务均分差异 `-0.03` 个百分点);`Q4_K_M.gguf` 保持了 `98.78%` 的 Top-1 决策一致率(概率 MAE `0.00387`,校准后均分差异 `-0.20` 个百分点)。表 7 给出了各精度版本在不同硬件(含 NVIDIA Tesla T4 `16 GiB` 实测)上的资源需求。
187
 
188
+ **表 6:10 任务分层子集(*N* = 1,473 道评分题)上 GGUF 量化模型的决策一致性与精度保持率。**
189
 
190
  | 评测指标 / 子任务 | PyTorch `bfloat16` (18.82 GB) | GGUF `Q8_0` (9.66 GB) | GGUF `Q4_K_M` (6.73 GB, `Q8_0` 决策头) |
191
  | :--- | :---: | :---: | :---: |
192
+ | Top-1 决策一致率 (*N* = 1,473) | 100.00% (基准) | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) |
193
+ | 预测概率平均绝对误差 — \ \(\lvert \hat{p}_q - \hat{p}_{\mathrm{bf16}} \rvert\\) | 0.00000 | 0.00203 | 0.00387 |
194
  | 10 任务宏平均(未校准原生输出) | 82.37% | 82.34% (-0.03) | 82.56% (+0.19) |
195
  | 10 任务宏平均(Schema 拓扑校准) | 83.68% | 83.65% (-0.03) | 83.48% (-0.20) |
196
  | ├─ 1. BFCL / 3. API-Bank / 4. BANKING77 | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% |