bigai
/

GS1-0.8B

@@ -7,73 +7,218 @@ tags:
 - custom_code
 - system-one
 - safetensors
 ---
 # GS1-0.8B
 [English](README.md) · [简体中文](README.zh-CN.md)
-![GS1](assets/banner.svg)
-**Fast, Faithful, Frontier.** GS1-0.8B is a native typed decision model. Supply state, instructions and runtime candidate meanings; get **Choice**, **Noul** and **Score** distributions with zero generated output tokens.
-[Code and optimized runtime](https://github.com/bigai-nlco/GS1) · [Model family](https://huggingface.co/collections/bigai/gs1) · [Detailed results](https://github.com/bigai-nlco/GS1/blob/main/benchmark/results.md)
-| Measurement | GS1-0.8B |
-|---|---:|
 | Stored parameters | 879,765,056 |
-| Public Decision Index, 37 benchmarks | **56.12** |
-| JevBench public task accuracy, 231 decisions | **77.92%** |
-| AG News test accuracy, 7,600 decisions | **90.99%** |
-| ToolACE selection accuracy, 640 decisions | **93.28%** |
-| Warm complete CUDA Graph decision P50 / P95 | **12.51 / 13.22 ms** |
-| Peak allocated GPU memory in that latency fixture | **4.07 GiB** |
-Decision Index: complete public edition 0.3, 115,479 unique requests and 279,333 decisions, 100% final coverage, BF16 native reference computation. This is a public score, separate from private Full scores. The three regular benchmarks were measured on 9 November 2026 with zero failures or retries. JevBench public task accuracy is separate from the sealed composite. ToolACE measures first-tool/no-tool selection on a fixed hash subset, not tool-argument generation.
-Latency measured on 9 November 2026: RTX 4090 reporting 48 GiB, BF16 computation, 256-token longest branch, one question, single GPU, five warmups and 30 measured calls. Complete in-process timing includes validation, compilation, model computation and probabilities; loading, graph capture, HTTP and SDK transport are excluded. [Machine-readable decision metrics](assets/results/public-evaluation.json) · [Raw latency samples](assets/results/inference-latency.json) · [Methodology and all models](https://github.com/bigai-nlco/GS1/blob/main/benchmark/latency.md)
-## Default Hugging Face inference
-**No installation or import of the GS1 repository is required.** Use Python 3.11+ and install this folder's dependencies:
 ```bash
-python -m pip install -r requirements.txt
 ```
 ```python
 from transformers import AutoModel
 model = AutoModel.from_pretrained(
-    "bigai/GS1-0.8B",  # a local GS1-0.8B directory also works
     trust_remote_code=True,
 ).eval().to("cuda")
-result = model.decide(
-    state="A customer was charged twice and asks for a refund.",
-    questions={
         "team": {
             "type": "choice",
-            "instructions": "Choose the team that should handle this ticket.",
-            "criteria": {"billing": "Payments and refunds", "technical": "Software problems"},
         },
-        "refund": {"type": "noul", "instructions": "Does the customer request a refund?"},
         "urgency": {
             "type": "score",
             "instructions": "Rate the urgency of handling this ticket.",
-            "criteria": ["Routine", "Soon", "Immediate"],
-        },
-    },
-)
 print(result["answers"])
 ```
-This works from a local release directory before the Hub upload as well. `trust_remote_code=True` loads the model code supplied in this folder. Default loading preserves the checkpoint's mixed storage dtypes; avoid a blanket dtype conversion when reproducing the measured precision policy. On a CPU-compatible PyTorch installation, use `.to("cpu")` and `decide(..., precision="fp32")`. Optional upstream CUDA kernels can be installed for faster eager inference; the measured stack uses `flash-linear-attention[cuda]==0.5.2` and `causal-conv1d==1.7.0`.
-`result` contains `model`, `answers` and `usage`. Choice returns a candidate ID, a probability for every candidate, and confidence. Noul returns `p(true)`. Score returns the expected zero-based rubric level and probabilities over all supplied levels. Output-token usage is zero. The default path evaluates complete question branches independently and has no runtime profile or graph-capture dependency.
-## Optimized GS1 inference
-The GS1 repository provides source-bound CUDA Graph replay, compatible batching, shared-state reuse, HTTP serving and a playground for these same weights.
 ```bash
 git clone https://github.com/bigai-nlco/GS1.git
@@ -82,34 +227,48 @@ uv sync --locked --extra serve --extra cuda
 uv run gs1 serve --model bigai/GS1-0.8B --model-name GS1
 ```
-From the source checkout, install `--extra eval` in addition to the serving/CUDA extras, then qualify the installed model, hardware and request geometry before enabling graphs:
-```bash
-uv run python -m benchmark.inference --model bigai/GS1-0.8B \
-  --out runs/runtime.json --runtime-profile-out runs/profile.json \
-  --graph-cache-mb 16384 --graph-cache-entries 128
-uv run gs1 serve --model bigai/GS1-0.8B --model-name GS1 \
-  --runtime-profile runs/profile.json --experimental-cuda-graphs \
-  --graph-warmup-requests runs/profile-warmup.json \
-  --graph-cache-mb 4096 --graph-cache-entries 128
-```
-The benchmark writes `profile-warmup.json`. Serve known warmed graphs, with native eager fallback for graph misses and single-reference fallback for unqualified geometries. Regenerate the profile when code, model, precision or hardware changes. These profiles belong to the local runtime and are not uploaded as model weights. [Full inference guide](https://github.com/bigai-nlco/GS1/blob/main/docs/inference.md)
-## Architecture and numerical policy
-![Architecture](assets/architecture.svg)
-GS1-0.8B has width 1,024, 24 decoder layers and 4 decision-attention heads. It combines a Qwen3.5 hybrid GDN/attention backbone, segmented bidirectional full attention, carry-preserving Attention Residuals and a native cross-attention decision head with two routing layers and a 256-dimensional final comparison space. There is no language-generation output head. Candidate meanings are supplied at inference time rather than fixed to a task-specific classifier vocabulary.
-Persistent text/head weights are FP32 and the retained vision tower is BF16; the checkpoint specifies BF16 head computation. Final logits/probability reductions are FP32. Complete branches support up to 65,536 tokens subject to explicit admission and memory budgets; the default HTTP service uses a tighter 4,096-token limit. The published decision results are text-only. Model definitions, token IDs and `inference_limits` are in `config.json`; all safetensors keys and dtypes are preserved.
-## Scope and license
-Use for routing, classification, tool selection, bounded judgments and other typed decisions. A returned probability is not a guarantee of correctness; calibration depends on the task distribution. The public results include limitations such as low HLE accuracy and zero AA-Omniscience non-hallucination rate at larger sizes when confidence rejection is disabled; inspect [all component results](assets/results/public-evaluation.json).
-Code and released weights: [Apache-2.0](LICENSE), with [upstream attribution](NOTICE). This folder contains model architecture and inference dependencies. Evaluation inputs and local runtime profiles are distributed separately.
 ```bibtex
 @techreport{zheng2026gs1,
@@ -118,4 +277,4 @@ Code and released weights: [Apache-2.0](LICENSE), with [upstream attribution](NO
   institution={BIGAI},
   year={2026}
 }
-```

 - custom_code
 - system-one
 - safetensors
+base_model:
+- Qwen/Qwen3.5-0.8B
 ---
 # GS1-0.8B
 [English](README.md) · [简体中文](README.zh-CN.md)
+![GS1 — Fast, Faithful, Frontier](assets/banner.svg)
+> ***Fast, Faithful, Frontier System One Intelligence.***
+[Code](https://github.com/bigai-nlco/GS1) · [Model](https://huggingface.co/collections/bigai/gs1) · [Documentation](https://github.com/bigai-nlco/GS1/blob/main/docs/README.md) · [Playground](https://github.com/bigai-nlco/GS1/blob/main/playground/README.md)
+**GS1-0.8B** is an open-source decision model developed by **BIGAI** (Beijing Institute for General Artificial Intelligence). It computes native probability distributions over runtime-defined candidates for agent routing, verification, extraction and control.
+## GS1 Highlights
+- **Fast - Native Runtime Decision Head.**
+  - Produce Choice, Noul and Score distributions directly, with zero generated output tokens.
+  - A native multi-head cross-attention decision head queries state evidence using question and candidate representations.
+  - With CUDA Graph, GS1-0.8B reaches 12.51 ms P50 for one short question and **6.05×** speedup over default inference for eight short questions.
+- **Faithful — multi-step RL with calibrated rewards**
+  - Reinforcement learning optimizes sampled action probabilities with a Brier reward. [RLCD](https://github.com/bigai-nlco/GS1/blob/main/docs/RLCD.md)
+  - Native probabilities and confidence support evidence gathering, deferral and explicit abstention choices.
+- **Frontier — Hybrid Architecture, Competitive Quality**
+  - **Causal Gated DeltaNet** and **Segmented Bidirectional Full Attention** give candidates their complete comparison context while preserving immutable prefixes for reuse across questions.
+  - Carry-preserving **Attention Residual** lets sparse full-attention receivers query earlier representations while retaining the explicit skip-connection anchor. [Architecture](https://github.com/bigai-nlco/GS1/blob/main/docs/model.md)
+  - GS1-0.8B reaches **56.12** on the latest Decision Index 0.4.1 Public index. All GS1 sizes lie on the displayed parameter–performance frontier.
+## Model Overview
+| Specification | GS1-0.8B |
+| --- | ---: |
+| Model type | Native typed decision model |
 | Stored parameters | 879,765,056 |
+| Hidden dimension | 1,024 |
+| Decoder layers | 24 |
+| Decision attention heads | 4 |
+| Candidate-routing layers | 2 |
+| Decision projection dimension | 256 |
+| Maximum complete branch length | 65,536 tokens |
+GS1 uses a Qwen3.5 hybrid backbone that interleaves three causal Gated DeltaNet layers with one segmented full-attention layer. A native cross-attention decision head compares runtime-defined candidates against state evidence. [Architecture](https://github.com/bigai-nlco/GS1/blob/main/docs/model.md)
+Text and decision-head weights are stored in FP32, and the retained vision tower in BF16. Inference supports BF16 computation with FP32 final logits and probability reductions. Stored dtypes are preserved by default loading.
+![GS1 architecture](assets/architecture.svg)
+| Primitive | Typical use | Output |
+| --- | --- | --- |
+| **Choice** | Routing, classification, tool selection | Candidate ID, all candidate probabilities, confidence |
+| **Noul** | Verification and binary judgments | Probability that the proposition is true: `p(true)` |
+| **Score** | Ordered ratings and rubric judgments | Expected zero-based level, level probabilities, confidence |
+## Benchmark Results
+### Decision Index
+The following are the **latest complete public-suite results** (edition 0.3). Each model completes **140,178 requests** and **312,320 decisions** across **43 text benchmarks**, with **100% coverage** and zero failed or unsupported requests. **37 benchmarks** contribute to the chance-corrected Public index; six additional benchmarks are displayed separately. The private Full score is evaluated by the leaderboard maintainers.
+| Metric | [GS1-0.8B](https://huggingface.co/bigai/GS1-0.8B) | [GS1-2B](https://huggingface.co/bigai/GS1-2B) | [GS1-4B](https://huggingface.co/bigai/GS1-4B) | [GS1-9B](https://huggingface.co/bigai/GS1-9B) |
+| --- | ---: | ---: | ---: | ---: |
+| Public index ↑ | **56.12** | **62.22** | **68.97** | **70.45** |
+| Raw index ↑ | 67.36 | 71.70 | 76.47 | 77.46 |
+| Coverage | 100% | 100% | 100% | 100% |
+![GS1 parameter–performance Pareto frontier](assets/gs1_pareto_frontier.svg)
+The frontier refers to the comparison models displayed in the figure.
+### Area Scores
+Chance-corrected area scores on a 0–100 scale; higher is better.
+| Area | GS1-0.8B | GS1-2B | GS1-4B | GS1-9B |
+| --- | ---: | ---: | ---: | ---: |
+| Knowledge & Reasoning | 33.32 | 41.84 | 52.40 | 56.01 |
+| Language Understanding | 56.25 | 63.67 | 71.12 | 72.62 |
+| Retrieval & Classification | 68.62 | 71.21 | 75.55 | 75.58 |
+| Tools & Automation | 79.26 | 83.57 | 87.49 | 87.61 |
+| Arts & Human Taste | 47.47 | 54.16 | 59.26 | 60.57 |
+<details>
+<summary>Complete text benchmark results — 43 benchmarks</summary>
+Native benchmark scores are scaled to 0–100 before chance correction. The metric is shown for each row. † marks the six display-only benchmarks that do not contribute to the Public index. All rows use the Decision Index typed-decision protocol.
+| Benchmark | Native metric | GS1-0.8B | GS1-2B | GS1-4B | GS1-9B |
+| --- | --- | ---: | ---: | ---: | ---: |
+| BFCL | case exact accuracy | 90.44 | 93.27 | 97.52 | 96.99 |
+| ToolRet | nDCG@10 | 67.52 | 69.25 | 71.16 | 70.90 |
+| API-Bank | accuracy | 67.52 | 76.97 | 84.45 | 85.43 |
+| BANKING77 | macro-F1 | 85.33 | 88.34 | 92.09 | 92.76 |
+| CLINC150+OOS | macro-F1 | 85.51 | 90.61 | 94.35 | 93.48 |
+| RouterBench † | selected quality (quality objective) | 79.63 | 71.76 | 79.06 | 79.86 |
+| Home appliance simulator | case exact accuracy | 98.86 | 100.00 | 100.00 | 100.00 |
+| SGD/SGD-X † | macro-F1 | 61.15 | 65.16 | 68.10 | 67.60 |
+| ContractNLI | macro-F1 | 84.41 | 84.14 | 85.99 | 86.16 |
+| ANLI | macro-F1 | 58.98 | 65.80 | 75.74 | 77.32 |
+| BPoMP | accuracy | 78.14 | 87.24 | 92.02 | 93.92 |
+| Humicroedit | accuracy | 61.80 | 62.56 | 65.68 | 64.50 |
+| POP909-CL | accuracy | 84.40 | 84.95 | 84.70 | 85.05 |
+| cfcolor | accuracy | 69.02 | 68.70 | 71.26 | 71.38 |
+| MMLU † | accuracy | 56.09 | 65.65 | 77.95 | 81.42 |
+| GPQA Diamond | accuracy | 33.67 | 35.71 | 44.39 | 48.47 |
+| ARC-Easy † | accuracy | 88.97 | 95.50 | 98.19 | 98.82 |
+| ARC-Challenge † | accuracy | 78.16 | 87.54 | 95.05 | 96.08 |
+| WinoGrande | accuracy | 71.11 | 80.82 | 89.66 | 92.74 |
+| HellaSwag | accuracy | 80.85 | 89.44 | 94.81 | 96.28 |
+| GSM8K | accuracy | 49.47 | 57.35 | 69.37 | 71.04 |
+| ChessBench | accuracy | 25.44 | 26.24 | 28.60 | 29.08 |
+| MuSR | accuracy | 62.10 | 70.08 | 71.94 | 74.60 |
+| SATA-Bench | case exact accuracy | 46.48 | 53.33 | 54.61 | 55.70 |
+| SimpleBench † | accuracy | 0.00 | 10.00 | 0.00 | 10.00 |
+| BRIGHT | nDCG@10 | 37.32 | 38.75 | 46.58 | 45.40 |
+| Amazon ESCI | macro-F1 | 56.30 | 57.51 | 60.69 | 61.48 |
+| ACOS | per-review F1 | 51.22 | 52.45 | 56.19 | 56.93 |
+| FinEntity | macro-F1 | 85.65 | 87.77 | 95.53 | 94.83 |
+| iSarcasmEval | Sarcasm F1 · track A, English | 44.64 | 59.44 | 63.76 | 68.02 |
+| VAST | macro-F1 | 76.52 | 77.34 | 78.62 | 78.49 |
+| NLI4CT | macro-F1 | 66.64 | 73.41 | 80.38 | 82.39 |
+| CRUXEval | accuracy | 42.98 | 57.89 | 78.60 | 81.40 |
+| CLadder | accuracy | 89.44 | 87.74 | 91.30 | 94.82 |
+| HLE | accuracy | 14.97 | 13.97 | 11.98 | 11.58 |
+| Habermas Machine | accuracy | 56.80 | 59.13 | 61.93 | 63.13 |
+| PhishNChips phishing decisions | accuracy | 98.20 | 97.85 | 99.30 | 98.90 |
+| MMLU-Pro | accuracy | 38.66 | 47.10 | 60.51 | 64.06 |
+| BBH fixed-option tasks | accuracy | 68.88 | 76.96 | 84.07 | 87.07 |
+| RAGTruth response-level hallucination | F1 on hallucinated class | 79.96 | 83.00 | 85.20 | 85.08 |
+| HoVer claim verification | accuracy | 85.85 | 88.10 | 89.15 | 90.05 |
+| When2Call MCQ | accuracy | 86.39 | 89.40 | 91.59 | 92.06 |
+| New Yorker caption matching | accuracy | 56.25 | 65.91 | 73.67 | 76.52 |
+</details>
+Evaluation uses BF16 native reference inference and the official scorer. Independent re-scoring of the compressed predictions reproduces the results exactly.
+### Inference Performance
+Latest family measurements for the 256-token, single-question fixture:
+| Model | GS1 Default P50 (ms) | CUDA Graph P50 / P95 (ms) | Speedup | Graph allocated (GiB) |
+| --- | ---: | ---: | ---: | ---: |
+| GS1-0.8B | 54.10 | **12.51 / 13.22** | 4.32× | 4.07 |
+| GS1-2B | 53.95 | **18.64 / 18.69** | 2.89× | 10.60 |
+| GS1-4B | 72.87 | **35.17 / 35.24** | 2.07× | 23.68 |
+Single RTX 4090 reporting 48 GiB, BF16 computation, five warmups and 30 timed calls per group. Timing covers the complete in-process decision call, including validation, prompt compilation, model execution, probabilities and typed responses; model loading, graph capture, HTTP and SDK transport are excluded. Speedup compares GS1 Default P50 with CUDA Graph P50 using identical weights and inputs. Latency and memory depend on request shape.
+![Inference Latency](assets/inference-latency.svg)
+## Quickstart
+### Hugging Face Transformers
+Use Python 3.11+ and install the dependencies:
 ```bash
+python -m pip install torch==2.9.1 torchvision==0.24.1 transformers==5.3.0 accelerate safetensors 'pydantic>=2.8,<3' typesafe-sdk==0.7.0 pillow
 ```
+Load the Hub ID or a local model directory and submit all three decision primitives in one call:
 ```python
+import torch
 from transformers import AutoModel
 model = AutoModel.from_pretrained(
+    "bigai/GS1-0.8B",
     trust_remote_code=True,
 ).eval().to("cuda")
+request = {
+    "state": "A customer was charged twice and requests an urgent refund.",
+    "questions": {
         "team": {
             "type": "choice",
+            "instructions": "Which team should handle this ticket?",
+            "criteria": {
+                "billing": "Payments and refunds",
+                "technical": "Software problems"
+            }
+        },
+        "refund": {
+            "type": "noul",
+            "instructions": "Does the customer request a refund?"
         },
         "urgency": {
             "type": "score",
             "instructions": "Rate the urgency of handling this ticket.",
+            "criteria": [
+                "Routine",
+                "Soon",
+                "Immediate"
+            ]
+        }
+    }
+}
+with torch.autocast("cuda", dtype=torch.bfloat16, cache_enabled=False):
+    result = model.decide(**request, precision="bf16")
 print(result["answers"])
+print(result["usage"])
 ```
+The response contains `model`, `answers` and `usage`. Choice preserves candidate IDs and returns every candidate probability; Noul returns `p(true)`; Score returns the expected zero-based rubric level and its full probability distribution. Generated `output_tokens` is zero.
+### GS1 Runtime
+The GS1 runtime adds shared-state reuse, compatible batching, HTTP serving and qualified CUDA Graph replay:
 ```bash
 git clone https://github.com/bigai-nlco/GS1.git
 uv run gs1 serve --model bigai/GS1-0.8B --model-name GS1
 ```
+The service exposes `POST /v1/systemone` and `GET /v1/models` at `http://127.0.0.1:8000`. The [Playground](https://github.com/bigai-nlco/GS1/blob/main/playground/README.md) provides an interactive Query workspace.
+### TypeSafe SDK
+Call the running service through the official SDK:
+```python
+from typesafe_sdk import Choice, Noul, Score, TypeSafeClient
+with TypeSafeClient(
+    base_url="http://127.0.0.1:8000", api_key="local", model="GS1",
+) as client:
+    result = client.system_one(
+        state="A customer was charged twice and requests an urgent refund.",
+        questions={
+            "team": Choice(
+                instructions="Which team should handle this ticket?",
+                criteria={"billing": "Payments and refunds", "technical": "Software problems"},
+            ),
+            "refund": Noul(instructions="Does the customer request a refund?"),
+            "urgency": Score(
+                instructions="Rate the urgency of handling this ticket.",
+                criteria=["Routine", "Soon", "Immediate"],
+            ),
+        },
+    )
+print(result.choices["team"].choice)
+print(result.choices["team"].probabilities)
+print(result.nouls["refund"].noul)
+print(result.scores["urgency"].score)
+```
+Use the API root as `base_url` and match `model` to the server alias configured by `--model-name`. The SDK accepts `api_key="local"` for this local service.
+## License
+Code and released model weights use [Apache-2.0](LICENSE).
+## Citation
+If you find our work helpful, feel free to give us a cite.
 ```bibtex
 @techreport{zheng2026gs1,
   institution={BIGAI},
   year={2026}
 }
+```