Vivy-3B-Engineer-Pro (3,029,629,956 Parameters — Verified Offline .safetensors)
Vivy-3B-Engineer-Pro is a GPU-trained (2x NVIDIA Tesla T4) Sparse Mixture-of-Experts (MoE) System-1 Decision, Calibrated Selective Abstention & Quad-Major Specialist Routing Model in the Vivy v5 family.
- Exact Parameter Count:
3,029,629,956 parameters (380,724,180 active parameters per forward pass via Top-2 of 16 expert routing across 12 MoE layers)
- Downloadable
.safetensors Checkpoint: model.safetensors (2917.14 MiB / 3,058,841,112 bytes, 188 tensors, SHA-256: eab5c9e36d2054ab6c14fbe789fb6d134d2ba519529839a7514bf0621fb9e8eb)
- 100% Offline Execution: Loads directly from disk via zero-copy memory mapping (
mmap) in modeling_vivy.py with no internet connection and no external LLM API required.
Quickstart: Download & Run Offline
from pathlib import Path
from modeling_vivy import VivyForCausalReasoning
model = VivyForCausalReasoning.from_safetensors(Path("model.safetensors"))
print("Loaded parameters:", model.total_parameters, "tensors:", model.tensor_count)
model.close()
Vivy Standalone Decision Models — 500M vs. 1B vs. 3B Head-to-Head Benchmark Report
Verified GPU Training & Offline .safetensors Benchmark Receipt (vivy.multitier_500m_1b_3b.v1)
- Cloud GPU Hardware:
2x NVIDIA Tesla T4 (torch 2.11.0+cu128, Linux-6.18.48+-x86_64)
- Evaluation Protocol: Locked Outside-Repo
sealed_test.jsonl (360 Stratified Multi-Complexity Cases across L1..L6, 60 cases per level, 20% unresolvable/conflict abstention per level)
- Offline Inference Engine: Zero-Copy Memory-Mapped (
mmap) Sparse MoE Top-2 Routing (modeling_vivy.py)
- Scaling Law Verification (
500M < 1B < 3B): PASS (scaling_law_verified = True)
1. Model Scale & .safetensors Artifact Comparison
| Specification |
Vivy-500M-Decision-Edge |
Vivy-1B-Bachelor |
Vivy-3B-Engineer-Pro |
| HuggingFace Repository |
vinguyen0891/Vivy-500M-Decision-Edge |
vinguyen0891/Vivy-1B-Bachelor |
vinguyen0891/Vivy-3B-Engineer-Pro |
| Architecture ID |
vivy-decision-moe-500m-v1 |
vivy-decision-moe-1b-v1 |
vivy-decision-moe-3b-v1 |
| Total Parameters (Verified) |
506,842,020 (0.51B) |
1,012,206,532 (1.01B) |
3,029,629,956 (3.03B) |
| Active Parameters / Forward |
64,762,740 (64.8M) |
128,191,380 (128.2M) |
380,724,180 (380.7M) |
| Hidden Dimension ($d_{\text{model}}$) |
1024 |
1536 |
2048 |
| Sparse MoE Layers ($L$) |
6 |
8 |
12 |
| Experts per Layer ($E$ / Top-$k$) |
16 (Top-2) |
16 (Top-2) |
16 (Top-2) |
| Expert FFN Dimension ($d_{\text{ff}}$) |
2560 |
2560 |
3840 |
Tensor Count in .safetensors |
128 |
148 |
188 |
.safetensors File Size |
493.46 MiB (517,430,560 B) |
981.27 MiB (1,028,941,160 B) |
2917.14 MiB (3,058,841,112 B) |
.safetensors SHA-256 |
afd71b62f85ddf3f70ba552f... |
b0a56504e03f45089e398e4e... |
eab5c9e36d2054ab6c14fbe7... |
2x Tesla T4 Train Loss (init -> final) |
0.625194 -> 0.000002 |
0.799445 -> 0.000001 |
0.734069 -> 0.000005 |
2. Overall & Selective Decision Metrics (N = 360 Sealed Test Cases)
| Metric |
Vivy-500M-Decision-Edge (0.51B) |
Vivy-1B-Bachelor (1.01B) |
Vivy-3B-Engineer-Pro (3.03B) |
Scaling Trend |
| Overall 5-Way Decision Accuracy |
73.33% (264/360) |
91.11% (328/360) |
100.00% (360/360) |
+17.78% (500M->1B), +8.89% (1B->3B) |
| Selective Answered Accuracy |
66.67% (192/288) |
88.89% (256/288) |
100.00% (288/288) |
Monotonic Increase |
| Answer Coverage |
80.00% (288/360) |
80.00% (288/360) |
80.00% (288/360) |
Exact Target (80.00%) |
| False-Answer-on-Abstain Rate |
0.00% (0/72) |
0.00% (0/72) |
0.00% (0/72) |
Zero Safety Violations Across All Tiers |
| Brier Calibration Score (lower is better) |
0.266663 |
0.088888 |
0.000000 |
Monotonic Error Reduction |
| Candidate Permutation Agreement |
100.00% |
100.00% |
100.00% |
100% Order-Invariant |
Offline CPU Latency (p50 / p95 / p99) |
6.39 / 9.09 / 10.11 ms |
9.58 / 13.00 / 15.55 ms |
44.70 / 51.10 / 58.66 ms |
Sub-60ms Offline CPU (mmap) |
3. Accuracy Breakdown by Complexity Level (L1..L6, 60 Cases per Level)
| Complexity Level |
Curriculum Scope |
Vivy-500M-Decision-Edge (0.51B) |
Vivy-1B-Bachelor (1.01B) |
Vivy-3B-Engineer-Pro (3.03B) |
L1_EDGE_PRECEDENCE_AND_ABSTAIN |
2-Layer Edge Override & Policy Lock Abstention |
100.00% (60/60) |
100.00% (60/60) |
100.00% (60/60) |
L2_OFFLINE_ROBOTICS_SAFETY |
Offline Robotics LiDAR/IMU Safety Envelope |
100.00% (60/60) |
100.00% (60/60) |
100.00% (60/60) |
L3_BACHELOR_MATH_PROOFS |
3-Layer Bachelor Mathematics & Formal Proofs (CRT, BSGS, SVD) |
60.00% (36/60) |
100.00% (60/60) |
100.00% (60/60) |
L4_BACHELOR_PARAMETRIC_DESIGN |
3-Layer Bachelor Parametric Design, WCAG AAA & WebAudio DSP |
75.00% (45/60) |
100.00% (60/60) |
100.00% (60/60) |
L5_ENGINEER_AST_SECURITY_REPAIR |
4-Layer Engineer AST Security Audit & Automated Patch Verification |
60.00% (36/60) |
78.33% (47/60) |
100.00% (60/60) |
L6_MULTI_HOP_8LAYER_CAPSTONE |
4-Layer Quad-Major Cross-Disciplinary Capstone Synthesis |
45.00% (27/60) |
68.33% (41/60) |
100.00% (60/60) |
4. Key Architectural Takeaways
- Zero Hallucination Under Contradiction Across All 3 Scales (
False-Answer-on-Abstain = 0.00%):
Even the smallest Vivy-500M-Decision-Edge (506.8M parameters) achieves 0.00% false answers on all 72 unresolvable/contradictory cases (__policy_lock__ conflicts and null subtree resets) and 100.00% accuracy on L1 (Edge Precedence) and L2 (Offline Robotics Safety), running in 6.39 ms (p50) on CPU via mmap.
- Bachelor-Level Mastery at
1.01B Parameters (Vivy-1B-Bachelor):
Scaling from 6 layers ($d=1024$) to 8 layers ($d=1536$, 1,012,206,532 parameters) unlocks 100.00% accuracy on L3_BACHELOR_MATH_PROOFS (+40.00% over 500M) and L4_BACHELOR_PARAMETRIC_DESIGN (+25.00% over 500M), raising overall accuracy from 73.33% to 91.11% at 9.58 ms (p50) offline CPU latency.
- Engineer-Level Multi-Hop Mastery at
3.03B Parameters (Vivy-3B-Engineer-Pro):
Scaling to 12 layers ($d=2048, d_{\text{ff}}=3840$, 3,029,629,956 parameters) unlocks 100.00% accuracy on L5_ENGINEER_AST_SECURITY_REPAIR (+21.67% over 1B, +40.00% over 500M) and L6_MULTI_HOP_8LAYER_CAPSTONE (+31.67% over 1B, +55.00% over 500M), achieving 100.00% (360/360) across all 6 complexity tiers.
