j0no12 commited on
Commit
c8b42b3
·
verified ·
1 Parent(s): 0b9b53f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -3
README.md CHANGED
@@ -40,9 +40,7 @@ Sol Nano is a 2,895,188-parameter language model trained on 5 billion tokens. It
40
  | Optimizer updates | 19,074 |
41
  | Weights | FP32 safetensors |
42
 
43
- The blocks use causal grouped-query attention, RoPE, QK normalization, and loop conditioning. Token embeddings share weights with the output head. TN-Gram provides factorized local memory for orders 2-5. XSA and Engram are disabled.
44
-
45
- Attention runs through compiled Triton FlexAttention. The backend does not use PyTorch SDPA.
46
 
47
  ## Use
48
 
 
40
  | Optimizer updates | 19,074 |
41
  | Weights | FP32 safetensors |
42
 
43
+ The blocks use causal grouped-query attention, RoPE, QK normalization, and loop conditioning. Token embeddings share weights with the output head. TN-Gram provides factorized local memory for orders 2-5.
 
 
44
 
45
  ## Use
46