davidhouseholter commited on
Commit
e218bcc
·
verified ·
1 Parent(s): a18affd

Note NeurIPS Reject provenance in GRT-10M model card

Browse files
Files changed (1) hide show
  1. README.md +6 -3
README.md CHANGED
@@ -5,6 +5,7 @@ tags:
5
  - geometric-rotation-transformer
6
  - causal-lm
7
  - rotational
 
8
  datasets:
9
  - nvidia/Nemotron-Pretraining-Dataset-sample
10
  language:
@@ -13,15 +14,17 @@ library_name: transformers
13
  pipeline_tag: text-generation
14
  ---
15
 
16
- # GRT-10M
 
 
17
 
18
  A Geometric Rotation Transformer pretrained on Nemotron. GRT replaces the dense
19
  linear projections of a standard transformer with cascades of learned Givens
20
  rotations on SO(D), so the residual path contains no `nn.Linear` at all.
21
 
22
- This is a research proof-of-concept at small scale. It is published to
23
  demonstrate that the architecture trains stably and evaluates end-to-end, not
24
- as a capable language model.
25
 
26
  ## Model Details
27
 
 
5
  - geometric-rotation-transformer
6
  - causal-lm
7
  - rotational
8
+ - neurips-reject
9
  datasets:
10
  - nvidia/Nemotron-Pretraining-Dataset-sample
11
  language:
 
14
  pipeline_tag: text-generation
15
  ---
16
 
17
+ # GRT-10M (NeurIPS Reject Edition)
18
+
19
+ > **Status: Official NeurIPS Reject.** Originally built as the small-scale proof-of-concept for our rejected NeurIPS submission introducing the Geometric Rotation Transformer (`GRT`). Kept intact as our historical `8.7M`-parameter (`2.2M` non-embedding backbone) baseline checkpoint—and now evaluated across the 10-task `SLM-Bench` suite with our classical symplectic & Huygens curriculum optimizers (`SymplecticAdam` and `HuygensSymplecticAdam`).
20
 
21
  A Geometric Rotation Transformer pretrained on Nemotron. GRT replaces the dense
22
  linear projections of a standard transformer with cascades of learned Givens
23
  rotations on SO(D), so the residual path contains no `nn.Linear` at all.
24
 
25
+ This is a research proof-of-concept at small scale (and a proud NeurIPS reject). It is published to
26
  demonstrate that the architecture trains stably and evaluates end-to-end, not
27
+ as a capable general-purpose assistant.
28
 
29
  ## Model Details
30