[bugf][act-halting][gate halted positions from weight accumulation][bugf][moe-router-bias][stop
Kye Gomez
289981b
load balance bias gradient leak][bugf][act-cache-consistency][keep all loops with kv
cache][bugf][lora-depth-extrapolation][clamp scale index beyond max
loops][improvement][pyproject-version][bump version to 0 4 0]
[fix][rope Every decode token was stuck at position 0, so <q_decoded, k_cached> lost the (n - m) term entirely]
Kye Gomez
18cca89
use gpt-oss tokenizer because it's a great tokenizer
Kye Gomez
97bc414
[feat][dropout][add dropout to config attn and
Kye Gomez
5cfef74
residuals][docs][datasets][add recommended training datasets
doc][docs][readme-datasets][link datasets doc in readme]
clean up
Kye Gomez
e522ecf
variants
Kye Gomez
9cce1c1
[bugf][lora-b-init][fix zero-init B making adapter always output zero][bugf][lti-get-a][fix 0 times
Kye Gomez
806a8da
inf NaN in log space computation][improvement][rope-theta-test][exclude degenerate dim0 from theta
angle comparison]
