Gemma-4-31B-it SRT-Sunstone
๐ฎ
3
A read-out that reads images โ trained only on words
River Rider PRO
RiverRider
AI & ML interests
Computational semiotics is empirically proven. It takes three to tango ๐๐ชฉ๐บ
Recent Activity
updated a Space 1 day ago
RiverRider/srt-sunstone posted an update 3 days ago
Train Once, Read Everywhere
Paper title:
Train Once, Read Everywhere: Substrate Invariance of the Linearly Readable Structure in Frozen Language Models
Paper URL:
https://github.com/space-bacon/SRT/blob/main/arxiv_program/paper.md
Repository URL:
https://github.com/space-bacon/SRT
The consolidated findings of the SRT research program are now available.
The program treats frozen production-scale language models as substrates whose internal states carry structure that small, inspectable instruments can read. Results include:
- A ~12 M-parameter adapter that surfaces per-token semiotic signals from a frozen 7 B backbone with zero cross-entropy degradation
- An activation verbalizer that recovers text from single hidden states up to a calibrated paraphrase ceiling
- Linear readout ports spanning dense 3 B models to 94-layer 235 B mixture-of-experts models
- A 22 MB linear head that gives a frozen multimodal chat model image-to-text retrieval performance matching fully trained 2018 dual encoders on the COCO benchmark
The central claim is substrate invariance. The readable structure is a stable property of the model class. A head trained once on one host reads, with no retraining and at most a 42 KB recalibration, across:
- Hosts ten times smaller (31 B โ 3 B)
- 4-bit weight precision
- Entirely different silicon and kernels (CUDA/bf16 to Apple Silicon/MLX-Q4)
Deployment tiers differ in latency and cost, never in capability.
All instruments, measurement protocols, invariance evidence, negative results, and artifacts are in the repository. updated a model 3 days ago
RiverRider/srt-sunstone-linear-headOrganizations
updated a
Space 1 day ago
posted an update 3 days ago
Post
128
Train Once, Read Everywhere
Paper title:
Train Once, Read Everywhere: Substrate Invariance of the Linearly Readable Structure in Frozen Language Models
Paper URL:
https://github.com/space-bacon/SRT/blob/main/arxiv_program/paper.md
Repository URL:
https://github.com/space-bacon/SRT
The consolidated findings of the SRT research program are now available.
The program treats frozen production-scale language models as substrates whose internal states carry structure that small, inspectable instruments can read. Results include:
- A ~12 M-parameter adapter that surfaces per-token semiotic signals from a frozen 7 B backbone with zero cross-entropy degradation
- An activation verbalizer that recovers text from single hidden states up to a calibrated paraphrase ceiling
- Linear readout ports spanning dense 3 B models to 94-layer 235 B mixture-of-experts models
- A 22 MB linear head that gives a frozen multimodal chat model image-to-text retrieval performance matching fully trained 2018 dual encoders on the COCO benchmark
The central claim is substrate invariance. The readable structure is a stable property of the model class. A head trained once on one host reads, with no retraining and at most a 42 KB recalibration, across:
- Hosts ten times smaller (31 B โ 3 B)
- 4-bit weight precision
- Entirely different silicon and kernels (CUDA/bf16 to Apple Silicon/MLX-Q4)
Deployment tiers differ in latency and cost, never in capability.
All instruments, measurement protocols, invariance evidence, negative results, and artifacts are in the repository.
Paper title:
Train Once, Read Everywhere: Substrate Invariance of the Linearly Readable Structure in Frozen Language Models
Paper URL:
https://github.com/space-bacon/SRT/blob/main/arxiv_program/paper.md
Repository URL:
https://github.com/space-bacon/SRT
The consolidated findings of the SRT research program are now available.
The program treats frozen production-scale language models as substrates whose internal states carry structure that small, inspectable instruments can read. Results include:
- A ~12 M-parameter adapter that surfaces per-token semiotic signals from a frozen 7 B backbone with zero cross-entropy degradation
- An activation verbalizer that recovers text from single hidden states up to a calibrated paraphrase ceiling
- Linear readout ports spanning dense 3 B models to 94-layer 235 B mixture-of-experts models
- A 22 MB linear head that gives a frozen multimodal chat model image-to-text retrieval performance matching fully trained 2018 dual encoders on the COCO benchmark
The central claim is substrate invariance. The readable structure is a stable property of the model class. A head trained once on one host reads, with no retraining and at most a 42 KB recalibration, across:
- Hosts ten times smaller (31 B โ 3 B)
- 4-bit weight precision
- Entirely different silicon and kernels (CUDA/bf16 to Apple Silicon/MLX-Q4)
Deployment tiers differ in latency and cost, never in capability.
All instruments, measurement protocols, invariance evidence, negative results, and artifacts are in the repository.
updated 2
models 3 days ago
published a
model 3 days ago
published a
model 8 days ago
updated a
model 8 days ago
liked a
model 10 days ago
reacted to Quazim0t0's post with ๐ฅ 14 days ago
Post
5034
๐ผ DaisyChain-Web: train a language model with friends or by yourself with multiple devices, in the browser, no install
Open a webpage, share a room link, and every device that joins becomes part of the training cluster. Phones, laptops, old PCs: they connect peer-to-peer over WebRTC and train one shared transformer together, entirely in the browser.
What's actually happening under the hood:
๐ง A mini transformer LM trains on FineWeb-Edu, streamed live from the HuggingFace Hub. Each device pulls its own slice (data parallelism), tokenized with our 16.5k-token Spikewhale tokenizer
โก Every single multiply runs through verified INT8 neural units, no float fallback. On WebGPU browsers it uses the GPU's DP4A integer dot-product hardware, admitted only after proving bit-identical results against the verified units, with a 3รINT8 fast-accurate scheme (CUTLASS's 3xTF32 trick, ported to 8-bit)
๐ Devices average gradients every step under a sync guard: a per-step roster protocol plus weight-hash verification keeps every device's model bit-identical. If anything drifts, training stops instead of silently forking
๐ Live logs show exactly what every device contributes, step by step
๐พ When you're done: test generations right on the page, download a checkpoint, or grab the inference kit, a single self-contained HTML file with the weights baked in that runs generations offline, anywhere
Works solo too. Every extra device just grows the effective batch.
๐ Try it: Quazim0t0/DaisyChain-Web
๐ Training framework: DaisyChainAI/DaisyChain-Train
Updates:
- Block-scaled INT8 quantization
- Batched attention GEMM
- Fused dequant+ReLU epilogue
- Weight-tied unembedding
- WebSocket relay fallback
- Server keepalive ping/pong every 30s
- disconnected-state redial
- Visibility/network-change reconnect
Open a webpage, share a room link, and every device that joins becomes part of the training cluster. Phones, laptops, old PCs: they connect peer-to-peer over WebRTC and train one shared transformer together, entirely in the browser.
What's actually happening under the hood:
๐ง A mini transformer LM trains on FineWeb-Edu, streamed live from the HuggingFace Hub. Each device pulls its own slice (data parallelism), tokenized with our 16.5k-token Spikewhale tokenizer
โก Every single multiply runs through verified INT8 neural units, no float fallback. On WebGPU browsers it uses the GPU's DP4A integer dot-product hardware, admitted only after proving bit-identical results against the verified units, with a 3รINT8 fast-accurate scheme (CUTLASS's 3xTF32 trick, ported to 8-bit)
๐ Devices average gradients every step under a sync guard: a per-step roster protocol plus weight-hash verification keeps every device's model bit-identical. If anything drifts, training stops instead of silently forking
๐ Live logs show exactly what every device contributes, step by step
๐พ When you're done: test generations right on the page, download a checkpoint, or grab the inference kit, a single self-contained HTML file with the weights baked in that runs generations offline, anywhere
Works solo too. Every extra device just grows the effective batch.
๐ Try it: Quazim0t0/DaisyChain-Web
๐ Training framework: DaisyChainAI/DaisyChain-Train
Updates:
- Block-scaled INT8 quantization
- Batched attention GEMM
- Fused dequant+ReLU epilogue
- Weight-tied unembedding
- WebSocket relay fallback
- Server keepalive ping/pong every 30s
- disconnected-state redial
- Visibility/network-change reconnect
- 11 replies
liked 2
Spaces 19 days ago
updated a
dataset 19 days ago
published a
dataset 19 days ago
published a
model 19 days ago
posted an update 24 days ago
Post
153
๐ฎ Gemma-4-31B-it SRT-Sunstone
A read-out that reads images โ trained only on words.
A 12.3M side-channel head on a frozen google/gemma-4-31B-it, taught meaning from text alone โ to separate discourse communities in the residual stream. It never saw a single picture in training. Hand it a picture and it names what the image means, with zero image training.
Cross-modal transfer. Give it a photo and it lands next to the right words: bicycle โ bicycle, rose โ flower, dog โ pet. It groups images by what they mean, not how they look: imageโreferent kNN 0.64 against chance 0.10.
Why this is a semiotic read-out, not an image classifier. A classifier is trained on labelled images and learns a fixed map from pixels to a closed label set; it only knows the labels it was shown. This read-out is different in kind. It never saw an image in training โ it was trained only on text, to separate discourse communities in the residual stream of a frozen gemma-4-31B. It can interpret a picture because gemma-4 already fuses image and word into one representational stream, and the read-out taps the shared interpretant: the meaning a sign carries, whatever form it arrived in. So it does not classify the image. It tells you what the image means to a system that learned meaning from words โ a transfer across modality, from a head that was never cross-trained. That is the result.
Each picture gets two readings: the words it means โ the load-bearing evidence โ and the discourse it evokes, the nearest of 35 communities it learned from text. Read the second as a flavour, not a category: cars into the automotive community, deer and mushrooms into gardening, cats and dogs into the cozy-domestic communities. Never a class label.
Scored offline through a frozen google/gemma-4-31B-it (62.5 GB, too large to run live)
Try it: RiverRider/srt-sunstone
Model: RiverRider/Gemma-4-31B-it-SRT-Sunstone
Code: https://github.com/space-bacon/SRT
A read-out that reads images โ trained only on words.
A 12.3M side-channel head on a frozen google/gemma-4-31B-it, taught meaning from text alone โ to separate discourse communities in the residual stream. It never saw a single picture in training. Hand it a picture and it names what the image means, with zero image training.
Cross-modal transfer. Give it a photo and it lands next to the right words: bicycle โ bicycle, rose โ flower, dog โ pet. It groups images by what they mean, not how they look: imageโreferent kNN 0.64 against chance 0.10.
Why this is a semiotic read-out, not an image classifier. A classifier is trained on labelled images and learns a fixed map from pixels to a closed label set; it only knows the labels it was shown. This read-out is different in kind. It never saw an image in training โ it was trained only on text, to separate discourse communities in the residual stream of a frozen gemma-4-31B. It can interpret a picture because gemma-4 already fuses image and word into one representational stream, and the read-out taps the shared interpretant: the meaning a sign carries, whatever form it arrived in. So it does not classify the image. It tells you what the image means to a system that learned meaning from words โ a transfer across modality, from a head that was never cross-trained. That is the result.
Each picture gets two readings: the words it means โ the load-bearing evidence โ and the discourse it evokes, the nearest of 35 communities it learned from text. Read the second as a flavour, not a category: cars into the automotive community, deer and mushrooms into gardening, cats and dogs into the cozy-domestic communities. Never a class label.
Scored offline through a frozen google/gemma-4-31B-it (62.5 GB, too large to run live)
Try it: RiverRider/srt-sunstone
Model: RiverRider/Gemma-4-31B-it-SRT-Sunstone
Code: https://github.com/space-bacon/SRT
updated a
model 24 days ago
published a
model 24 days ago
published a
Space 24 days ago
replied to ginigen-ai's post 24 days ago
Thanks @NovusEdge the work is understood by few so I am thrilled you get it.
It performs extremely well semantically and beyond across communities.
Check 11.5.2 in paper nla
https://github.com/space-bacon/SRT/blob/main/paper_nla.md
liked a
Space 25 days ago