Embarrassingly Simple Self-Distillation Improves Code Generation
Paper • 2604.01193 • Published • 56
Dominick Wirzba
Chronuid
AI & ML interests
None yet
Recent Activity
reacted to AbstractPhil's post with 🔥 15 days ago
Massive AlephLM success. The task collective is producing powerful MOE shared knowledge adapters. A serious success and a massive first step towards the next stage. The current family collective results are present here; https://huggingface.co/AbstractPhil/geolip-aleph-qwen/blob/main/exp009_family/results/results.json
This is akin to a stackable non-intrusive lora that enables increased shared collective behavior.
This includes the three mentioned json tasks, a math task, a tinystories task, and a diffusion task for cifar10. Each adapter anchored to the knowledge within model that already exists while enhancing the knowledge through anchored lookup systems and decision-driven hierarchical access trees.
All tasks activate independently upon manual override, all tasks handle direct shared knowledge when left to greedy decoding, each task issued multiple tests alongside to determine fidelity and accuracy throughout the process.
The results show the gating is more than willing to hop from sector to sector, using alternating weight shifts from the cooperative anchored systems - even systems never trained for the tasks contributing to the accuracy of the results for other tasks due to the lookup accuracy to the heuristic chains, never having seen the tasks before. Each structure is independently trained and the collective cooperates together through a dense activation network.
Full writeup and article https://huggingface.co/blog/AbstractPhil/aleph-autoregression-differentiation-ft2. reacted to alexanderbering's post with 🔥 22 days ago
We just put our money where our manifesto is.
Our position has been that the layer of an AI system that touches your data should be open and auditable, not something you rent on trust. So instead of asking you to believe that, we made it runnable.
The ZenBrain Playground is a static HF Space that executes our open-source memory library, @zensation/algorithms (Apache-2.0, zero-dependency), live in your browser. No install, no backend, no mockup: it runs the actual published code. Four panels drive real functions from the library — Ebbinghaus-style retention curves, Hebbian strengthening/decay, sleep-consolidation replay + pruning, and Bayesian retrievability with confidence intervals. The library is vendored into the page, so what you run is the same code that's published on npm.
ZenBrain is a neuroscience-inspired 7-layer memory architecture for AI agents. The open core is the 20-module algorithm library (FSRS, Hebbian, Ebbinghaus, sleep-consolidation, Bayesian confidence, and more). On LongMemEval-500 it reaches 91.3% of a long-context oracle's accuracy at 1/106 of the per-query token budget, and takes the highest mean rank across all 12 system-judge cells (4 systems x 3 LLM judges) against Letta, Mem0, and A-Mem. It ships with 11,589 automated tests. Full details are in the arXiv preprint.
If you build agent memory, we'd genuinely like your eyes on it — open the Space, poke the algorithms, read the code, tell us where it breaks.
Live demo: https://huggingface.co/spaces/zensation-ai/zenbrain-playground
Model: https://huggingface.co/zensation-ai/zenbrain
Paper: arXiv:2604.23878Organizations
upvoted a paper 3 months ago