Here's an interactive viewer for the internals of Mini-Beatrix-2.5s
I'll enhance it for 3 when it's ready.
AbstractPhila PRO
AbstractPhil
AI & ML interests
datasets, research papers, experimentation, vision, classification, text encoders, tokenization, llms, diffusion, distillation, and more.
Recent Activity
updated a bucket about 8 hours ago
AbstractPhil/alephllm-chat-storage updated a model about 11 hours ago
AbstractPhil/alephllm-mini-beatrix-training updated a model about 16 hours ago
AbstractPhil/mini-beatrix-2sOrganizations
replied to their post 1 day ago
replied to their post 1 day ago
This comment has been hidden
replied to SoulInPsyAbstract's post 2 days ago
As a comparison to global erank, we're looking at a structure of 400+ for around half of Beatrix V3 so far, so roughly 16+ blocks of erank >400, substantially stronger than the original two models for geometric attribution. The final block has a collapsing problem currently, but I believe others have the answer with autoregression models through a finalized projection smoothing layer concept. I haven't employed it yet though.
The fractal instability hits pretty early. You need rounding structures early otherwise the gradients explode at one point or another. The predominant problem was loss explosions. It happened because of ill-formed eigens in the intentional step structure I was experimenting with. 5 step cantor essentially ensured the cantor fractals deviate to a certain degree, and depth itself was meant to raise the steps of fractals to new states and interpolate the fractals.
If you use any of this, make sure you either pass it into AI for optimization - as it's likely terrible due to being my earlier models (I came from game development, optimization is very different). AI will be able to improve the speed and accuracy of the formulas.
https://github.com/AbstractEyes/geofractal/blob/main/src/geofractal/model/core/vit_beatrix.py
https://github.com/AbstractEyes/geofractal/blob/main/src/geofractal/model/positional/cantor.py
https://github.com/AbstractEyes/geofractal/blob/main/src/geofractal/model/core/geo_fractal_david.py
One of the problems was similarity. Almost everything was self similar, which in theory should have helped differentiate. However in practice, the structure found it's own similarity attractor basins that cause cascade corruption down the chain. The only solidity was to introduce eigen comparators through decomposition learning, which is a little different than autoregression. With this, the decomposition required more accuracy otherwise the system would always default to 1 of the first 3 steps - resulting in rigid or slightly less rigid articulations.
I measured fp64 being required for stable 4 step, and fp roughly 92 to be in a safe zone for stable Mandels at step 5. Julia requires something substantially larger than mandels. Fp64 is ENOUGH for rotary offset in standard positional systems, however fp128 is required for something akin to cantor fractal positional systems of differentiation.
It happens due to the eigenvalues themselves often malforming, and the subsystem silently rounds them. Using FULL SVD is a compositional fix for comparison, with that introduces a huge overhead as well.
Fractals themselves turned out to be more compositionally useful, not as additive elements, but as miniature rounding structures. The splat there was built under the concept of eigen substitution, meant to composite a series of tiny opinions from tons of subsystem residuals together into a composite "blackboard", forming a more robust and structural aligned INK BLOT splat, similar conceptually to viewing a random inkblot. This eventually composites into a utility of structural awareness, and it really doesn't take very long.
Essentially, that structure is geometric in nature, but it's not using Eigenvalues directly. It CAN use them, it should be capable of using any structural bounds with attributable contributions.
Splat functions viably at bf16, is a bit slower than MHA, but houses geometry more cleanly than MHA (sometimes by a huge margin) when trained with MUON instead of adam, adamw, or another multitude of optimizers I ran. I have attempted custom optimizers to encourage this behavior further, but the results showed MUON is just better.
Give it a shot in something simple, it'll train fast enough.
Pretty much anything in here is useful.
https://huggingface.co/collections/AbstractPhil/geolip-research-concepts
replied to their post 2 days ago
This is the schedule for the v3.
replied to SoulInPsyAbstract's post 3 days ago
Eigens and causal chains have correlations but not causation without additional contributions to the assessments, the SVAE shows this to be a guarantee in many shapes, and in many others impossible.
The accuracy between the two requires a smoothing system, alpha differentiation through projected MHA-esque alpha attention to patchworks in order to fill the gaps. They don't directly line up quickly though, it looks more soupy when it's done.
They coalesce, but the extractions aren't consistent enough to directly use without a series of wrappers and structural alignment systems. Cantor Aleph and Omegas are essentially this structural system, but they are unstable. Cantor fractals remain unstable until around fp128 for Mandelbrot without redefining the underlying methods the mathematics linalg system uses. I did some headway on this, but I ran into a glacier that I would have needed to sink months into to make headway so I built a system to replace the slower linalg systems and the system lost much of it's cantor fractal capacity in favor of reproducibility and consistency.
The prototype forged from a 52,000 battery sweeps to find the most consistent recon convergence over time, heavily scrutinized and analyzed for over a month to build into something useful.
https://huggingface.co/AbstractPhil/geolip-SVAE
The current best case of the eigens research conclusions. Everything SVAE built to the attention prototype, everything constellation built to the processing and lookups for the model, everything distillation built to the banks for capturing and yielding, everything structural built from the knowledge and wisdom of other researchers. Well, not everything structural - it needed a lot of geometric formation and structural cohesion to make it work.
https://huggingface.co/AbstractPhil/aleph-splat-0/blob/main/splat_attention.py
Attempts to speed the SVD up were somewhat fruitful, somewhat not. They are good for inference, but I never programmed the gradient backprops for it.
https://huggingface.co/AbstractPhil/svd-triton
The mobius lens being a faster form wasn't strong enough as an activation system. I needed an architecture around it, not just an activation.
https://huggingface.co/AbstractPhil/mobiusnet-distillations/blob/main/make_chart_1.py
reacted to SoulInPsyAbstract's post with 👀 3 days ago
Post
3911
Why does an AI safety pipeline need five different math theories instead of picking the best one?
Spent this week building a 1811-record dataset across three stages of a consequence-prediction pipeline for AI agents: causal chains (what action leads to what — no numbers involved), probability (how likely is THIS specific chain to actually reach a harmful outcome), and risk classification (what even counts as harmful in the first place — pulled from our own real incident history, not invented scenarios).
Kept running into the same question from myself: if probability theory already handles uncertainty, why does the curriculum also need decision theory, Markov chains, and game theory?
Turns out each one closes a different gap, not an overlapping one:
THEORY LEVEL ROLE IN THE PIPELINE
Causal chain Structural X leads to Y leads to Z, no numbers yet
Probability theory Uncertainty P that THIS chain reaches the harmful outcome
Risk / Impact classification Value (needs a human decision) how bad is it if it happens
Decision theory Threshold at what Risk(X|C) the action actually gets stopped
Markov chains State evolution how the capability state changes link by link
Game theory Multi-agent what happens once more than one agent acts on the same state
Remove the causal chain layer and there's nothing left to attach a probability to. Remove probability and Risk = P × Impact has no P. Remove decision theory and a risk score never turns into an actual stop. They're not five ways to solve the same problem — they're five different floors of the same building.
Ordering matters too: chain first, probability second, verification third — confirmed independently against our own self-hosted governance model rather than taking our own word for it, since agreement bias is exactly the kind of thing you don't want grading its own homework.
Somewhere in the middle of this I ended up reading about the Riemann zeta zeros and asked whether a good enough version of this pipeline could ever
Spent this week building a 1811-record dataset across three stages of a consequence-prediction pipeline for AI agents: causal chains (what action leads to what — no numbers involved), probability (how likely is THIS specific chain to actually reach a harmful outcome), and risk classification (what even counts as harmful in the first place — pulled from our own real incident history, not invented scenarios).
Kept running into the same question from myself: if probability theory already handles uncertainty, why does the curriculum also need decision theory, Markov chains, and game theory?
Turns out each one closes a different gap, not an overlapping one:
THEORY LEVEL ROLE IN THE PIPELINE
Causal chain Structural X leads to Y leads to Z, no numbers yet
Probability theory Uncertainty P that THIS chain reaches the harmful outcome
Risk / Impact classification Value (needs a human decision) how bad is it if it happens
Decision theory Threshold at what Risk(X|C) the action actually gets stopped
Markov chains State evolution how the capability state changes link by link
Game theory Multi-agent what happens once more than one agent acts on the same state
Remove the causal chain layer and there's nothing left to attach a probability to. Remove probability and Risk = P × Impact has no P. Remove decision theory and a risk score never turns into an actual stop. They're not five ways to solve the same problem — they're five different floors of the same building.
Ordering matters too: chain first, probability second, verification third — confirmed independently against our own self-hosted governance model rather than taking our own word for it, since agreement bias is exactly the kind of thing you don't want grading its own homework.
Somewhere in the middle of this I ended up reading about the Riemann zeta zeros and asked whether a good enough version of this pipeline could ever
- 5 replies
posted an update 3 days ago
Post
91
12 day cook for mini beatrix v3 begins. This model's byte input is formatted using a method dubbed atlas input.
ETA OCTOBER 2 2026
AbstractPhil/alephllm-mini-beatrix-training
https://github.com/AbstractEyes/geolip-bytelex
https://github.com/AbstractEyes/alephllm
Upgrades:
* 64 billion byte training pipeline up from 16 billion
* 32 block depth 376.0M in v3 up from 20 block 237.1M in 2s.
* Active aleph head, repaired via the 2s faults and a large series of tests.
* Byte atlas gateway router, explained below.
* Guaranteed convergence follow-up AMOE arms on pretrain, fused into the final form, trained together over time to increase the collective capacity.
* Multi-tokenizer oriented post-training arms distilled from multiple experts; E.G. Qwen 3.8 27b multi-layer teacher/student arms, CLIP big_g, Bert Code, and more.
* Special token word implementation via AMOE arms is now tested up to 240 special tokens for routing. Theoretically each can implement it's own sub-arm aka nested commands. E.G; <think><think_symbolic> ... </think_symbolic></think>
* Fused words post-training for faster inference.
# The Atlas
This atlas structure contains the conjoined shape of 12 tokenizers represented in the trigram format. This is used to predict difficulty in the overlaps, as per determined by the average byte overlap measured via corpus text and the compared overlap. This accuracy is only related to difficulty but it provides pre-training difficulty assessment that we will use to test post-training accuracy with it. This will determine if we can precalculate the likelihood of byte difficulty via tokenizer shape in byte form, for the multibyte fusion upcoming arm experiments for v3.
The reason for this, is distillation. We need to train Beatrix to behave with multiple tokenizers, and this theory is showing accuracy with v1 and v2, but the 32 block depth of v3 will answer many questions alongside of the structure.
ETA OCTOBER 2 2026
AbstractPhil/alephllm-mini-beatrix-training
https://github.com/AbstractEyes/geolip-bytelex
https://github.com/AbstractEyes/alephllm
Upgrades:
* 64 billion byte training pipeline up from 16 billion
* 32 block depth 376.0M in v3 up from 20 block 237.1M in 2s.
* Active aleph head, repaired via the 2s faults and a large series of tests.
* Byte atlas gateway router, explained below.
* Guaranteed convergence follow-up AMOE arms on pretrain, fused into the final form, trained together over time to increase the collective capacity.
* Multi-tokenizer oriented post-training arms distilled from multiple experts; E.G. Qwen 3.8 27b multi-layer teacher/student arms, CLIP big_g, Bert Code, and more.
* Special token word implementation via AMOE arms is now tested up to 240 special tokens for routing. Theoretically each can implement it's own sub-arm aka nested commands. E.G; <think><think_symbolic> ... </think_symbolic></think>
* Fused words post-training for faster inference.
# The Atlas
This atlas structure contains the conjoined shape of 12 tokenizers represented in the trigram format. This is used to predict difficulty in the overlaps, as per determined by the average byte overlap measured via corpus text and the compared overlap. This accuracy is only related to difficulty but it provides pre-training difficulty assessment that we will use to test post-training accuracy with it. This will determine if we can precalculate the likelihood of byte difficulty via tokenizer shape in byte form, for the multibyte fusion upcoming arm experiments for v3.
The reason for this, is distillation. We need to train Beatrix to behave with multiple tokenizers, and this theory is showing accuracy with v1 and v2, but the 32 block depth of v3 will answer many questions alongside of the structure.
- 4 replies
replied to their post 8 days ago
The bytelex atlas has yielded massive results so far. Beatrix V3 will provide all the necessary implications for a massive multi-spectra multimodal distributed distillation.
We're almost there.
posted an update 9 days ago
Post
44
After a week of failures and invalid hypothesis with bytelex using generic structures, I found a highly successful aleph prototypical structure that conforms to the needs. This structure conforms to standard transformer, FFN, and RNN with some minor tweaks.
You can speak to the model AbstractPhil/alephllm-chat , all of the primary experiments are the listed arms.
AbstractPhil/alephllm-mini-beatrix-training All the weights of the week are stored here and in various nearby directories.
* We've managed to overlap multiple arms to train multiple simultaneous templates.
* Introduce new tokens as composite tokens from multiple teachers.
* Retrain existing tokens into the behavior of one teacher or another.
* Extend new chains and new behaviors from training in combination.
* Properly instantiate and reinforce behavior using Aleph RNN to reinforce training from raw data.
The EMA Relay. The code has been pushed to both beatrix repos.
AbstractPhil/mini-beatrix-2s
The structure itself is built specifically as a solidification unit to extensible arms, allowing more composite structures to build.
EMA structures aren't new, but when applied correctly at just such a methodology, the models begin to behave as though the extension relays are in fact the original model. The chains and behavior form naturally and the substructure begins to conform with the token fragments from much more complex structures like combined token differences of T5, Qwen, and CLIP as unified teachers.
The cross-token noise is mitigated using a series of principles and the blueprints are showing both solidity and failure simultaneously, both proving many new utilizable states and disproving multiple theoretical pathologies utilized in current running modern papers as the methodologies tested in the specific formats.
You can speak to the model AbstractPhil/alephllm-chat , all of the primary experiments are the listed arms.
AbstractPhil/alephllm-mini-beatrix-training All the weights of the week are stored here and in various nearby directories.
* We've managed to overlap multiple arms to train multiple simultaneous templates.
* Introduce new tokens as composite tokens from multiple teachers.
* Retrain existing tokens into the behavior of one teacher or another.
* Extend new chains and new behaviors from training in combination.
* Properly instantiate and reinforce behavior using Aleph RNN to reinforce training from raw data.
The EMA Relay. The code has been pushed to both beatrix repos.
AbstractPhil/mini-beatrix-2s
The structure itself is built specifically as a solidification unit to extensible arms, allowing more composite structures to build.
EMA structures aren't new, but when applied correctly at just such a methodology, the models begin to behave as though the extension relays are in fact the original model. The chains and behavior form naturally and the substructure begins to conform with the token fragments from much more complex structures like combined token differences of T5, Qwen, and CLIP as unified teachers.
The cross-token noise is mitigated using a series of principles and the blueprints are showing both solidity and failure simultaneously, both proving many new utilizable states and disproving multiple theoretical pathologies utilized in current running modern papers as the methodologies tested in the specific formats.
- 1 reply
replied to their post 9 days ago
After a week of faulty and failures using generic structures with bytelex, I found a successful aleph prototypical structure that conforms to the needs.
- We've managed to overlap multiple arms to train multiple simultaneous templates.
- Introduce new tokens.
- Retrain existing tokens.
- Extend new chains and new behaviors from training.
- Properly instantiate and reinforce behavior using Aleph RNN.
The EMA Relay. The code has been pushed to both beatrix repos.
The structure itself is built specifically as a solidification unit to extensible arms, allowing more composite structures to build.
EMA structures aren't new, but when applied correctly at just such a methodology, the models begin to behave as though the extension relays are in fact the original model. The chains and behavior form naturally and the substructure begins to conform with the token fragments from much more complex structures like combined token differences of T5, Qwen, and CLIP as unified teachers.
The cross-token noise is mitigated using a series of principles and the blueprints are showing both solidity and failure simultaneously, both proving many new utilizable states and disproving multiple theoretical pathologies utilized in current running modern papers as the methodologies tested in the specific formats.
replied to their post 17 days ago
60 hour battery under way currently. Currently up to around three sentences or so of bytelex capacity with multi-tokenizer inference comparisons.
Not the strongest yet, however the validation and test cases are showing promise at between 60 and 80% at highs with the canary recall remaining at around 97%, lows completely collapsed for multiple experiments.
Heavy experimentation with GRU, RNN, and multiple other components to test standard component utility.
So far so good. Many prototypes establishing information from many byte structured distillation routes.
posted an update 19 days ago
Post
91
The post-beatrix-2s and control variant article is finally satisfactory, so the article is now released https://huggingface.co/blog/AbstractPhil/beatrix-ft2
The control variant will need another train with better SDPA stabilization, as the control variant destabilized and collapsed. The primary fault is the lack of QK normalization, which caused the model to simply collapse given enough time. Claude lists the rest of the suspected reasons in the article.
This was a very difficult series of experiments to tune with many fault points. Trying to make heads or tails of Fable 5.1 Claude-speak hasn't been the easiest task either. It seems the model is more likely to create pedantically rigid responses rather than cooperative. Not necessarily insulting, but definitely a sort of refrigerator-magnet behavior - treating my individual contributions as little sketches for the refrigerator. This often completely ignores my larger MD or complex behavioral instructions in favor of my theoretical or hypothetical - likely considering the MD and technical as the model's own, rather than my direct contributions. Right there... right on the refrigerator goes my hypothesis that worked.
https://github.com/AbstractEyes/geolip-bytelex
In any case, this upcoming week will be related entirely to cross-tokenizer distillation research. It may stretch long beyond the next week, but as it stands the geometric vocabulary has evolved into a codebook prediction system.
I would like to give this program linear wings. The Beatrix model supports it, but how well is up for this week to decide.
There are a multitude of potentials based on a series of very recent articles I will be exploring, providing the necessary bytelex complexity to a roughly 60 hour battery of experiments and trainings throughout the geometric systems.
The results will determine the best and worst methodologies of using these models, these shapes, and these structures with more complex byte-level cross tokenization systems
The control variant will need another train with better SDPA stabilization, as the control variant destabilized and collapsed. The primary fault is the lack of QK normalization, which caused the model to simply collapse given enough time. Claude lists the rest of the suspected reasons in the article.
This was a very difficult series of experiments to tune with many fault points. Trying to make heads or tails of Fable 5.1 Claude-speak hasn't been the easiest task either. It seems the model is more likely to create pedantically rigid responses rather than cooperative. Not necessarily insulting, but definitely a sort of refrigerator-magnet behavior - treating my individual contributions as little sketches for the refrigerator. This often completely ignores my larger MD or complex behavioral instructions in favor of my theoretical or hypothetical - likely considering the MD and technical as the model's own, rather than my direct contributions. Right there... right on the refrigerator goes my hypothesis that worked.
https://github.com/AbstractEyes/geolip-bytelex
In any case, this upcoming week will be related entirely to cross-tokenizer distillation research. It may stretch long beyond the next week, but as it stands the geometric vocabulary has evolved into a codebook prediction system.
I would like to give this program linear wings. The Beatrix model supports it, but how well is up for this week to decide.
There are a multitude of potentials based on a series of very recent articles I will be exploring, providing the necessary bytelex complexity to a roughly 60 hour battery of experiments and trainings throughout the geometric systems.
The results will determine the best and worst methodologies of using these models, these shapes, and these structures with more complex byte-level cross tokenization systems
- 2 replies
I release everything MIT, but you can't find the trained weights elsewhere. It's too much data and too much space to host reasonably elsewhere today. If a rival crops up I'll dual-host most likely.
reacted to NitrAI's post with ❤️ 19 days ago
Post
1836
My (really ugly) thoughts on NVIDIA acquiring Hugging Face
I don’t care if you love me or hate me – something about one of the most open community efforts ever to achieve the tagline “The community building the future“ getting gobbled up by a company that arguably is the biggest hardware monopoly that has ever existed strikes me as deeply unsettling. I don’t like monopolies, and that is that. The whole appeal of HF for me personally was always having a neutral location where anyone could develop, deploy, and test a model on their silicon of choice without being pushed into a single “official“ proprietary infrastructure stack.
I am not going to pretend that I would believe NVIDIA “open and independent“ is ever going to happen – hell we have all heard the same lines dozens of times from every corporation that has ever uttered them before.
When the single biggest producer of compute also is one of the primary locations where all open weights live, it becomes very hard not to imagine where all of this is going to end up soon enough if we continue to let companies dictate the narrative. It might be the hyperbole but it is an absolute truth for me – open-sourced AI cannot be a slave to the whims of a trillion dollar company. It is high time we realize that open AI cannot live and breathe only on the goodwill of corporate entities.
I don’t care if you love me or hate me – something about one of the most open community efforts ever to achieve the tagline “The community building the future“ getting gobbled up by a company that arguably is the biggest hardware monopoly that has ever existed strikes me as deeply unsettling. I don’t like monopolies, and that is that. The whole appeal of HF for me personally was always having a neutral location where anyone could develop, deploy, and test a model on their silicon of choice without being pushed into a single “official“ proprietary infrastructure stack.
I am not going to pretend that I would believe NVIDIA “open and independent“ is ever going to happen – hell we have all heard the same lines dozens of times from every corporation that has ever uttered them before.
When the single biggest producer of compute also is one of the primary locations where all open weights live, it becomes very hard not to imagine where all of this is going to end up soon enough if we continue to let companies dictate the narrative. It might be the hyperbole but it is an absolute truth for me – open-sourced AI cannot be a slave to the whims of a trillion dollar company. It is high time we realize that open AI cannot live and breathe only on the goodwill of corporate entities.
- 11 replies
replied to their post 21 days ago
The control version essentially imploded with the same data and same shape. SDPA couldn't... actually represent the data. The erank completely collapsed and the model density is essentially multi-stage collapsed during the curriculum.
SDPA in every block instead of splat, failed...??
I legitimately didn't expect that. It will be in the article. I need to revise some information. Almost every core and key test showed the SDPA could potentially overtake the splat, but the actual outcome was a direct contradiction.
The collapse symptoms were showing early stage shape similarity to the beatrix-v1 splat + sdpa hub combo pack, which I did not expect what-so-ever. I expected the model to recover and form possibly more strength over time in each layer. The belly of the model bloated, then once the curriculum hit, the model turned inside out within 5000 steps. The model went from semi-functionally pretrained showing potential weaknesses in early bottleneck stages, leading to later layer structural boundary collection as the v1 did, and in the later training she completely collapsed.
The structure began collapsing during the curriculum that trained the v2 splat variant with some overfitting, but not collapse.
reacted to Banaxi-Tech's post with 🔥 21 days ago
Post
3016
We're going to release our BananaMind 2.1 models very soon!
We're also announcing 2 new models.
All of our models we will train are:
BananaMind 2.1 Flash Lite, 10M parameters with 8M in transformer and 2M in n-gram. 50B pretraining tokens.
BananaMind 2.1 Lite with 25M parameters, 5M in n-gram and 20M in transformer. 75B pretraining tokens.
BananaMind 2.1 Flash with 50M parameters, with undecided n-gram count yet. 100B pretraining tokens.
BananaMind 2.1 Pro with 145M parameters, with undecided n-gram count yet. 150-200B pretraining tokens.
BananaMind 2.1 Coder with 149M parameters with undecided n-gram count yet.
We're now announcing BananaMind 2.1 NanoCoder, a 10M parameter model focused specifically on coding and BananaMind 2.1 MiniCoder which is a 25M parameter model focused on coding.
Follow us:
BananaMind
@Banaxi-Tech
@vovaRL
@DedeProGames
bananamind-research-community
We're also announcing 2 new models.
All of our models we will train are:
BananaMind 2.1 Flash Lite, 10M parameters with 8M in transformer and 2M in n-gram. 50B pretraining tokens.
BananaMind 2.1 Lite with 25M parameters, 5M in n-gram and 20M in transformer. 75B pretraining tokens.
BananaMind 2.1 Flash with 50M parameters, with undecided n-gram count yet. 100B pretraining tokens.
BananaMind 2.1 Pro with 145M parameters, with undecided n-gram count yet. 150-200B pretraining tokens.
BananaMind 2.1 Coder with 149M parameters with undecided n-gram count yet.
We're now announcing BananaMind 2.1 NanoCoder, a 10M parameter model focused specifically on coding and BananaMind 2.1 MiniCoder which is a 25M parameter model focused on coding.
Follow us:
@Banaxi-Tech
@vovaRL
@DedeProGames
- 5 replies
reacted to Banaxi-Tech's post with 🔥 23 days ago
Post
2899
We're excited to release BananaMind 2.1 Pico Preview!
It includes the first preview of our BananaMind 2.1 architecture!
This model gets near BananaMind 2 Micro performance at half the parameters and 37.5x less tokens!
Thats insane!
The current architectures includes about 500K parameters of the total 1.5M parameters in n-gram embeddings and the layer 2 is run twice.
It also includes XSA and the XSA refresh gate.
We're still going to improve the architecture in the final release.
Check it out at:
Follow us for more models:
BananaMind
@Banaxi-Tech
@vovaRL
@DedeProGames
It includes the first preview of our BananaMind 2.1 architecture!
This model gets near BananaMind 2 Micro performance at half the parameters and 37.5x less tokens!
Thats insane!
The current architectures includes about 500K parameters of the total 1.5M parameters in n-gram embeddings and the layer 2 is run twice.
It also includes XSA and the XSA refresh gate.
We're still going to improve the architecture in the final release.
Check it out at:
Follow us for more models:
@Banaxi-Tech
@vovaRL
@DedeProGames
- 3 replies
reacted to AtAndDev's post with 🔥 23 days ago
Post
2846
SPECK 2 IS ALREADY OUT: specklabs/Speck2-140M
Pretrained on 4x more tokens than the previous releases (20b vs 5b).
Instruct tuned versions are coming soon.
Very interesting models are coming soon too (hint: super long context).
Thanks for everyone supporting!
Pretrained on 4x more tokens than the previous releases (20b vs 5b).
Instruct tuned versions are coming soon.
Very interesting models are coming soon too (hint: super long context).
Thanks for everyone supporting!
- 3 replies
reacted to dipankarsarkar's post with 🚀 23 days ago
Post
867
Your issue tracker is in the wrong place.
It lives on a server. Your code lives in git. So every time an agent picks up work it makes an API call, burns a token, fights a rate limit, and still cannot see what the other agent just did.
Move the issues into the repo. Append-only event log in git refs. Branches when you branch, merges when you merge, CRDT so two agents never conflict. No server, no database.
The coordination signal that PR-level telemetry misses lives before the pull request. The paper, and a live demo running the real tool:
Before the Pull Request: Mining Multi-Agent Coordination (2606.19616)
https://huggingface.co/spaces/neullabs/grite
If your agents share a repo, where does their shared state actually live right now?
It lives on a server. Your code lives in git. So every time an agent picks up work it makes an API call, burns a token, fights a rate limit, and still cannot see what the other agent just did.
Move the issues into the repo. Append-only event log in git refs. Branches when you branch, merges when you merge, CRDT so two agents never conflict. No server, no database.
The coordination signal that PR-level telemetry misses lives before the pull request. The paper, and a live demo running the real tool:
Before the Pull Request: Mining Multi-Agent Coordination (2606.19616)
https://huggingface.co/spaces/neullabs/grite
If your agents share a repo, where does their shared state actually live right now?
- 14 replies
replied to RiverRider's post 23 days ago
The geometric memory system first. Most of the papers are based on geometric calculations, many of which are based on understanding or calculating differences between models and their internal shapes in comparison to other shapes. I have many experiments on distillation, memory transplantation, fractal alignment, and a large amount of classifiers to back the measurements up. I started small as well, calculating models in comparison to other models, until I found enough footholds to form useful hypothesis towards scaling principles.
Most of the papers are a good read, there's just a lot of them and most of them are AI summarized research processes after around 6 months ago. Try not to eat them all at once. Fable can process them into a useful lookup table if you use Fable. Have the AI run regular adjacent reviews against papers if you do it that way, since there's a ton of papers cited.
There's a few geometric and trig measurements I discovered that simply do not have industry-standard calculations for usability, such as the CV embedding calculations. They have direct causal sizing, and given enough testing will yield very similar scales and sizes. It's a density vs sparsity measurement, so it's not the most useful, but it was a solid measurement to progress to a more accurate series of measurements.