Universal Computing Research
🌍
1
AI & ML interests
None defined yet.
Recent Activity
View all activity
ucr-max
updated a
Space about 5 hours ago
ucr-max
updated a
Space about 7 hours ago
Post
980
Introducing Limen0.2B
We are releasing Limen0.2B, a 222.5M-parameter base language model developed as a research platform for efficient pretraining and superword tokenization at smaller scales.
Limen0.2B was trained from scratch on 50B tokens and uses a compact 16K BoundlessBPE vocabulary. The project explores whether SuperBPE-style tokenization can remain effective in a substantially smaller model and vocabulary regime than those examined in earlier large-scale experiments.
The model also combines a deep-and-narrow transformer design with Exclusive Self-Attention, grouped-query attention, and tied embeddings. Its compact vocabulary reduces the embedding footprint and leaves a larger share of the parameter budget available to the transformer layers.
Despite its relatively modest training budget, Limen0.2B achieves competitive results for its scale across the reported language understanding, commonsense reasoning, and grammatical evaluation tasks. Comparisons with other compact models are provided as context rather than strict rankings, as their training data, token budgets, architectures, and evaluation settings differ.
The release includes the model weights, implementation, training configuration, checkpoint progression, and evaluation results, all under Apache 2.0.
UniversalComputingResearch/Limen0.2B
Technical feedback, independent evaluations, and further experiments with the model and tokenizer are welcome.
We are releasing Limen0.2B, a 222.5M-parameter base language model developed as a research platform for efficient pretraining and superword tokenization at smaller scales.
Limen0.2B was trained from scratch on 50B tokens and uses a compact 16K BoundlessBPE vocabulary. The project explores whether SuperBPE-style tokenization can remain effective in a substantially smaller model and vocabulary regime than those examined in earlier large-scale experiments.
The model also combines a deep-and-narrow transformer design with Exclusive Self-Attention, grouped-query attention, and tied embeddings. Its compact vocabulary reduces the embedding footprint and leaves a larger share of the parameter budget available to the transformer layers.
Despite its relatively modest training budget, Limen0.2B achieves competitive results for its scale across the reported language understanding, commonsense reasoning, and grammatical evaluation tasks. Comparisons with other compact models are provided as context rather than strict rankings, as their training data, token budgets, architectures, and evaluation settings differ.
The release includes the model weights, implementation, training configuration, checkpoint progression, and evaluation results, all under Apache 2.0.
UniversalComputingResearch/Limen0.2B
Technical feedback, independent evaluations, and further experiments with the model and tokenizer are welcome.
ucr-max
updated a
model 3 days ago
ucr-max
published a
model 3 days ago
Model Request
1
#1 opened 6 days ago
by 
Banaxi-Tech
ucr-max
authored a
paper 6 days ago
Post
2969
We have released the first version of CompactLMIndex UniversalComputingResearch/clmi, a curated benchmark for smaller language models.
The index evaluates models across six benchmarks covering commonsense, science and world knowledge, and grammatical competence. Scores are normalized for random chance, so results are easier to compare across tasks with different baselines.
The composite gives 50% weight to commonsense, 25% to science and world knowledge, and 25% to grammatical competence. BLiMP is macro-averaged across its 12 linguistic categories. Every result includes 95% bootstrap intervals, and rank ranges reflect paired comparisons on matched evaluation items.
The goal is to compare compact models under one fixed, zero-shot evaluation protocol and make differences in architecture, training, and inference visible without treating parameter count as a substitute for quality.
If you have results for a model that fits the benchmark scope, please open a discussion on the Space.
The index evaluates models across six benchmarks covering commonsense, science and world knowledge, and grammatical competence. Scores are normalized for random chance, so results are easier to compare across tasks with different baselines.
The composite gives 50% weight to commonsense, 25% to science and world knowledge, and 25% to grammatical competence. BLiMP is macro-averaged across its 12 linguistic categories. Every result includes 95% bootstrap intervals, and rank ranges reflect paired comparisons on matched evaluation items.
The goal is to compare compact models under one fixed, zero-shot evaluation protocol and make differences in architecture, training, and inference visible without treating parameter count as a substitute for quality.
If you have results for a model that fits the benchmark scope, please open a discussion on the Space.
- 6 replies
ucr-max
published a
Space 7 days ago
Kudos on Atom 2.7M
🤗 1
4
#1 opened 19 days ago
by 
Mmorgan-ML
Post
141
Published under Universal Computing Research: Atom2.7m, a 2.74M-parameter causal LM experiment with arithmetic-aware numeric representation.
It keeps regular BPE-style text continuation, while adding structure for numeric spans: digit ordering, place_ids, and role_ids.
Atom2.7m reaches 69.24% accuracy on ArithMark2.0.
The question behind the project is narrow: how much of small-model arithmetic performance depends on representation, rather than scale alone?
Model: UniversalComputingResearch/Atom2.7m
Article: https://huggingface.co/blog/ucr-max/atom2-7m-arithmetic-representation
Demo: UniversalComputingResearch/Atom2.7-Demo
It keeps regular BPE-style text continuation, while adding structure for numeric spans: digit ordering, place_ids, and role_ids.
Atom2.7m reaches 69.24% accuracy on ArithMark2.0.
The question behind the project is narrow: how much of small-model arithmetic performance depends on representation, rather than scale alone?
Model: UniversalComputingResearch/Atom2.7m
Article: https://huggingface.co/blog/ucr-max/atom2-7m-arithmetic-representation
Demo: UniversalComputingResearch/Atom2.7-Demo
- 1 reply
ucr-max
updated a
Space 21 days ago
ucr-max
published a
Space 21 days ago
ucr-max
updated a
model 21 days ago
ucr-max
updated a
dataset 27 days ago
ucr-max
published a
dataset 27 days ago
ucr-max
published a
Space 27 days ago
ucr-max
published a
model 27 days ago
ucr-max
updated a
model about 1 month ago
ucr-max
published a
model about 1 month ago