ucr-max commited on
Commit
cce2eea
·
verified ·
1 Parent(s): 449abcb

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +70 -5
README.md CHANGED
@@ -1,10 +1,75 @@
1
  ---
2
- title: README
3
- emoji: 🌖
4
- colorFrom: yellow
5
- colorTo: red
6
  sdk: static
7
  pinned: false
 
 
8
  ---
9
 
10
- Edit this `README.md` markdown file to author your organization card.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: Universal Computing Research
3
+ colorFrom: blue
4
+ colorTo: indigo
 
5
  sdk: static
6
  pinned: false
7
+ license: apache-2.0
8
+ emoji: 🌍
9
  ---
10
 
11
+ ![Background image](bg.png)
12
+
13
+ # Universal Computing Research
14
+
15
+ **Universal Computing Research** is an independent AI research organization focused on efficient, compact, and architecture-driven deep learning.
16
+
17
+ We build small language models, parameter-efficient neural layers, custom tokenizers, and research artifacts that test how far useful intelligence can be pushed under strict compute, memory, and parameter budgets.
18
+
19
+ ## Research direction
20
+
21
+ Our work is centered on a simple question:
22
+
23
+ > How much capability can be recovered through better architecture, tokenization, data curricula, and parameterization, without relying only on scale?
24
+
25
+ Current focus areas:
26
+
27
+ - Small language models
28
+ - Parameter-efficient architectures
29
+ - Random projection layers
30
+ - Custom tokenization pipelines
31
+ - Arithmetic and algorithmic reasoning
32
+
33
+ ## Released models
34
+
35
+ ### Atom3.4m
36
+
37
+ A 3.41M parameter decoder-only language model trained from scratch for studying compact architectures, curricula, and small-model benchmarking.
38
+
39
+ - Grouped-query attention
40
+ - RoPE positional embeddings
41
+ - RMSNorm
42
+ - Gated SiLU feed-forward layers
43
+ - Custom 4,096-token byte-level BPE tokenizer
44
+ - Approximately 5B training tokens
45
+
46
+ [View Atom3.4m](https://huggingface.co/UniversalComputingResearch/Atom3.4m)
47
+
48
+ ### Atom2.7m
49
+
50
+ A 2.74M parameter causal language model with an arithmetic-aware tokenizer and digit-structure features.
51
+
52
+ - Custom byte-level BPE tokenizer
53
+ - Atomic digit and operator handling
54
+ - Least-significant-digit-first numeric representation
55
+ - Place and role embeddings for integer arithmetic
56
+ - Strong ArithMark-2.0 performance for its size
57
+
58
+ [View Atom2.7m](https://huggingface.co/UniversalComputingResearch/Atom2.7m)
59
+
60
+ ## Research
61
+
62
+ ### Parametrized Random Projection
63
+
64
+ We study **Parametrized Random Projection** layers as lightweight replacements for dense linear layers.
65
+
66
+ The core idea is to separate fixed feature mixing from learnable adaptation: a non-trainable random projection performs the mixing, while small learnable element-wise parameters modulate the input and output.
67
+
68
+ This reduces trainable parameter count from quadratic to linear scale while preserving much of the utility of dense projections.
69
+
70
+ [Read the paper](https://arxiv.org/abs/2512.13480)
71
+
72
+ ## Open source
73
+
74
+ Our models and research artifacts are released to support reproducible, open, and practical AI research.
75
+