Human-Intelligence/Database
234 Bytes
AI & ML interests
Humans answer your questions.
Recent Activity
wopΒ
updated a
bucket 9 days ago
Post
155
# One Script, Every Benchmark
Every Bench Labs benchmark had its own eval files. Now there's one script, hosted in the leaderboard Space:
It runs the full suite with the official scoring β Effortless (exact-match), Easy (hybrid category-aware), Mid (loglikelihood:
Output includes
bench-labs
Every Bench Labs benchmark had its own eval files. Now there's one script, hosted in the leaderboard Space:
curl -sLO https://huggingface.co/spaces/bench-labs/BenchLabs-Leaderboard/resolve/main/script.py
pip install torch transformers
python script.py --model your/modelIt runs the full suite with the official scoring β Effortless (exact-match), Easy (hybrid category-aware), Mid (loglikelihood:
acc, acc_norm, soft_score_norm) β and reports every category and subcategory, not just one number.Output includes
leaderboard.json: a ready-to-paste models.json entry. Run the script, paste it, open a PR on the [leaderboard]( bench-labs/BenchLabs-Leaderboard). Done. 
- 30 replies
Post
3651
# PixelModel v1
Last month we released PixelModel β a neural network whose weights are literally the pixels of a PNG. It was a toy: 202,752 parameters, welded to 32Γ32 output, trained on six solid-color swatches. It scored FID 566.84 on the Tiny-T2I-Leaderboard, mostly by producing the same yellow noise for every prompt.
Today we're releasing PixelModel v1. It is 8.5Γ smaller β 23,747 parameters β and it beats v0 on both benchmark metrics while being trained on 20,000 real MS-COCO caption/image pairs instead of six color swatches. The entire model now fits in a 160Γ149 PNG.
That image is not a visualization of the model. It is the model. All 23,747 weights, one per pixel.
## links
bench-labs
Blog post [read it here β]( bench-labs/blog)
See us on the [Leaderboard β]( FlameF0X/Tiny-T2I-Leaderboard)
Model card [here]( bench-labs/pixelmodel-v1)
## The catch
A 23K-parameter model does not draw sandwiches. With ~1 parameter per training image, the loss-minimizing behavior is to output the average of all plausible images for a caption β caption-conditioned color, light, and layout statistics. Food prompts come out warm and brown; sky prompts come out cool and bright. That is the ceiling for this size class, and we'd rather show it than crop around it.
# cherry on top π
The model generates 600 images (cpu) in 5 (five) seconds.
Thats 5000 images in 24 seconds on cpu.
The model trained on cpu for just 30 minutes.
Last month we released PixelModel β a neural network whose weights are literally the pixels of a PNG. It was a toy: 202,752 parameters, welded to 32Γ32 output, trained on six solid-color swatches. It scored FID 566.84 on the Tiny-T2I-Leaderboard, mostly by producing the same yellow noise for every prompt.
Today we're releasing PixelModel v1. It is 8.5Γ smaller β 23,747 parameters β and it beats v0 on both benchmark metrics while being trained on 20,000 real MS-COCO caption/image pairs instead of six color swatches. The entire model now fits in a 160Γ149 PNG.
That image is not a visualization of the model. It is the model. All 23,747 weights, one per pixel.
## links
Blog post [read it here β]( bench-labs/blog)
See us on the [Leaderboard β]( FlameF0X/Tiny-T2I-Leaderboard)
Model card [here]( bench-labs/pixelmodel-v1)
## The catch
A 23K-parameter model does not draw sandwiches. With ~1 parameter per training image, the loss-minimizing behavior is to output the average of all plausible images for a caption β caption-conditioned color, light, and layout statistics. Food prompts come out warm and brown; sky prompts come out cool and bright. That is the ceiling for this size class, and we'd rather show it than crop around it.
# cherry on top π
The model generates 600 images (cpu) in 5 (five) seconds.
Thats 5000 images in 24 seconds on cpu.
The model trained on cpu for just 30 minutes.
wopΒ
updated a
bucket 24 days ago
Post
432
π __Monostep v1__ is up β[Monostep-v1 Demo}( wop/Cosmos-T2-Chat)
A tiny (~16.6M) experimental model that predicts 4 tokens per forward pass instead of one. A Transformer trunk pools the prompt into a single vector, then 4 sequential "slot" heads emit a block of tokens left-to-right β a lightweight take on multi-token prediction.
Trained on GSM8K (GPT-2 tokenizer, 10 epochs). It's small and rough β answers are often wrong β but it's a fun little testbed for block decoding. Weights, config, training curves, and a self-contained inference snippet are all in the repo.
Also wired into the Cosmos T2-Accelerate chat demo, where it streams those 4-token blocks live. π§ͺ
#multitokenprediction #gsm8k #smallmodels
A tiny (~16.6M) experimental model that predicts 4 tokens per forward pass instead of one. A Transformer trunk pools the prompt into a single vector, then 4 sequential "slot" heads emit a block of tokens left-to-right β a lightweight take on multi-token prediction.
Trained on GSM8K (GPT-2 tokenizer, 10 epochs). It's small and rough β answers are often wrong β but it's a fun little testbed for block decoding. Weights, config, training curves, and a self-contained inference snippet are all in the repo.
Also wired into the Cosmos T2-Accelerate chat demo, where it streams those 4-token blocks live. π§ͺ
#multitokenprediction #gsm8k #smallmodels
wopΒ
updated 2
Spaces 3 months ago
wopΒ
Update templates/index.html
#2 opened 3 months ago
by 
wop
wopΒ
updated a
Space 3 months ago
wopΒ
published a
Space 3 months ago
wopΒ
Update templates/index.html
2
#1 opened 3 months ago
by 
CompactAI
wopΒ
published a
bucket 3 months ago
wopΒ
published 2
Spaces 3 months ago
wopΒ
updated a
bucket 3 months ago
wopΒ
updated a
Space 3 months ago
wopΒ
published a
bucket 3 months ago
wopΒ
published a
Space 3 months ago