♾️ Thinking

Wop

wop

84 30 458

https://bench-labs-org.github.io

koo1140

AI & ML interests

AI research AGI

Recent Activity

liked a model about 3 hours ago

bench-labs/tinctura-v1

updated a Space 1 day ago

bench-labs/BenchLabs-Leaderboard

updated a Space 1 day ago

bench-labs/blog

View all activity

Organizations

reacted to FlameF0X's post with 🔥 2 days ago

Post

4357

Personally, I don't think bot accounts on Hugging Face are a good thing, as we don't know how many accounts are run by automated systems versus how many actual users there are. Dead Internet theory is already a thing.

To clarify I am taking about LLM powered bot accounts and NOT rule base once like @parquet-converter or others.

also I'd like to talk with HUMANS not a machine so I'm going to hide messages from bots.

22 replies

reacted to TobiasLogic's post with 🔥 4 days ago

Post

3393

We’ve been cooking something new at Bench Labs.

Introducing Cagliostro-v3, our new 146M parameter language model trained completely from scratch.

The run isn’t even finished yet.

At the current checkpoint:

• 146M parameters
• 72.7B / 75B tokens trained
• 26.27 Open SLM Index
• 43.80 ArithMark-3
• Trained on a single RTX 5090
• ~90K to 103K tokens/sec during training
• ~9 days for the full run
• Apache 2.0

For some context, SmolLM2-135M scores 27.13 on the same Index after being trained on roughly 2 trillion tokens.

Cagliostro-v3 is currently at 26.27 with only ~72.7B.

That’s around 27x fewer training tokens.

The model also currently Hold the number 3rd spot for ArithMark-3, scoring 43.80

This wasn’t achieved by just throwing more tokens at the model. A huge part of v3 has been figuring out architecture, data mixture, and training dynamics at this scale.

The model uses a custom 30-layer decoder architecture with grouped-query attention and cross-head subspace attenuation, SwiGLU, RMSNorm, RoPE, tied embeddings, and a warmup-stable-decay training schedule.

During cooldown we also substantially shifted the data mixture toward higher-quality synthetic textbook and mathematics data, with the mathematics share increasing from 10% to 28%.

And everything is open.

The repository contains the training history with checkpoints pushed roughly every 30 minutes, so you can inspect how the model evolved throughout training rather than only seeing the final weights.

This is still a pre-final checkpoint. We have roughly 2.3B tokens left and the learning-rate cooldown is still running.

So 26.27 isn’t the final number.

Really excited to see where the last part of the run lands.

Cagliostro-v3:
bench-labs/cagliostro-v3
Built by Bench Labs.

Open SLM Leaderboard:
AxiomicLabs/Open_SLM_Leaderboard

8 replies

reacted to RiverRider's post with 🔥 8 days ago

Post

117

Black Window — a chat model in your browser tab, on your hardware. A memory that stays on the device that opened the page.

https://blackwindow.xyz

Open the site, pick a model (about 0.6B to 8B), hit Load. The weights run in that tab, on that computer. After they load, the network can drop. The context window is a working set, auto-sized to that device, up to ~32K tokens.

Behind the window is the Weave. Every file, picture, recording, link, lookup, and reply is embedded as it arrives. Drop in audio and it is transcribed. Drop in an image and it is described. A question pulls the nearest passages back as notes. A long document is walked once so later questions can use the whole file, not the first pages.

Nothing leaves that tab unless you turn on live lookup or connect a rented GPU box, and the chat says so each time. Prompts can go to the box. Files and the Weave stay in the tab.

Console on that page: bw.ask, bw.search, bw.digest, bw.notes. A local relay exposes /v1/chat/completions on localhost so other tools on the same computer can talk to the tab. The tab polls the relay. That is the boundary.

Not a server with a policy. Your hardware, a window, a Load button.

If on mobile add to home-screen for best performance. If you break it lmk. It can serve a few hundred of you at a time before I have to buy a real server.

5 replies

reacted to branikita's post with 🔥🔥 20 days ago

Post

2232

SO-ARM 102 goes open source in the next few weeks.

What's new:
- A parallel gripper: the jaws stay parallel through the whole stroke instead of pivoting around the object as they close.
- PET-CF instead of PLA+ for a much stiffer frame.
- A topology-optimized structure.
- Wider joint rotation and folding range.
- STS3250 servos on the first shoulder joint.

Compared with the SO-ARM 101, that adds up to 2.5x the payload, roughly 2x better positioning accuracy, roughly 1.6x the movement speed, and about 36 mm more reach.

It runs on Hugging Face LeRobot, so the same tooling, training pipeline and tutorials for the SO-ARM 101 work on it from day one.

1 reply

reacted to NitrAI's post with ❤️ 20 days ago

Post

1836

My (really ugly) thoughts on NVIDIA acquiring Hugging Face
I don’t care if you love me or hate me – something about one of the most open community efforts ever to achieve the tagline “The community building the future“ getting gobbled up by a company that arguably is the biggest hardware monopoly that has ever existed strikes me as deeply unsettling. I don’t like monopolies, and that is that. The whole appeal of HF for me personally was always having a neutral location where anyone could develop, deploy, and test a model on their silicon of choice without being pushed into a single “official“ proprietary infrastructure stack.

I am not going to pretend that I would believe NVIDIA “open and independent“ is ever going to happen – hell we have all heard the same lines dozens of times from every corporation that has ever uttered them before.

When the single biggest producer of compute also is one of the primary locations where all open weights live, it becomes very hard not to imagine where all of this is going to end up soon enough if we continue to let companies dictate the narrative. It might be the hyperbole but it is an absolute truth for me – open-sourced AI cannot be a slave to the whims of a trillion dollar company. It is high time we realize that open AI cannot live and breathe only on the goodwill of corporate entities.

11 replies

reacted to FlameF0X's post with 👍 24 days ago

Post

2629

Hello HuggingFace!

I would like to share a small preview of a language model that I have been experiencing for a while. In the first attached image there is a small sample of the Myosotis-1, an attempt to make a small 100m parameter flagship model that is built on my bizarre architecture that is somewhat similar to an S4/S5 model with WKV added. I call it FWKV (Feed-Forward WKV). The model is currently still in training because of the nature of RNN-like models. It can also be seen that the model has insane prompt processing and token generation speed (evaluation done on a 2x Titan XP); even for its small size, some similar Transformer models do struggle to get the same results without custom kernels (some Transformer models can achieve this level of throughput on cheap hardware).

In the second image, you can see checkpoint 20k of the model in its next token prediction state (this means it can't chat), ranking in the top 100 on AxiomicLabs/Open_SLM_Leaderboard (the results have not been submitted since the model is not done training).

- Why not just use Transformers?
Have you seen any pure non-Transformers SLMs besides RWKV and Mamba?

- Should you expect this project to become the next LFM or another very fast language model thing on some Raspberry Pi?
No, the model is still an experiment; it's very sensible and prone to collapse (by the time of this post, it can be seen in the 1st image).

- Should you use it?
Maybe not yet; the architecture itself is still very "naive"—that's how I could call it at its current level. If you just want to play with it and see what you could do or how fast the model is on your hardware, then you can do it.

Once the training is finished and I feel satisfied with the model next token prediction (the base model) and "assistants" (the instruction-tuned model) capabilities, I will make open weights at

FWKV with full support of the 🤗 Transformers.

10 replies

reacted to appvoid's post with 🔥 27 days ago

Post

2647

Nobody knows what is doing, when you train a model, you are experimenting to advance the frontier, so keep failing 🫵

17 replies

reacted to salma-remyx's post with 🔥🔥🔥 28 days ago

Post

2721

If you can’t explain why your AI-generated contribution belongs in the repo, don’t put it in a maintainer’s queue 🙅🏻‍♀️

Before asking for review, you should be able to answer:

1. What project need does it address?
2. Where does it fit, and does it duplicate existing work?
3. What evidence shows it works, and will you own it through review?

If you can’t answer those, you haven’t saved anyone time. You’ve passed the buck to the maintainer.

Outrider has made us better contributors by doing more of this work before upstream review by
* reading contribution rules, accepted PRs, and open issues
* finding needs and integration points
* drafting the code, tests, and context.

We still decide what deserves to go upstream, verify the claims, coordinate with maintainers and contributors, and stay involved through review.

On huggingface/peft, only 4 of 20 Outrider runs opened draft PRs. Two contributions have now merged:

✅ Riemannian-preconditioned LoRA: https://github.com/huggingface/peft/pull/3382
✅ Super-Tuning: https://github.com/huggingface/peft/pull/3518

We have more contributions in review and far more ideas were filtered out before they reached a maintainer.

Full case study: https://remyx.ai/case-study
Outrider: https://github.com/remyxai/outrider

11 replies

reacted to danielhanchen's post with 🔥 about 1 month ago

Post

5878

Introducing Unsloth Desktop 🦥
The first desktop app to run and train models locally.

• Open-source. Runs on Mac, Windows and Linux
• Supports MLX, diffusion image/video, audio, GGUF
• Connect Claude Code and Codex to local LLMs
• 50% more accurate, self-healing tool calls + sandboxed code exec
• Works for CPU + multiGPU setups - NVIDIA, AMD, Intel, Mac
• Train models 2× faster with 70% less VRAM
• Private web search, deep research, RAG, MCP and exports (NVFP4, GGUF)
• Use Unsloth’s OpenAI-compatible API and cloud models
• Securely deploy LLMs remotely and access anywhere

Unsloth Desktop is now available on http://unsloth.ai
and GitHub.

GitHub: https://github.com/unslothai/unsloth
Blog and Guide: https://unsloth.ai/docs/desktop

6 replies

reacted to their post with 🔥 about 1 month ago

Post

2199

bench-labs/GCTokenizer-v1 , a multilingual tokenizer which does not require a training corpus

bench-labs developed **GCTokenizer-v1**, which is a multi-lingual tokenizer
Available in four sizes: 32K, 65K, 131K and 262K tokens "S, M, L, XL"
It utilizes an encoding scheme which allows it to handle characters in any language around the world

General (multi lingual)
Consensus (from multiple model tokenizers consensus)
Tokenizer

We included an implementation script too,
built like BPE- it can encode arbitrary text, most of the time, efficiently

posted an update about 1 month ago

Post

2199

bench-labs/GCTokenizer-v1 , a multilingual tokenizer which does not require a training corpus

posted an update about 2 months ago

Post

2326

all models now also run on zerogpu!
BenchLabs/Demo-FAST

Speed measurements:
image ~5s (pixelmodel v5 ~0.5s)
3D ~2s
audio ~3s +(upload)

insane.
Yes, we made a new model: bench-labs/AudioModel-v1
BIG thanks to @TobiasLogic

reacted to appvoid's post with 🔥 about 2 months ago

Post

2389

I don't know if it was us or one of you guys or maybe all of us at once but lately we have seen a finetuning/pretraining explosion of models below 200m params and we can't be more happy about it keep coming tinkerers all of this is possible because of you!

8 replies

replied to their post about 2 months ago

Yeah, that's basically it SlopFinder isnt defining slop, its trying to measure the perceived / averaged version of what humans call slop.
Kind of like human preference data: some of it's obviously objective (broken hands, logical inconsistency, obvious asset-flips like you mentioned), some of it's just taste (what's "enjoyable" varies person to person).
The dataset is the collection of a lot of different definitions like yours.

replied to their post about 2 months ago

implemented almost all of these

reacted to their post with 🔥 about 2 months ago

Post

2326

🧪 SlopFinder is here!!

We're building a dataset to study what humans actually consider AI slop.

SlopFinder shows you a random piece of AI-generated text and gives you one simple control: **how slop is it?**
No categories. No complicated forms. Just vote and move on.

Every vote helps build the dataset. 🧩

How does it work?
Samples are pulled from existing datasets, shown anonymously, and collected into our annotation pool. After enough votes, they're exported to Hugging Face for everyone to use.

This is an early MVP, so the dataset is small and the system is still evolving.

Vote here:
https://bench-labs.web.app/slopfinder.html
(refresh page if you want to skip)

Dataset:
bench-labs/slop-classification

@benchlabs

17 replies

replied to their post about 2 months ago

This is growing pretty fast,I just uploaded another batch, theres now ~115 rows (before 85) 🔥

I will soon make an upgrade for better batching + some big improvements (context & diversity) and maybe some visual upgrade, and maybe auto upload🎉

I wonder what improvements you would like to see (maybe an API?) - tho its suposted not to be automated with AI, but voted by humans🤷🏻

Wop

AI & ML interests

Recent Activity

Organizations

wop's activity