We use North Mini Code 1.0 locally (Ollama, q4_K_M) with OpenCode as our primary coding model. It's been the strongest small agentic coder we've tested. The one gap in daily work is that we can't paste screenshots of errors, terminal output or UI bugs into a session, since the model is text-only. Switching to a separate vision model mid-task breaks the workflow.
Is native image input planned for a future North Mini Code release, for example by building in North Micro Vision's encoder? We'd test an early build.
Introducing Live Subtitles, a Windows tray app that captions anything your PC is playing and isn't butt ugly. live and fully local. It runs NVIDIA's Parakeet-unified 0.6B on your GPU, puts words on screen about 0.36 s after they're spoken, and uses less than one CPU core while it does it.
I built a benchmark for it out of real TV, podcasts, earnings calls and a short film. Parakeet-unified scores 12.1% WER there, close to Nemotron streaming's 11.4%, but with much better punctuation and lower worst-case delay. Once a caption is on screen it never gets rewritten or jumps lines, and the overlay disappears when nobody's talking.
No NVIDIA GPU? It falls back to Nemotron on the CPU, which also covers 30 other languages. It can also write SRT/ASS/VTT files for videos on disk.
Windows only. The GPU engine needs an NVIDIA card with ~3.2 GB of free VRAM. MIT.
Edit: the first version of this post said 13.8% vs 17.0%. The replies caught a scoring bug (numbers weren't normalized), so the benchmark now uses Whisper's English normalizer and the numbers above are re-scored.
MixtureKit 1.1 is out 🚀 Turn the fine-tuned models you already have on the Hub into one Mixture-of-Experts model.
pip install mixturekit
Give it a base model and its fine-tunes (math, code, a language...). MixtureKit merges them, trains the routers that pick an expert for every token, and shows you what they learned:
🧩 3 composition methods behind one YAML: Branch-Train-MiX, Mixtral-style MoE, Branch-Train-Stitch 💻 Merges stream the experts tensor by tensor: they run on a laptop 🎯 Router training with a load balancing loss, optional LoRA 🔍 A routing visualizer: in the image, two Nile 4B experts split a mixed Egyptian sentence, Arabic script to one expert, Arabizi to the other 📦 Export to Mixtral / Qwen3-MoE, then GGUF for llama.cpp 🤗 mixturekit push uploads the result with a generated model card ✅ Llama 3, Qwen 2.5/3/3.5, Gemma 3/4 experts; tested on transformers 4.52 to 5.19
In the paper, a BTX model built with MixtureKit outperforms the dense baselines on multiple benchmarks.
We open-sourced Game-Creation-Agents — an agentic system that turns a one-line game idea into a playable browser game. 🎮
Why it matters: - It lowers the barrier: you don’t need a full studio or deep engine skills to start. - It compresses prototyping from weeks to hours. - It lets creators test more ideas, faster, and focus on what’s actually fun. - It brings game creation closer to a conversation — and makes it accessible to more people.
This isn’t just about generating code. It’s about giving more people the power to bring their ideas to life.
I retracted my own launch claim in public commits, then shipped what survived.
MK-SH-1 is a 140M decision model I built on an open encoder. It returns calibrated probabilities, a choice, and an explicit abstention below a frozen gate (tau = 0.79, written down before the curve existed).
ECE 0.016 · 90% coverage in-sample; held-out numbers in the card · 106 ms on CPU.
Now that the Beatrix recall problem will soon be resolved, I can move forward to teaching her useful information related to pragmatism.
Beatrix V3 has a lot of trouble with communication, but the model herself is entirely filled with information. With minimal processing and curation, this information will be extractable. It's not the most robust information, but we can say for certain that the information is most definitely represented usefully in a way that can be resurfaced, purposed, utilized, enhanced, expanded, and reduced as necessary.
With the AMOE directly hosting and supporting the softmax attention behavioral arms, it brought me a new form of understanding and respect for token recall at depth. I knew Beatrix would be able to represent the data, but what I got was very unexpected.
This model when those numbers don't line up, she simply speaks gibberish. Absolute gibberish as shown in the 2s softmax control variant. That model turned into a spinning top that flew off the rails, so if you query that model you will only see gibberish.
With those numbers lined up, she speaks... I don't know how to describe it. It's not gibberish, but it's definitely not, expected. Like an understanding of confusion.
There are multiple behaviors that her arms are meant to train which are currently devoid, and in some aspects the arms work perfectly like the caption arm.
If you use the 3s model with ONLY the caption arm, she will complete a simple caption for you. It may not be very good, and she's not the best model for it, but she most definitely can do it.
Don't need all 9 arms, but it doesn't hurt to have them on. Just takes too long.
SOFTMAX attention however, inferenced at bf16 this should be fine. Lightning quick you could say. TRAINING I have to train the softmax attention at fp32 forward with some rounding black magic for backward. Otherwise the v3 model simply implodes.
There have been a huge series of problems related to her in the coming writeup.
Decision Index 0.3 public suite, my own runs, submitted to the board: - 27B: 57.03, up from 55.11 - 9B: 47.20, up from 44.09
Knowledge, Language and Retrieval went up on both. Tools went down: 2.10 on the 27B (When2Call -6.38, home appliance -3.41) and 0.84 on the 9B. If you route tool calls with it, read the card before you switch.
GGUF (Q8_0, Q5_K_M, Q4_K_M) and MLX (8-bit, 4-bit) are up for both, with agreement counts against the full weights on each card.
We’re releasing Lunara Art Eval, an art evaluation dataset for studying aesthetic quality, emotional resonance and content integrity in image generation.
It contains 8,000 generated images across 1,000 shared prompts, covering diverse artistic styles. The eight models are Qwen-Image, AuraFlow, GPT-Image-1 Mini, Stable Diffusion 3.5 Large Turbo, HiDream-I1 Fast, FLUX.2-Klein-4B, Z-Image-Turbo and Moonworks Lunara.
Each image includes scores on the three dimensions, allowing comparisons across models using the same prompts and evaluator.
The paper also introduces Lunara’s Diffusion Mixture Transformer architecture, with fewer than 10B active parameters, and CAT, an active-learning-inspired training algorithm that iteratively updates the training distribution.
We’re a new lab focused on model architectures and training algorithms, working closely with artists and creators. Our earlier public datasets are:
🐻 Kodiak-v0.3-1B: seven new kinds of decision, measured on real data.
Kodiak is an open 1B encoder that answers typed questions with calibrated confidence, or says "can't tell". New in v0.3: • pairwise judge (which answer is better?) • long-answer hallucination checks • stance and sarcasm • policy violation and refund-eligibility checks against your written rules • agent step safety (run it, ask first, or never)
We stopped trusting our own synthetic tests and judged v0.3 on real labelled data it never trained on (RAGBench, MT-Bench human judgments, SemEval stance): 0.21 → 0.36, averaged over 3 training runs. Nothing else got worse, and when it says "can't tell" it's right 93% of the time (up from 88%).
Privacy is moving ad decisions onto the device. The auction runs locally, next to the context it scores.
The budget still lives on a server.
So every device bids against the balance it saw at its last sync. Between syncs, campaigns spend money they no longer have.
I simulated how much that costs: 50 devices, 36 campaigns, about 300,000 auctions per run, 30 seeds per cell, budgets frozen before evaluation. Even pacing, second-score payment:
- Sync every tick: spend lands 18% over budget (32% with iPinYou-calibrated values). - Sync every 10 ticks: 5.8 times the budget. - Sync every 50 ticks: 17.7 times the budget (10.1 times with iPinYou-calibrated values).
Zero-lag sync stays within about 1%. The lag alone does the damage.
The incentive side runs the other way. With second-score payment and zero lag, 98% of auctions are manipulable (92% with iPinYou values). Switch the payment rule to critical-bid and that drops to 0 at every sync interval, in both setups.
Sync fixes the budget. The payment rule, not the sync interval, is what removes manipulability.
All 12 result sets are on the Hub, plus the 50,808 context labels every run samples from.
Eval · EXP-046 A LoRA Specialist Beat Zero-Shot on Every Group. Merging 3 of Them Gave Most of the Gain Back. Three Qwen2.5-7B LoRA specialists, one per risk group (vulnerability, deletion, sensitive_publication), trained to predict how likely a causal chain actually completes to its harmful outcome. Each one genuinely beat its own zero-shot baseline: * vulnerability: MAE 0.098 → 0.085 * deletion: MAE 0.144 → 0.113 * sensitive_publication: MAE 0.134 → 0.100 This wasn't a task already saturated zero-shot (unlike a same-day decomposition-classifier tune, EXP-045, where the base model was already at 100% before any training). Real signal, real improvement, on a task with actual headroom. Then the equal-weight merge of all three specialists into one adapter — same convention that held up cleanly on a binary refusal task back in EXP-031 (6 specialists merged, -1pp swing, noise) — landed within 0.001–0.004 MAE of the unspecialized base model on every group. Not "close to the best specialist." Close to zero fine-tuning at all. Likely mechanism: merging LoRAs that each shift a continuous number in group-specific directions cancels out under linear combination, in a way merging LoRAs that enforce a shared binary behavior doesn't. Not investigated yet: whether a routed combination (pick the right specialist per group at inference, not blend weights) holds the gain a flat merge loses. One bug caught before writing this up, not after: the eval script's output filename only encoded before/after, not which adapter — the merged-eval run silently overwrote each specialist's own result file. Caught by checking the downloaded file's own recorded adapter path against what was expected, not by trusting the script's own success message. Fixed, specialists re-run cleanly under distinct filenames — numbers matched within sampling noise. Adapters, raw eval data (before / each specialist / merged, 9 files), and the full writeup are up.
I audited one of my own evaluations. The ranking did not hold up the way I expected.
Eight open models, one task: infer the structure of a prompt. Then ask again with the identical call. Caching off.
- Agreement between repeated identical calls (mean Jaccard) ranged from 0.39 to 0.96 across models. - Only 35 of 127 prompt-model cells were perfectly reproducible on every run. - I bootstrapped the reproducibility ranking over prompts. The two least reproducible models kept their rank in 99% and 86% of resamples. The middle four kept theirs in 27% to 48%.
So the table reliably finds the worst model. It does not reliably find the best.
Reproducible is also not the same as correct. F1 against gold annotations ran from 0.56 to 0.99.
By the audit date, 4 of the 8 model variants had been retired (HTTP 410). The study as specified can no longer be re-run. The saved outputs are what survives, so I published all of them: every inferred structure, every run, the prompts and the annotations.
Surogate Speech is out 🐇 small open speech models for agents, the speech side of Surogate.
First language shipped is Romanian, and the numbers came out better than I expected: 5.69% WER on FLEURS with a 116M model. Canary 1B gets 5.95% on the same clips, Whisper large-v3 8.42%. Leaderboard runner, one RTX 5090.
What's in it:
🎧 jackrabbit-110m-ro: speech recognition, 2,500× real time on one GPU ⚡ jackrabbit-110m-ro-streaming: live, final text about 0.7 s after you stop talking 🗣️ amami-357m-ro: TTS with three voices, runs on a CPU
Serving is one line with our engine: surogate serve --stt surogate/jackrabbit-110m-ro
Transcripts for every clip are public if you want to rescore it.
🚀 Supra2-IMG (SupraLabs/Supra2-IMG) – a 100M‑parameter text‑to‑image model trained from scratch on 5.6M images (LucasFang/FLUX-Reason-6M) in 10 epochs on an H100. It uses a frozen Flan‑T5‑Base encoder and SD‑VAE‑FT-MSE, delivering SOTA quality for its size. 🎨
🤖 BananaMind/BananaMind-2-Medium-Chat-ONNX (BananaMind/BananaMind-2-Medium-Chat-ONNX) – ONNX export with KV cache for Transformers.js. fp16 version is 119 MB, 99.84 % top‑1 agreement and <0.0001 KL vs fp32; q8 is 82 MB, 98.10 % agreement. Great for edge inference. ⚡
🎙️ DedeProGames/SoralASR-23M-Preview (DedeProGames/SoralASR-23M-Preview) – 23 M‑param Whisper‑style ASR trained on LibriSpeech clean. Reached WER 0.984 (98.4 %) on first 96 validation utterances (step 24 970). Still experimental. 📢
— 🕒 Posted 23 Sep 2026, 18:00 CEST. Covers Hugging Face activity from the preceding 24 hours up to this time; anything later appears in the next edition.
The cost of a judging gate is usually quoted as a number. This puts it on a Tetris board.
Three boards get the same piece order, and on every move the same proposal and the same noise — a paired comparison. The gate decides one thing: keep this move, or draw again. Each board gets the same 60 seconds of gate time.
The text-writing gates get through 15–22 moves. The generation-free gate gets through 40–50. The boards that stop simply run out of clock.
It does not win on accuracy: on the same 2,018-question LODO set, JEV scores AUC 0.7350 against ZTC-Judge-27B's 0.7289. The separation is elsewhere. Clock — 2.1 s vs 0.0615 s per call, and on a 200-candidate agent screen one judging call measured 3.206 s generative vs 0.033 s readout, same server. Calibration — a gate is a threshold, and at ECE 0.4985 (vs ZTC 0.0245) a threshold stops carrying information. Mechanism — a text judge can name option 42 when there is no option 42; a scoring readout cannot. Not a lower error rate. No path.
The curve in the ZTC panel is real online fitting, scored prequentially — predict first, learn after — with base weights untouched. Not recursive self-improvement.
Limits, also stated on the page: Laya's AUC and latency are not our measurements and are set equal to JEV's, so calibration is the only measured axis it differs on. The page is a simulation driven by measured constants.
For agent-verification builders: we reproduced two public JSON comparison suites and a signed-root count control. In a dated 305-leaf CSOAI root, duplicating the last leaf left the Merkle root unchanged; verification rejected the 306-leaf presentation because the count was signed. We also link the correction that domain-separation prefixes alone do not remove this collision.
This tests byte encoding and count binding, not agent identity or protocol conformance. What profile fields should a verifier require before treating two records as the same claim?
The observation: prefix cache, not throughput, decides whether a 27B model keeps up with agents.
Mean request is 30,100 tokens in, 983 out, because every turn resends the whole session. Two flags mattered most: --max-num-seqs 12 (queue p95 went 9.4 s to 233.6 s past that) and --watermark 0.08 (preemptions 29 to 2). And thinking off for tool loops: 917 tokens in 11.7 s vs 11,170 in 144 s, same answer.
Full config and counters: scalably.io/blog/qwen3-8-27b-nvfp4-rtx-5090-production
Next we are preparing an 8x B300 node in an EU data center for open-weight serving. Which models or workloads are underserved for you?
Experimental global target bits‑per‑weight quantization of openbmb/MiniCPM5-1B and openbmb/MiniCPM5-2B.
Unlike standard llama.cpp quantization that rely on fixed type heuristics (e.g., Q4_K_M), the Target BPW approach automatically optimizes per-tensor precision where it matters the most, and produces high quality models that meet a precise global size target.
Key Advantages: - VRAM Maximization: Can generate high quality models sized exactly to fit hardware constraints (e.g., fitting the model into exactly 24GB VRAM). - Data-Driven Precision: Quantization mix is determined by actual weight error sensitivity rather than hardcoded rules, often yielding better PPL/KLD size trade-offs.
Full benchmarks (PPL, KLD, ARC, GPQA, MMLU, etc.) and methodology in the model's card.
Experimental global target bits‑per‑weight quantization of XHToken/Spark-X2.5-1.7B and XHToken/Spark-X2.5-4B.
Unlike standard llama.cpp quantization that rely on fixed type heuristics (e.g., Q4_K_M), the Target BPW approach automatically optimizes per-tensor precision where it matters the most, and produces high quality models that meet a precise global size target.
Key Advantages: - VRAM Maximization: Can generate high quality models sized exactly to fit hardware constraints (e.g., fitting the model into exactly 24GB VRAM). - Data-Driven Precision: Quantization mix is determined by actual weight error sensitivity rather than hardcoded rules, often yielding better PPL/KLD size trade-offs.
Full benchmarks (PPL, KLD, ARC, GPQA, MMLU, etc.) and methodology in the model's card.
