xingaouk/kobutai-media-classifier

Kobutai media classifier

A 2 MB model that guesses what is playing from text alone: the title, the artist or channel name, the website and the duration. No audio, no network, under 1 ms per item on a laptop CPU.

It powers Kobutai, a browser extension that puts a small character on your desk who reacts to what you are listening to or watching. The character changes with the answer, so the model only has to be right about the things the extension acts on:

Output	Labels
type	music, video, live, podcast
genre (music only)	rock, western-pop, jpop, anime, cpop, electronic, lofi, classical, hiphop, other
video class (video only)	ent (entertainment, memes, gaming, vlogs), info (education, news, science, how-to)

Each answer comes with a confidence. Below the thresholds in config.json the model abstains (returns null), and the app falls back to a neutral choice or asks a larger model.

Try it: live demo (runs in your browser).

Paper: Knowledge or Capacity? Lessons from a 2 MB On-Device Classifier for What Is Playing (preprint). Full research record: github.com/xingaobio/kobutai-model.

Quick start

Python (standard library only):

from huggingface_hub import snapshot_download
import sys
path = snapshot_download("xingaouk/kobutai-media-classifier")
sys.path.insert(0, path)
from kobutai import Classifier

clf = Classifier(path)
clf.classify("YOASOBI「アイドル」Official Music Video", "YOASOBI", "www.youtube.com", 213)
# {'type': 'music', 'typeConfidence': 0.9999, 'genre': 'jpop', 'genreConfidence': 0.886, 'genreVia': 'table', ...}

clf.classify("黎曼猜想到底是什么？", "李永乐老师", "www.bilibili.com", 1502)
# {'type': 'video', 'vclass': 'info', ...}

JavaScript (browser, service worker or Node; no dependencies):

import { Classifier } from "./kobutai.js";
const base = "https://huggingface.co/xingaouk/kobutai-media-classifier/resolve/main/";
const [cfg, bin, tsv] = await Promise.all([
  fetch(base + "config.json").then(r => r.json()),
  fetch(base + "kobutai.bin").then(r => r.arrayBuffer()),
  fetch(base + "artists.tsv").then(r => r.text()),
]);
const clf = Classifier.load(cfg, bin, tsv);
clf.classify({ title: "Lose Yourself", artist: "Eminem", source: "open.spotify.com", duration: 326 });

Inputs: title, artist (artist or channel), source (host name, e.g. www.youtube.com), duration in seconds (null or 0 if unknown), isLive, and an optional platformCategory (music or podcast) when the player already knows.

How it works

Deliberately simple, so it is small, fast and identical on every platform.

1. Normalise the text (Unicode NFKD, strip accents, lower-case ASCII, collapse spaces).
2. Hash features into 131,072 buckets with FNV-1a: character 1–4-grams of the title and artist (works for Chinese and Japanese without a tokenizer), English words and word pairs, the site, a duration bucket, and site × duration.
3. Three linear heads (multinomial logistic regression) for type, genre and video class, weights quantised to int8.
4. Site rules: music sites are always music, podcast apps always podcasts, film and course sites can only be video, live streams are live or music.
5. Artist table (13,787 names and aliases, including Chinese, Japanese and romanised spellings): a known artist in the artist field or the title adds a fixed bonus to that genre.

File	Size	What
kobutai.bin	2.1 MB	weights, compact binary format (KBT1, described in kobutai.py)
kobutai.safetensors	2.1 MB	the same weights as safetensors (<head>.weight_int8, .scale, .bias), for inspection
artists.tsv	0.22 MB	normalised artist name → genre index
config.json	2 KB	labels, thresholds, artist bonus, site rules
kobutai.py, kobutai.js		reference implementations (Python, JavaScript + .d.ts)
test-vectors.json		379 inputs with exact expected outputs; any port must match to 1e-9
eval/results.json		every number below

Evaluation

Real test set: 383 titles taken on 13 October 2026 from live charts and trending pages (Billboard, Spotify, NetEase Cloud Music, YouTube, Bilibili, Twitch, podcast charts and others) in Chinese, Japanese and English. None were seen in training. "Fully right" means the type and, where relevant, the genre or video class are all correct.

	Fully right	Type	Genre	Video class
This model (always answers)	79.9%	94.5%	73.8%	81.9%
This model with its thresholds	answers 91.1%, right on 84.5% of those
DeepSeek V4.1 Flash (large LLM, for reference)	95.3%	100%	93.5%	95.7%
This model, handing items below 0.9 confidence to DeepSeek (44% of items)	94.8%	99.2%	93.0%	94.7%

By language (always answers): English 84.3%, Chinese 79.8%, Japanese 75.4%.

Set	n	Fully right
Internet memes, Japanese idols, anime (curated set)	360	91.7%
Held-out synthetic validation set	963	95.9%

Genre recall on the real test set is weakest for lofi (47%), other (50%) and hiphop (61%), strongest for classical (100%), western-pop (88%) and cpop (88%).

Training data

About 13,800 labelled examples:

• ~13,200 written by AI assistants following a labelling guide: realistic titles and channel names across the three languages, sites and categories, with dedicated sets for hard cases, internet memes, Japanese idols and anime.
• 575 real titles from the public Bilibili popular list and YouTube trending page, labelled with the same guide.
• Artist table compiled by AI assistants from general knowledge, then de-duplicated and conflict-checked.

A title that appears in any evaluation set is removed from training. The LLM-written training data, artist lists and synthetic evaluation sets are published with the full research record at github.com/xingaobio/kobutai-model (CC BY-NC 4.0); titles collected from websites are not redistributed.

The Kobutai extension ships a version that is also trained on public Apple Music chart data and scores 84.9% on the same real test set. That data is left out here because its terms of use for this purpose have not been checked.

Limitations

• Text only. An instrumental titled "Track 01" from an unknown artist cannot be classified; the model will abstain or guess.
• Genres are coarse and opinionated. Ten buckets chosen for one app; other collects everything from jazz to folk to K-pop.
• Knowledge freezes. New artists are recognised only through character patterns until the artist table is updated.
• The evaluation is small. 383 real items give roughly ±4 percentage points of uncertainty.
• Built for Chinese, Japanese and English. Other languages fall back to site and duration signals.

Research notes

Seven rounds of experiments led to this design. Two things that did not help:

• Distilling DeepSeek into this model (relabelling and adding LLM-labelled data) made the real test score worse, mostly through podcast episodes posted on YouTube and the imbalance of the added data.
• A 100M-parameter multilingual transformer (mMiniLMv2, 428 MB) scored about the same as this 2 MB model (84.1% vs 84.9% on the full-data version), and its int8 version scored lower.

What did help: curated artist lists, site rules, targeted hand-written hard cases, and handing only the uncertain items to a large model.

License and citation

CC BY-NC 4.0: free to use, share and adapt for non-commercial purposes with attribution. For commercial use, please get in touch.

@misc{gao2026kobutai,
  author = {Xin Gao},
  title  = {Knowledge or Capacity? Lessons from a 2 MB On-Device Classifier for What Is Playing},
  year   = {2026},
  url    = {https://huggingface.co/xingaouk/kobutai-media-classifier}
}

中文简介

只看文字（标题、歌手或频道名、网站、时长）判断"正在播放什么"的 2 MB 小模型：类型（音乐、视频、直播、播客）、音乐的 10 种曲风、视频是娱乐还是知识。纯离线，每条不到 1 毫秒，Python 和 JavaScript 结果完全一致。是浏览器扩展 Kobutai（小舞台）里桌面角色的"耳朵"。真实榜单标题上完全判对 79.9%；把不确定的约 44% 交给 DeepSeek 后为 94.8%。