--- license: agpl-3.0 base_model: meta-models/Muse-Glimmer-30B pipeline_tag: image-text-to-text language: - ja library_name: transformers tags: - ocr - document-transcription - japanese - vllm --- # Wasabi-30B Wasabi-30B reads a Japanese document page image and writes the page as Markdown: - full text in reading order; - tables as HTML ``; - math as LaTeX, `$...$` inline and `$$...$$` for display; - ruby (furigana) kept inline as `basereading`. ## Model details - Fine-tune of Muse-Glimmer-30B (https://huggingface.co/meta-models/Muse-Glimmer-30B, revision `97c77dff50b2797bcc558fa2d909761dbc575c59`). - A LoRA adapter trained for Japanese page transcription, merged into BF16 weights. - Two safetensors shards. ## Files in this repository | path | what it is | |---|---| | `model-00001-of-00002.safetensors`, `model-00002-of-00002.safetensors`, `model.safetensors.index.json` | merged BF16 weights | | `config.json`, `generation_config.json`, `processor_config.json` | model and processor configuration | | `tokenizer.json`, `tokenizer_config.json`, `chat_template.jinja` | tokenizer and chat template | | `inference/` | the page reader package (`wasabi_read`, AGPL-3.0) and its two prompts | | `LICENSE`, `NOTICE`, `notices/` | licence texts: this model (AGPL-3.0), the base model (Apache-2.0) and its usage policy | | `SHA256SUMS` | checksums of every file | ## Inference The package reads a folder of page images (`.jpg`, `.jpeg`, `.png`) and writes one `.md` file per page. pip install -e "./inference[dewarp]" Python 3.10+. The reader flattens every page locally with UVDoc on CPU, so torch and OpenCV are required in every mode, including endpoint mode. Local serving needs vLLM with muse_glimmer support, built from commit `6adad08767583f52eb4d2122111af0bf638ed5e6` (version `0.26.1rc1.dev769`). Released vLLM 0.26.0 has no muse_glimmer model, so it cannot serve this model. A source build needs a CUDA toolchain: pip install "vllm @ git+https://github.com/vllm-project/vllm.git@6adad08767583f52eb4d2122111af0bf638ed5e6" "transformers==5.15.0" Then, with the Python of that vLLM environment, apply the one-file processor fix. Without it, the muse_glimmer processor of this vLLM commit rejects the `max_image_tokens` setting that every server below is started with, and the server does not start. The script edits one file, checks its sha256 before and after, and can undo the edit (`--check`, `--revert`). A server you start yourself (endpoint mode, direct model use) needs it too. python inference/patch_vllm_max_image_tokens.py --apply An endpoint run needs no local vLLM. Tested with Python 3.12, vLLM `0.26.1rc1.dev769` (commit 6adad087), transformers 5.15.0, torch 2.13.0, opencv-python-headless 5.0.0.93, numpy 2.3.5 and Pillow 13.0.0. Wasabi-30B is the only checkpoint the reader needs: it checks the page orientation and reads the page. Local vLLM (the package starts and stops the servers itself: first the orientation check, then the reader, both from the same weights): python inference/wasabi_read.py pages/ out/ --engine vllm --model-dir /models/Wasabi-30B --tensor-parallel-size 1 --whole-upscale-tokens 8192 Against OpenAI-compatible servers you already run, both serving Wasabi-30B: the reader started with `--mm-processor-kwargs '{"max_image_tokens": 8192}'`, the orientation check with `--mm-processor-kwargs '{"max_image_tokens": 4096}'`, and both with `--reasoning-parser muse_glimmer`. Name what each server serves with `--endpoint-revision` and `--orient-endpoint-revision`; the addresses and revisions are part of the output identity, so a rerun against another server is refused instead of mixed into the same output directory. python inference/wasabi_read.py pages/ out/ --engine endpoint --endpoint-url http://gpu-host:8000 --endpoint-revision wasabi-30b-v1.0 --orient-endpoint-url http://gpu-host:8001 --orient-endpoint-revision wasabi-30b-v1.0 `--plan` prints the stages and exits without loading a model. The API key, the UVDoc cache location and offline UVDoc use are described under [Configuration](inference/README.md#configuration). ### Direct model use This sends the page image as it is: no orientation check, no flattening, no enlargement and no retries. It is a basic transcription path, separate from the reader above. It uses the vLLM build and the processor fix from the Inference section. pip install openai vllm serve /models/Wasabi-30B --served-model-name wasabi --max-model-len 36864 \ --mm-processor-kwargs '{"max_image_tokens": 8192}' --reasoning-parser muse_glimmer Then send one page with the OpenAI-compatible API, using the recommended decoding (greedy, repetition penalty 1.10, seed 42, up to 24,576 new tokens, low reasoning strength): import base64 from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="none") png = base64.b64encode(open("page.png", "rb").read()).decode() prompt = open("inference/wasabi_read/prompts/draft.txt", encoding="utf-8").read() r = client.chat.completions.create( model="wasabi", messages=[{"role": "user", "content": [ {"type": "image_url", "image_url": {"url": "data:image/png;base64," + png}}, {"type": "text", "text": prompt}]}], temperature=0.0, seed=42, max_tokens=24576, extra_body={"repetition_penalty": 1.10, "repetition_detection": {"max_pattern_size": 512, "min_pattern_size": 4, "min_count": 16}, "chat_template_kwargs": {"reasoning_strength": "low", "current_date": "2026-10-16"}}, ) print(r.choices[0].message.content) The transcription prompt is the file `inference/wasabi_read/prompts/draft.txt`. The reader pins `current_date` so that the output does not depend on the day you run it, and `repetition_detection` stops a looping answer early. `generation_config.json` keeps the base model's sampling defaults; pass `temperature=0` (as above) for the recommended greedy transcription. ## What the reader does Per page: 1. Orientation check: Wasabi-30B answers a four-way upright probe at 4,096 image tokens; a turn is applied only when the margin is at least 5.0. 2. Flattening with UVDoc on CPU. The UVDoc code and weights are fetched at first use from a pinned commit and checked by sha256 before loading. 3. A whole-page read at 8,192 image tokens, greedy. 4. If the answer is degenerate, a retry ladder at temperatures 0.1, 0.3, 0.5 and 0.8. 5. Output is written per page as `.md`, atomically. Rerunning the same command resumes: finished pages are skipped. Pages with no usable answer are listed in `out/.wasabi-read/run-report.json`. ## Benchmark submission The MDPBench submission includes additional inference components provided to the organizers. The complete inference code is maintained in a [private repository](https://github.com/sionic-ai/Wasabi-30B). The code included here runs independently of that repository. ## Limitations - Japanese-focused. Other languages are not a target. - The `transformers` engine has not been measured and is not guaranteed to follow the vLLM path's image size and decoding; use the vLLM or endpoint engine for the behaviour documented here. - Each orientation decision comes from one greedy probe per rotation, so a different serving stack (attention backend, CUDA build, batching) can change the turn on a few pages. - Ruby appears only as the model writes it inline; there is no separate ruby pass. - Photographed pages and very small print can be misread. - The reader package runs on Linux and macOS only (it uses POSIX file locks). Its local vLLM engine starts the servers with the FlashInfer vision-encoder attention backend; to use another backend, serve the model yourself and use endpoint mode. ## License - This model and the reader package: GNU Affero General Public License v3.0 (`LICENSE`). - Base model Muse-Glimmer-30B: Apache License 2.0 (`notices/Muse-Glimmer-30B-LICENSE.txt`), together with its publisher's usage policy (`notices/Muse-Glimmer-30B-USAGE_POLICY.md`). - UVDoc code (commit 4c9b82b5): MIT License, Copyright (c) 2023 Tanguy MAGNE. The reader downloads the UVDoc code and its pretrained weights at first use and does not redistribute them. The weights carry no separate licence statement and remain under their upstream terms. Copyright 2026 Sionic AI Inc.