Muse-Glimmer-30B, GInfer INT
This repo holds the GInfer packages of Muse-Glimmer-30B in INT: single-file packages for the GInfer inference engine (and GChat, which is built on it). They run only on GInfer / GChat. They are not safetensors or GGUF files and other runtimes cannot load them.
GInfer is Sectile Labs' inference engine for fast, high-concurrency, long-context work on NVIDIA GPUs.
Packages
| File | Drafter | Tensor-parallel size | Size in bytes | SHA-256 |
|---|---|---|---|---|
muse_glimmer_30b_int_df2.ginfer |
Q4 DFlash2 | 1 (one GPU) | 19,776,776,704 | 46b5d54ee64ccd278f2a1a4d2c6d5aa9fc9097e8db0c3d165179aee5f2cb5b66 |
muse_glimmer_30b_int_df2_tp2.ginfer |
Q4 DFlash2 | 2 (two GPUs) | 19,776,896,000 | 9208757adff8bf6c0b01013de3a9910236bbca67311717382e09a8cf895237ec |
Format: INT (groupwise integer weights).
GPUs: INT runs on NVIDIA Ampere, Ada and Blackwell GPUs.
The TP2 packages split the model across two identical GPUs; load them with tensor-parallel size 2. TP4 packages will be added later.
Check each download with sha256sum muse_glimmer_30b_int_df2.ginfer or sha256sum muse_glimmer_30b_int_df2_tp2.ginfer.
Upstream models and licenses
- Base model: meta-models/Muse-Glimmer-30B, revision
a4e59da52a7bc87ae7251dd5545c0dd437c44b68. License fieldapache-2.0; the repo LICENSE file is the Apache License 2.0 (full text inLICENSEhere). - Drafter: z-lab/Muse-Glimmer-30B-DFlash2, revision
b54ffdd11fa9cfe2af370012e5763d492c904128. License fieldapache-2.0.
Quantized by Sectile Labs to groupwise INT (AutoRound-based) from the official BF16 weights of meta-models/Muse-Glimmer-30B (revision a4e59da52a7bc87ae7251dd5545c0dd437c44b68), then converted to a .ginfer package.
This package is a conversion of the upstream weights into the GInfer format; GInfer does not change the upstream license. The upstream license text is in LICENSE.