ginfer

Qwen3.8-27B, GInfer SMOL 2-bit mix

qwen38_27b_smol_df2.ginfer is a single-file GInfer package of Qwen3.8-27B for the GInfer inference engine (and GChat, which is built on it). It runs only on GInfer / GChat. It is not a safetensors or GGUF file and other runtimes cannot load it.

GInfer is Sectile Labs' inference engine for fast, high-concurrency, long-context work on NVIDIA GPUs.

Package

File qwen38_27b_smol_df2.ginfer
Size 10,667,321,600 bytes
SHA-256 483343c28912ae5db3d00478f16fffb10dcbb0c1317f8c90c09c396fcb6e4ca4
Format SMOL 2-bit mix (mostly Q2, with Q4/Q5 for attention and late MLP-down layers); Q4 DFlash2 drafter included; tensor-parallel size 1 (TP1)
GPUs SMOL runs on NVIDIA Ampere, Ada and Blackwell GPUs.

Check the download with sha256sum qwen38_27b_smol_df2.ginfer. TP2/TP4 packages of the same format will be added to this repo later.

Upstream models and licenses

  • Base model: Qwen/Qwen3.8-27B, revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0. License field apache-2.0; the repo LICENSE file is the Apache License 2.0 (full text in LICENSE here).
  • Drafter: incoai/Qwen3.8-27B-DFlash2, revision dedf8df68adfb1afeaf7b7480c0a0243108177b4. License field apache-2.0.

Quantized by Sectile Labs to SMOL 2-bit mix from the official BF16 weights of Qwen/Qwen3.8-27B (revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0) using a signed sensitivity map (uncalibrated splice; not the separately exported calibrated artifact), then converted to a .ginfer package.

This package is a conversion of the upstream weights into the GInfer format; GInfer does not change the upstream license. The upstream license text is in LICENSE.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Collection including SectileLabs/qwen38_27b_smol