ginfer

Qwen3.8-27B, GInfer INT

This repo holds the GInfer packages of Qwen3.8-27B in INT: single-file packages for the GInfer inference engine (and GChat, which is built on it). They run only on GInfer / GChat. They are not safetensors or GGUF files and other runtimes cannot load them.

GInfer is Sectile Labs' inference engine for fast, high-concurrency, long-context work on NVIDIA GPUs.

Packages

File Drafter Tensor-parallel size Size in bytes SHA-256
qwen38_27b_int_df2.ginfer Q4 DFlash2 1 (one GPU) 18,424,985,856 bcf29abd1598f5666c641da7c59ac28bd999aa98a8d6ba533ae7aa579abd06a7
qwen38_27b_int_df2_tp2.ginfer Q4 DFlash2 2 (two GPUs) 18,428,540,928 fd2a26f09d577b6200168acc63c2d6af923eace1690ecadabc5df48aa5c96b76

Format: INT (groupwise integer weights).

GPUs: INT runs on NVIDIA Ampere, Ada and Blackwell GPUs.

The TP2 packages split the model across two identical GPUs; load them with tensor-parallel size 2. TP4 packages will be added later.

Check each download with sha256sum qwen38_27b_int_df2.ginfer or sha256sum qwen38_27b_int_df2_tp2.ginfer.

Upstream models and licenses

  • Base model: Qwen/Qwen3.8-27B, revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0. License field apache-2.0; the repo LICENSE file is the Apache License 2.0 (full text in LICENSE here).
  • Drafter: incoai/Qwen3.8-27B-DFlash2, revision dedf8df68adfb1afeaf7b7480c0a0243108177b4. License field apache-2.0.

Quantized by Sectile Labs to groupwise INT (AutoRound-based) from the official BF16 weights of Qwen/Qwen3.8-27B (revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0), then converted to a .ginfer package.

This package is a conversion of the upstream weights into the GInfer format; GInfer does not change the upstream license. The upstream license text is in LICENSE.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Collection including SectileLabs/qwen38_27b_int