README.md

---
title: 88plug
---

<div style="font-family: system-ui, -apple-system, sans-serif; max-width: 860px; margin: 0 auto; padding: 2rem 1rem; color: #1a1a1a;">

<h1 style="font-size: 1.75rem; font-weight: 700; margin-bottom: 0.25rem;">88plug</h1>
<p style="color: #6b7280; font-size: 1rem; margin-top: 0; margin-bottom: 2rem;">
  Post-training quantization for production GPU inference
</p>

<hr style="border: none; border-top: 1px solid #e5e7eb; margin-bottom: 2rem;" />

<h2 style="font-size: 1.1rem; font-weight: 600; margin-bottom: 0.75rem;">What we publish</h2>
<!-- CATALOG_STATS -->
<p style="color:#6b7280;font-size:0.95rem;">Canonical catalog: <strong>48</strong> repos · product_done: <strong>10/48</strong> (gold/provisional + smoke=pass + HF).</p>

<p style="line-height: 1.7; margin-bottom: 1rem;">
  Compressed-tensors models quantized for native
  <a href="https://github.com/vllm-project/vllm" style="color: #2563eb;">vLLM</a> deployment.
  No extra flags, no monkey-patching — weights load automatically on any <strong>Ampere+ GPU</strong>.
</p>

<table style="width: 100%; border-collapse: collapse; margin-bottom: 2rem; font-size: 0.9rem;">
  <thead>
    <tr style="border-bottom: 2px solid #e5e7eb;">
      <th style="text-align: left; padding: 0.5rem 0.75rem; font-weight: 600;">Format</th>
      <th style="text-align: left; padding: 0.5rem 0.75rem; font-weight: 600;">Precision</th>
      <th style="text-align: left; padding: 0.5rem 0.75rem; font-weight: 600;">Method</th>
      <th style="text-align: left; padding: 0.5rem 0.75rem; font-weight: 600;">Use case</th>
    </tr>
  </thead>
  <tbody>
    <tr style="border-bottom: 1px solid #f3f4f6;">
      <td style="padding: 0.5rem 0.75rem;"><code style="background:#f3f4f6;padding:2px 6px;border-radius:4px;">W8A16</code></td>
      <td style="padding: 0.5rem 0.75rem;">INT8 weights · BF16 activations</td>
      <td style="padding: 0.5rem 0.75rem;">compressed-tensors PTQ</td>
      <td style="padding: 0.5rem 0.75rem;">Max throughput, near-lossless quality</td>
    </tr>
    <tr>
      <td style="padding: 0.5rem 0.75rem;"><code style="background:#f3f4f6;padding:2px 6px;border-radius:4px;">W4A16</code></td>
      <td style="padding: 0.5rem 0.75rem;">INT4 weights · BF16 activations</td>
      <td style="padding: 0.5rem 0.75rem;">AutoRound (iters=200)</td>
      <td style="padding: 0.5rem 0.75rem;">Lowest VRAM footprint, high accuracy</td>
    </tr>
  </tbody>
</table>

<h2 style="font-size: 1.1rem; font-weight: 600; margin-bottom: 0.75rem;">Quick start</h2>
<pre style="background:#f8fafc;border:1px solid #e2e8f0;border-radius:8px;padding:1rem;overflow-x:auto;font-size:0.85rem;line-height:1.6;"><code>pip install vllm&gt;=0.21.0

vllm serve 88plug/&lt;model-name&gt; --dtype bfloat16</code></pre>

<p style="line-height: 1.7; margin-top: 1rem; margin-bottom: 2rem;">
  No <code>--quantization</code> flag required. The compressed-tensors config is embedded in the model weights
  and auto-detected by vLLM v0.21.0+.
</p>

<h2 style="font-size: 1.1rem; font-weight: 600; margin-bottom: 0.75rem;">Hardware</h2>
<ul style="line-height: 1.8; padding-left: 1.25rem; margin-bottom: 2rem;">
  <li>Quantized on <strong>NVIDIA RTX A6000</strong> (48 GB)</li>
  <li>Tested on Ampere, Ada Lovelace, and Hopper architectures</li>
  <li>W8A16 models require ~50% the VRAM of BF16</li>
  <li>W4A16 models require ~25% the VRAM of BF16</li>
</ul>

<h2 style="font-size: 1.1rem; font-weight: 600; margin-bottom: 0.75rem;">Stack</h2>
<ul style="line-height: 1.8; padding-left: 1.25rem; margin-bottom: 0;">
  <li><a href="https://github.com/vllm-project/vllm" style="color:#2563eb;">vLLM</a> — inference engine</li>
  <li><a href="https://github.com/neuralmagic/llm-compressor" style="color:#2563eb;">llm-compressor</a> — quantization pipeline</li>
  <li><a href="https://github.com/intel/auto-round" style="color:#2563eb;">AutoRound</a> — W4A16 calibration (iters=200)</li>
  <li><a href="https://github.com/neuralmagic/compressed-tensors" style="color:#2563eb;">compressed-tensors</a> — weight format</li>
</ul>

</div>