Hugging Face's logo

Premchan369
/
Q-TensorFormer

Text Generation
Transformers
Safetensors
PyTorch
ONNX
English
qtensorformer
q-tensorformer
high-throughput-inference
tensor-train
tensor-networks
tensorrt-llm
vllm
continuous-batching
paged-attention
speculative-decoding
zero-svd
adaptive-computation
dynamic-neural-networks
kv-cache
kv-cache-quantization
attention-sinks
lyapunov-stability
control-theory
contextual-bandits
edge-ai
edge-inference
on-device-ai
model-compression
hopper-h100
blackwell-b200
jetson-orin
jetson-thor
grouped-query-attention
rotary-position-embeddings
custom_code
academic-paper
ieee
neurips
vit-ap-university
deepseek
deepseek-r1
reasoning
chain-of-thought
gsm8k
openai-compatible
fastapi
rest-api
sse-streaming
onnx-runtime
embedded-ai
hardware-telemetry
power-efficiency
green-ai
raspberry-pi
smollm2
qwen2.5
llama-3.2
apple-silicon
Model card Files
xet
 Community
Q-TensorFormer / site /benchmarks.html
Premchandyadav369
feat(seo): maximize discoverability, Google Scholar metadata, interactive HF widget, LLM crawler schemas, zero em dashes, and one-click quickstart downloads
5175adb
Raw History Blame Contribute Delete
9.74 kB
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Empirical Benchmarks &amp; Evaluation - Q-TensorFormer</title>
<meta name="description" content="Reproducible empirical benchmarks for Q-TensorFormer: Tensor-Train parameter compression, CPU latency, TTFT, cold-start loading, and multi-precision KV caching.">
<link rel="canonical" href="https://premchan369.github.io/Q-TensorFormer/benchmarks.html">
<meta property="og:type" content="article">
<meta property="og:site_name" content="Q-TensorFormer Research Project">
<meta property="og:title" content="Empirical Benchmarks &amp; Evaluation - Q-TensorFormer">
<meta property="og:description" content="Detailed reproducible hardware measurements across latency, memory, parameters, and perplexity.">
<meta property="og:url" content="https://premchan369.github.io/Q-TensorFormer/benchmarks.html">
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "TechArticle",
"headline": "Q-TensorFormer Empirical Benchmarks and Performance Evaluation",
"description": "Verifiable latency, memory bandwidth, parameter compression, and perplexity measurements.",
"url": "https://premchan369.github.io/Q-TensorFormer/benchmarks.html",
"author": { "@type": "Person", "name": "Premchand Yadav" },
"datePublished": "2026-10-13"
}
</script>
<link rel="stylesheet" href="assets/style.css">
</head>
<body>
<header>
<div class="header-container">
<div class="logo">
<a href="index.html"><span>Q-TensorFormer</span> Research</a>
</div>
<nav>
<a href="index.html">Overview</a>
<a href="architecture.html">Architecture</a>
<a href="methodology.html">Methodology</a>
<a href="benchmarks.html" class="active">Benchmarks</a>
<a href="edge-inference.html">Edge Inference</a>
<a href="training.html">Training</a>
<a href="documentation.html">Docs & API</a>
<a href="research.html">Research</a>
<a href="limitations.html">Limitations</a>
<a href="contributing.html">Contributing</a>
</nav>
</div>
</header>
<main class="container">
<h1>Empirical Benchmarks &amp; Evaluation</h1>
<p class="lead">
All figures reported below are generated from actual execution runs on host hardware (Intel 20-thread CPU, PyTorch 2.10.0+cpu, Windows 10) under test identifier <code>RUN-20261013-QTF-HARDENING</code>.
</p>
<div class="callout">
<p><strong>Scientific Integrity Classification:</strong> Every reported metric is tagged as <code>MEASURED</code> (recorded from real hardware timers and memory allocation), <code>SIMULATED</code> (statevector emulation), or <code>ESTIMATED</code> (analytical cost models). Zero metrics are fabricated or projected.</p>
</div>
<section>
<h2>1. Language Model Training &amp; Quality Validation</h2>
<p>
Evaluated on <code>data/qtensorformer_corpus.txt</code> across 70% train, 15% validation, and 15% test splits (source: <code>results/raw/training_and_quality_validation.json</code>):
</p>
<table>
<thead>
<tr>
<th>Architecture</th>
<th>Parameters</th>
<th>Initial Test Loss</th>
<th>Initial Perplexity</th>
<th>Final Test Loss</th>
<th>Final Perplexity</th>
<th>Perplexity Drop</th>
<th>Label</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>QTensorFormer (Canonical)</strong></td>
<td>68,604</td>
<td>5.8646</td>
<td>352.35</td>
<td><strong>3.8086</strong></td>
<td><strong>45.09</strong></td>
<td><strong>-87.2%</strong></td>
<td>MEASURED</td>
</tr>
<tr>
<td><strong>DenseBaseline (Matched)</strong></td>
<td>116,480</td>
<td>5.7282</td>
<td>307.40</td>
<td><strong>3.8242</strong></td>
<td><strong>45.80</strong></td>
<td><strong>-85.1%</strong></td>
<td>MEASURED</td>
</tr>
</tbody>
</table>
</section>
<section>
<h2>2. Layer-Level Tensor-Train Rank Sweep (512 &rarr; 2048 Projection)</h2>
<p>
40 timed runs per rank level; zero-SVD rank slicing overhead measured at 98.29 &mu;s (source: <code>results/raw/tt_compression_audit.json</code>):
</p>
<table>
<thead>
<tr>
<th>Layer / Rank</th>
<th>Active Params</th>
<th>Weight Bytes</th>
<th>Compression Ratio</th>
<th>Param Reduction</th>
<th>Frobenius Error</th>
<th>Prefill (B=4, T=64)</th>
<th>Label</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>nn.Linear</code> (Dense)</td>
<td>1,050,624</td>
<td>4,202,496 B</td>
<td>1.00x</td>
<td>0.00%</td>
<td>0.0</td>
<td>3.59 ms</td>
<td>MEASURED</td>
</tr>
<tr>
<td><code>TTLinear</code> (r=16)</td>
<td>266,304</td>
<td>1,065,216 B</td>
<td>3.95x</td>
<td>74.65%</td>
<td>1.3e-6</td>
<td>13.49 ms</td>
<td>MEASURED</td>
</tr>
<tr>
<td><code>TTLinear</code> (r=8)</td>
<td>133,664</td>
<td>534,656 B</td>
<td>7.86x</td>
<td>87.28%</td>
<td>5.0e-7</td>
<td>7.71 ms</td>
<td>MEASURED</td>
</tr>
<tr>
<td><code>TTLinear</code> (r=4)</td>
<td>67,728</td>
<td>270,912 B</td>
<td>15.51x</td>
<td>93.55%</td>
<td>4.0e-7</td>
<td>5.35 ms</td>
<td>MEASURED</td>
</tr>
<tr>
<td><code>TTLinear</code> (r=2)</td>
<td>34,856</td>
<td>139,424 B</td>
<td>30.14x</td>
<td>96.68%</td>
<td>1.0e-7</td>
<td>4.17 ms</td>
<td>MEASURED</td>
</tr>
<tr>
<td><code>TTLinear</code> (r=1)</td>
<td>18,444</td>
<td>73,776 B</td>
<td><strong>56.96x</strong></td>
<td><strong>98.24%</strong></td>
<td>0.0</td>
<td>5.93 ms</td>
<td>MEASURED</td>
</tr>
</tbody>
</table>
</section>
<section>
<h2>3. Adaptive KV-Cache Scaling at Sequence Length L=1024</h2>
<p>
4 sink tokens protected, maximum eviction capacity 128 (source: <code>results/raw/kv_cache_audit.json</code>):
</p>
<table>
<thead>
<tr>
<th>Cache Configuration</th>
<th>Physical Cache Bytes</th>
<th>Memory Reduction vs FP16</th>
<th>Output Cosine Similarity</th>
<th>Evaluation Finding</th>
<th>Label</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>FP16 Full Cache</code></td>
<td>524,288 B</td>
<td>0.00%</td>
<td>1.000000</td>
<td>Baseline uncompressed reference</td>
<td>MEASURED</td>
</tr>
<tr>
<td><code>INT8 Full Cache</code></td>
<td>278,528 B</td>
<td>-46.88% (1.88x)</td>
<td>0.999973</td>
<td>Near-lossless quantization</td>
<td>MEASURED</td>
</tr>
<tr>
<td><code>INT4 Packed Full Cache</code></td>
<td>163,840 B</td>
<td>-68.75% (3.20x)</td>
<td>0.995593</td>
<td>High compression with minimal drift</td>
<td>MEASURED</td>
</tr>
<tr>
<td><code>Sink-Protected INT8 (cap=128)</code></td>
<td><strong>34,816 B</strong></td>
<td><strong>-93.36% (15.06x)</strong></td>
<td><strong>0.996337</strong></td>
<td><strong>Bounded memory ceiling, stable attention</strong></td>
<td>MEASURED</td>
</tr>
<tr>
<td><code>Sliding Window without Sinks</code></td>
<td>262,144 B</td>
<td>-50.00%</td>
<td>0.004714</td>
<td>Severe attention collapse</td>
<td>MEASURED</td>
</tr>
</tbody>
</table>
</section>
<section>
<h2>4. Reproduction Commands</h2>
<p>To reproduce all benchmark figures directly on your hardware:</p>
<pre><code># Run full scientific benchmark suite
python scripts/run_scientific_benchmark.py
# Run training quality validation
python scripts/train_and_evaluate.py
# Inspect results
cat results/processed/benchmark_summary.csv</code></pre>
</section>
</main>
<footer>
<div class="footer-links">
<a href="index.html">Home</a>
<a href="architecture.html">Architecture</a>
<a href="methodology.html">Methodology</a>
<a href="benchmarks.html">Benchmarks</a>
<a href="documentation.html">Docs</a>
<a href="limitations.html">Limitations</a>
<a href="sitemap.xml">Sitemap</a>
<a href="llms.txt">llms.txt</a>
<a href="https://huggingface.co/Premchan369/Q-TensorFormer">Hugging Face</a>
</div>
<p>&copy; 2026 Premchand Yadav. Licensed under Apache-2.0.</p>
</footer>
</body>
</html>