Q-TensorFormer / site /architecture.html
Premchandyadav369
feat(seo): maximize discoverability, Google Scholar metadata, interactive HF widget, LLM crawler schemas, zero em dashes, and one-click quickstart downloads
5175adb
Raw History Blame Contribute Delete
7.87 kB
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Architecture &amp; Design - Q-TensorFormer</title>
<meta name="description" content="Detailed architectural design of Q-TensorFormer: Nested Tensor-Train FFNs with zero-SVD dynamic rank slicing, 8D token information state engine, and zero-copy GQA.">
<link rel="canonical" href="https://premchan369.github.io/Q-TensorFormer/architecture.html">
<meta property="og:type" content="article">
<meta property="og:site_name" content="Q-TensorFormer Research Project">
<meta property="og:title" content="Architecture &amp; Design - Q-TensorFormer">
<meta property="og:description" content="Deep dive into Nested Tensor-Train FFNs, 8D information vectors, and zero-copy Grouped-Query Attention.">
<meta property="og:url" content="https://premchan369.github.io/Q-TensorFormer/architecture.html">
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "TechArticle",
"headline": "Q-TensorFormer Architecture and Structural Design",
"description": "Exploration of nested Tensor-Train core slicing, token information state tracking, and memory-efficient attention.",
"url": "https://premchan369.github.io/Q-TensorFormer/architecture.html",
"author": { "@type": "Person", "name": "Premchand Yadav" },
"datePublished": "2026-10-13"
}
</script>
<link rel="stylesheet" href="assets/style.css">
</head>
<body>
<header>
<div class="header-container">
<div class="logo">
<a href="index.html"><span>Q-TensorFormer</span> Research</a>
</div>
<nav>
<a href="index.html">Overview</a>
<a href="architecture.html" class="active">Architecture</a>
<a href="methodology.html">Methodology</a>
<a href="benchmarks.html">Benchmarks</a>
<a href="edge-inference.html">Edge Inference</a>
<a href="training.html">Training</a>
<a href="documentation.html">Docs & API</a>
<a href="research.html">Research</a>
<a href="limitations.html">Limitations</a>
<a href="contributing.html">Contributing</a>
</nav>
</div>
</header>
<main class="container">
<h1>Architecture &amp; System Design</h1>
<p class="lead">
Q-TensorFormer decouples static parameter overprovisioning from dynamic inference execution by wrapping classical Transformer decoder blocks with closed-loop information tracking and nested tensor factorization.
</p>
<section>
<h2>1. The 8-Dimensional Token Information State (z_t)</h2>
<p>
In standard Transformers, token representations pass through each layer without internal monitoring. Q-TensorFormer deploys a normalized 8D feature probe at the entrance of each block (implemented in <code>src/information_state.py</code>):
</p>
<div class="grid">
<div class="card">
<h4>S_t: Shannon Entropy</h4>
<p>Measures the uncertainty of the token's predictive probability distribution over the vocabulary.</p>
</div>
<div class="card">
<h4>H_t: Hidden State L2 Norm</h4>
<p>Quantifies representation magnitude, tracking whether activations are saturated or decaying.</p>
</div>
<div class="card">
<h4>U_t: Top-2 Prediction Margin</h4>
<p>Difference between top-1 and top-2 token probabilities; small margins indicate epistemic ambiguity.</p>
</div>
<div class="card">
<h4>A_t: Normalized Position</h4>
<p>Relative token position within the active context window (<code>t / max_seq_len</code>).</p>
</div>
<div class="card">
<h4>R_t: Residual Norm Delta</h4>
<p>Rate of change between consecutive layer representations, detecting semantic convergence.</p>
</div>
<div class="card">
<h4>L_t: Latency Budget Drift</h4>
<p>Tracks accumulated real-time inference latency relative to target SLO milestones.</p>
</div>
<div class="card">
<h4>M_t: Memory Footprint Stress</h4>
<p>Monitors active KV-cache allocation relative to hardware DRAM/SRAM limits.</p>
</div>
<div class="card">
<h4>B_t: Dynamic Batch Pressure</h4>
<p>Concurrency index to adjust shared hardware core contention on edge processors.</p>
</div>
</div>
</section>
<section>
<h2>2. Nested Tensor-Train FFN (Zero-SVD Rank Slicing)</h2>
<p>
Standard linear feed-forward layers store an uncompressed 2D matrix <code>W &isin; R^{D &times; 4D}</code>. Q-TensorFormer factorizes <code>W</code> into a train of contracted 3-way cores <code>G^{(1)}, G^{(2)}, ..., G^{(d)}</code> using the Tensor-Train (TT) decomposition (<code>src/tensor_layers.py</code>):
</p>
<pre><code>W_{(i1..id), (o1..od)} = sum_{alpha_1..alpha_{d-1}} G^(1)_{1, o1, i1, alpha_1} * ... * G^(d)_{alpha_{d-1}, od, id, 1}</code></pre>
<div class="callout">
<p><strong>Zero-SVD Slicing:</strong> Because the TT cores are nested, reducing the rank from 16 to 8, 4, 2, or 1 simply requires slicing the tensor view <code>core[..., :r]</code> without executing costly Singular Value Decomposition (SVD) at runtime. This provides instantaneous capacity switching in zero memory allocation time.</p>
</div>
</section>
<section>
<h2>3. Zero-Copy Grouped-Query Attention (GQA)</h2>
<p>
To prevent high KV-cache memory traffic, Q-TensorFormer pairs multi-head queries (<code>n_heads</code>) with grouped key-value heads (<code>n_kv_heads</code>).
</p>
<p>
Unlike naive implementations that allocate duplicated contiguous tensors via <code>repeat_interleave</code>, Q-TensorFormer uses <strong>5D zero-copy broadcasting</strong> (<code>src/attention.py</code>):
</p>
<pre><code># Zero-copy view broadcasting without expanding K/V in memory:
q_grouped = q.view(B, n_kv_heads, num_queries_per_kv, T, head_dim)
k_grouped = k.unsqueeze(2) # Shape: (B, n_kv_heads, 1, T_k, head_dim)
scores = torch.matmul(q_grouped, k_grouped.transpose(-2, -1)) * scale</code></pre>
<p>
This guarantees that K and V tensors share the exact same underlying storage pointer (<code>last_kv_shared_storage = True</code>), completely eliminating memory bloat.
</p>
</section>
<section>
<h2>4. IEEE-754 NaN-Safe Boolean Masking</h2>
<p>
Many open-source Transformer repositories contain a silent numerical bug where attention masks multiply padding zeros by negative infinity (<code>mask * float("-inf")</code>). In IEEE-754 arithmetic, <code>0.0 * -inf = NaN</code>!
</p>
<p>
Q-TensorFormer replaces this pattern with boolean <code>masked_fill</code> and guards against completely padded query rows (e.g. left-padding in mixed batches):
</p>
<pre><code>row_all_masked = hide_mask.all(dim=-1, keepdim=True)
safe_hide = hide_mask &amp; ~row_all_masked
masked_scores = scores.masked_fill(safe_hide, float("-inf"))
weights = F.softmax(masked_scores, dim=-1).masked_fill(row_all_masked, 0.0)</code></pre>
</section>
</main>
<footer>
<div class="footer-links">
<a href="index.html">Home</a>
<a href="architecture.html">Architecture</a>
<a href="methodology.html">Methodology</a>
<a href="benchmarks.html">Benchmarks</a>
<a href="documentation.html">Docs</a>
<a href="limitations.html">Limitations</a>
<a href="sitemap.xml">Sitemap</a>
<a href="llms.txt">llms.txt</a>
<a href="https://huggingface.co/Premchan369/Q-TensorFormer">Hugging Face</a>
</div>
<p>&copy; 2026 Premchand Yadav. Licensed under Apache-2.0.</p>
</footer>
</body>
</html>