Text Generation
Transformers
Safetensors
PyTorch
ONNX
English
qtensorformer
qtf
q-tensorformer
qtensor
high-throughput-inference
tensor-train
tensor-networks
tensorrt-llm
vllm
continuous-batching
paged-attention
speculative-decoding
zero-svd
adaptive-computation
dynamic-neural-networks
kv-cache
kv-cache-quantization
attention-sinks
lyapunov-stability
control-theory
contextual-bandits
edge-ai
edge-inference
on-device-ai
model-compression
hopper-h100
blackwell-b200
jetson-orin
jetson-thor
grouped-query-attention
rotary-position-embeddings
custom_code
academic-paper
ieee
neurips
vit-ap-university
deepseek
deepseek-r1
reasoning
chain-of-thought
gsm8k
openai-compatible
fastapi
rest-api
sse-streaming
onnx-runtime
embedded-ai
hardware-telemetry
power-efficiency
green-ai
raspberry-pi
smollm2
qwen2.5
llama-3.2
apple-silicon
Instructions to use Premchan369/Q-TensorFormer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Premchan369/Q-TensorFormer with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Premchan369/Q-TensorFormer", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("Premchan369/Q-TensorFormer", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Premchan369/Q-TensorFormer with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Premchan369/Q-TensorFormer" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Premchan369/Q-TensorFormer", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Premchan369/Q-TensorFormer
- SGLang
How to use Premchan369/Q-TensorFormer with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Premchan369/Q-TensorFormer" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Premchan369/Q-TensorFormer", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Premchan369/Q-TensorFormer" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Premchan369/Q-TensorFormer", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Premchan369/Q-TensorFormer with Docker Model Runner:
docker model run hf.co/Premchan369/Q-TensorFormer
Premchandyadav369
feat(seo): maximize discoverability, Google Scholar metadata, interactive HF widget, LLM crawler schemas, zero em dashes, and one-click quickstart downloads
5175adb Download site/architecture.html from Premchan369/Q-TensorFormer: direct link, hf CLI and curl.
- Browser
- Download file 7.87 kB
-
https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/site/architecture.html
- Command line
-
hf download hf://Premchan369/Q-TensorFormer/site/architecture.html
-
curl -L -o architecture.html https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/site/architecture.html
7.87 kB
| <html lang="en"> | |
| <head> | |
| <meta charset="UTF-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> | |
| <title>Architecture & Design - Q-TensorFormer</title> | |
| <meta name="description" content="Detailed architectural design of Q-TensorFormer: Nested Tensor-Train FFNs with zero-SVD dynamic rank slicing, 8D token information state engine, and zero-copy GQA."> | |
| <link rel="canonical" href="https://premchan369.github.io/Q-TensorFormer/architecture.html"> | |
| <meta property="og:type" content="article"> | |
| <meta property="og:site_name" content="Q-TensorFormer Research Project"> | |
| <meta property="og:title" content="Architecture & Design - Q-TensorFormer"> | |
| <meta property="og:description" content="Deep dive into Nested Tensor-Train FFNs, 8D information vectors, and zero-copy Grouped-Query Attention."> | |
| <meta property="og:url" content="https://premchan369.github.io/Q-TensorFormer/architecture.html"> | |
| <script type="application/ld+json"> | |
| { | |
| "@context": "https://schema.org", | |
| "@type": "TechArticle", | |
| "headline": "Q-TensorFormer Architecture and Structural Design", | |
| "description": "Exploration of nested Tensor-Train core slicing, token information state tracking, and memory-efficient attention.", | |
| "url": "https://premchan369.github.io/Q-TensorFormer/architecture.html", | |
| "author": { "@type": "Person", "name": "Premchand Yadav" }, | |
| "datePublished": "2026-10-13" | |
| } | |
| </script> | |
| <link rel="stylesheet" href="assets/style.css"> | |
| </head> | |
| <body> | |
| <header> | |
| <div class="header-container"> | |
| <div class="logo"> | |
| <a href="index.html"><span>Q-TensorFormer</span> Research</a> | |
| </div> | |
| <nav> | |
| <a href="index.html">Overview</a> | |
| <a href="architecture.html" class="active">Architecture</a> | |
| <a href="methodology.html">Methodology</a> | |
| <a href="benchmarks.html">Benchmarks</a> | |
| <a href="edge-inference.html">Edge Inference</a> | |
| <a href="training.html">Training</a> | |
| <a href="documentation.html">Docs & API</a> | |
| <a href="research.html">Research</a> | |
| <a href="limitations.html">Limitations</a> | |
| <a href="contributing.html">Contributing</a> | |
| </nav> | |
| </div> | |
| </header> | |
| <main class="container"> | |
| <h1>Architecture & System Design</h1> | |
| <p class="lead"> | |
| Q-TensorFormer decouples static parameter overprovisioning from dynamic inference execution by wrapping classical Transformer decoder blocks with closed-loop information tracking and nested tensor factorization. | |
| </p> | |
| <section> | |
| <h2>1. The 8-Dimensional Token Information State (z_t)</h2> | |
| <p> | |
| In standard Transformers, token representations pass through each layer without internal monitoring. Q-TensorFormer deploys a normalized 8D feature probe at the entrance of each block (implemented in <code>src/information_state.py</code>): | |
| </p> | |
| <div class="grid"> | |
| <div class="card"> | |
| <h4>S_t: Shannon Entropy</h4> | |
| <p>Measures the uncertainty of the token's predictive probability distribution over the vocabulary.</p> | |
| </div> | |
| <div class="card"> | |
| <h4>H_t: Hidden State L2 Norm</h4> | |
| <p>Quantifies representation magnitude, tracking whether activations are saturated or decaying.</p> | |
| </div> | |
| <div class="card"> | |
| <h4>U_t: Top-2 Prediction Margin</h4> | |
| <p>Difference between top-1 and top-2 token probabilities; small margins indicate epistemic ambiguity.</p> | |
| </div> | |
| <div class="card"> | |
| <h4>A_t: Normalized Position</h4> | |
| <p>Relative token position within the active context window (<code>t / max_seq_len</code>).</p> | |
| </div> | |
| <div class="card"> | |
| <h4>R_t: Residual Norm Delta</h4> | |
| <p>Rate of change between consecutive layer representations, detecting semantic convergence.</p> | |
| </div> | |
| <div class="card"> | |
| <h4>L_t: Latency Budget Drift</h4> | |
| <p>Tracks accumulated real-time inference latency relative to target SLO milestones.</p> | |
| </div> | |
| <div class="card"> | |
| <h4>M_t: Memory Footprint Stress</h4> | |
| <p>Monitors active KV-cache allocation relative to hardware DRAM/SRAM limits.</p> | |
| </div> | |
| <div class="card"> | |
| <h4>B_t: Dynamic Batch Pressure</h4> | |
| <p>Concurrency index to adjust shared hardware core contention on edge processors.</p> | |
| </div> | |
| </div> | |
| </section> | |
| <section> | |
| <h2>2. Nested Tensor-Train FFN (Zero-SVD Rank Slicing)</h2> | |
| <p> | |
| Standard linear feed-forward layers store an uncompressed 2D matrix <code>W ∈ R^{D × 4D}</code>. Q-TensorFormer factorizes <code>W</code> into a train of contracted 3-way cores <code>G^{(1)}, G^{(2)}, ..., G^{(d)}</code> using the Tensor-Train (TT) decomposition (<code>src/tensor_layers.py</code>): | |
| </p> | |
| <pre><code>W_{(i1..id), (o1..od)} = sum_{alpha_1..alpha_{d-1}} G^(1)_{1, o1, i1, alpha_1} * ... * G^(d)_{alpha_{d-1}, od, id, 1}</code></pre> | |
| <div class="callout"> | |
| <p><strong>Zero-SVD Slicing:</strong> Because the TT cores are nested, reducing the rank from 16 to 8, 4, 2, or 1 simply requires slicing the tensor view <code>core[..., :r]</code> without executing costly Singular Value Decomposition (SVD) at runtime. This provides instantaneous capacity switching in zero memory allocation time.</p> | |
| </div> | |
| </section> | |
| <section> | |
| <h2>3. Zero-Copy Grouped-Query Attention (GQA)</h2> | |
| <p> | |
| To prevent high KV-cache memory traffic, Q-TensorFormer pairs multi-head queries (<code>n_heads</code>) with grouped key-value heads (<code>n_kv_heads</code>). | |
| </p> | |
| <p> | |
| Unlike naive implementations that allocate duplicated contiguous tensors via <code>repeat_interleave</code>, Q-TensorFormer uses <strong>5D zero-copy broadcasting</strong> (<code>src/attention.py</code>): | |
| </p> | |
| <pre><code># Zero-copy view broadcasting without expanding K/V in memory: | |
| q_grouped = q.view(B, n_kv_heads, num_queries_per_kv, T, head_dim) | |
| k_grouped = k.unsqueeze(2) # Shape: (B, n_kv_heads, 1, T_k, head_dim) | |
| scores = torch.matmul(q_grouped, k_grouped.transpose(-2, -1)) * scale</code></pre> | |
| <p> | |
| This guarantees that K and V tensors share the exact same underlying storage pointer (<code>last_kv_shared_storage = True</code>), completely eliminating memory bloat. | |
| </p> | |
| </section> | |
| <section> | |
| <h2>4. IEEE-754 NaN-Safe Boolean Masking</h2> | |
| <p> | |
| Many open-source Transformer repositories contain a silent numerical bug where attention masks multiply padding zeros by negative infinity (<code>mask * float("-inf")</code>). In IEEE-754 arithmetic, <code>0.0 * -inf = NaN</code>! | |
| </p> | |
| <p> | |
| Q-TensorFormer replaces this pattern with boolean <code>masked_fill</code> and guards against completely padded query rows (e.g. left-padding in mixed batches): | |
| </p> | |
| <pre><code>row_all_masked = hide_mask.all(dim=-1, keepdim=True) | |
| safe_hide = hide_mask & ~row_all_masked | |
| masked_scores = scores.masked_fill(safe_hide, float("-inf")) | |
| weights = F.softmax(masked_scores, dim=-1).masked_fill(row_all_masked, 0.0)</code></pre> | |
| </section> | |
| </main> | |
| <footer> | |
| <div class="footer-links"> | |
| <a href="index.html">Home</a> | |
| <a href="architecture.html">Architecture</a> | |
| <a href="methodology.html">Methodology</a> | |
| <a href="benchmarks.html">Benchmarks</a> | |
| <a href="documentation.html">Docs</a> | |
| <a href="limitations.html">Limitations</a> | |
| <a href="sitemap.xml">Sitemap</a> | |
| <a href="llms.txt">llms.txt</a> | |
| <a href="https://huggingface.co/Premchan369/Q-TensorFormer">Hugging Face</a> | |
| </div> | |
| <p>© 2026 Premchand Yadav. Licensed under Apache-2.0.</p> | |
| </footer> | |
| </body> | |
| </html> | |