title: Q-TensorFormer
emoji: ⚛️
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 6.30.0
python_version: '3.12'
app_file: app.py
pinned: false
license: apache-2.0
short_description: Real-Time High-Throughput Edge LLM Inference
models:
- Premchan369/Q-TensorFormer
tags:
- edge-ai
- reasoning
- deepseek-style
- openai-api
- tensor-train
- hardware-adaptive
- green-ai
- high-throughput
- vit-ap-university
⚛️ Q-TensorFormer: Real-Time High-Throughput Edge LLM Inference
Welcome to the live interactive research showcase of Q-TensorFormer, a real-time high-throughput edge LLM inference architecture developed by Dr. Madala Guru Brahmam (Assistant Professor) and V C Premchand Yadav (Final Year B.Tech CSE AIML) at VIT-AP University, Amaravati, Andhra Pradesh, India.
📄 Instant Paper Download: Download Camera-Ready 6-Page IEEE Conference Paper (PDF)
📦 Pretrained Weights & Code: Premchan369/Q-TensorFormer on Hugging Face Hub
What This Space Demonstrates:
- 📖 Beginner & Layman Interactive Guide: Test drive the automatic transmission AI car across driving conditions and explore the 5 core inventions with zero math.
- 💬 Live Dynamic Tensor-Train Slicing: Type any prompt to watch the model slice Tensor-Train rank cores ($r \in [4, 16]$) token-by-token based on marginal informational value and target energy budgets with live lookahead horizon predictions.
- 🔮 Edge Digital Twin & Predictive Future Forecaster: Simulate real-time physical dynamics into the future (5 to 60 minutes) across 5 hardware platforms (Raspberry Pi 5, Jetson Orin Nano, Apple M3/M4, Drone MCU, Intel Core Ultra 7) forecasting junction temperature ($T_j$), thermal throttling cliffs, battery depletion, and zero-chattering rank evolution.
- 🌐 Autonomous Swarm & Global Fleet ROI Calculator: Predict fleet survival across 10-100 edge nodes, and compute annual MWh saved, tons of CO2 avoided, and cloud hosting cost reductions.
- 🎛️ Real-Time Dynamic SLA Controller Sandbox: Test closed-loop PID disturbance rejection with live phase portraits and zero limit-cycle chattering ($\Delta V_t \le 0$).
- ⚡ Zero-SVD Speculative Self-Drafting Engine: Single-model self-speculative decoding simulator ($r_{\text{draft}}=1 \to r_{\text{target}}=4$) delivering $1.42\times$ acceleration with 0.00 MB extra DRAM allocation.
- 🔬 Core Slicing & Singular Spectrum Inspector: Direct mathematical inspection of TT core shapes ($\mathcal{G}^{(1)}, \mathcal{G}^{(2)}, \mathcal{G}^{(3)}$) and interactive singular value energy waterfall chart ($\sum_{i=1}^r \sigma_i^2 / \sum \sigma_i^2$), validating Theorem 1.
- 👁️ Multimodal Vision Projection: Spatial-variance-guided visual patch ranking (Rank 16 for foreground, Rank 2 for background, 65.2% FLOP reduction).
- ⚡ LLM Compression Studio: Interactive parameter savings calculator for open-weight models (Llama-3, Qwen, Mistral).
- 🎓 NeurIPS & IEEE Research Deep-Dive & Paper Download: 1-click download of the camera-ready 6-page IEEE conference paper PDF, plus interactive verification of Theorems 1-4 and 50-trial empirical hypothesis testing.
- Official Repository: https://huggingface.co/Premchan369/Q-TensorFormer
- Technical Documentation Portal: https://premchan369.github.io/Q-TensorFormer/
📖 Beginner & Layman's Guide: How Q-TensorFormer Works (No Math Required)
1. The Big Problem: Why Today's AI Overheats Your Devices
Imagine driving a car where the gas pedal is permanently glued to the floor at 150 mph - even when stopped at a red light. That is how standard LLMs run today: they spend the exact same massive computation and memory on simple filler words (like "the", "and") as on complex reasoning problems.
2. The Solution: The Smart Reader with Adaptive Gears
Q-TensorFormer turns the model into an adaptive reader equipped with automatic transmission, cruise control, and selective memory.
Figure 1: Complete end-to-end flow of Q-TensorFormer. Tokens enter, are measured by lightweight sensors, routed through compressed tensor layers, and regulated by closed-loop hardware feedback.
🟢 Step 1: The Sensor Gauge
Before spending compute, a lightweight sensor checks: Is this word difficult? Is the battery low? Is memory running out?
Figure 2: Relative importance of the 8 real-time sensor signals.
🟢 Step 2: Russian Nesting Dolls (Nested Tensor-Train Weights)
Instead of huge flat matrices, weights are stored as compact tensor cores. For easy words, we open only the small inner doll (Rank 1 or 2); for hard words, we open the full doll (Rank 8 or 16). Slicing happens instantly in micro-seconds!
Figure 3: Representation quality across nested Tensor-Train ranks.
Figure 4: Active rank directly controls latency and DRAM bus traffic.
🟢 Step 3: Cruise Control with Anti-Pothole Buffer (Hysteresis PID Controller)
A closed-loop PID controller prevents erratic speed spikes. A 0.15 hysteresis deadband ensures the model doesn't flicker between consecutive words, eliminating jitter.
Figure 5: Closed-loop PID controller tracking target latency budgets.
Figure 6: Rank-switching chatter drops from 25.40% down to 0.00%.
🟢 Step 4: The Selective Highlighter & Attic Trunk (Semantic Anchor KV Cache)
Instead of keeping every past word in full memory, we keep the first 4 anchor words (attention sinks) and critical facts in crisp full fidelity, and compress the rest to 8-bit/4-bit integers.
Figure 7: Memory vs attention fidelity curve.
Figure 8: Attention memory traffic reduction across context lengths.
🟢 Step 5: The In-Kitchen Chef (Fused Hardware Contraction Engine)
Instead of sending numbers back and forth to slow DRAM memory, our fused kernel keeps numbers directly in the processor's fast registers, cutting memory bus traffic by 96.90%!
Figure 9: Arithmetic intensity and memory bandwidth roofline.
🟢 Step 6: Multi-Dimensional Comparison Radar
Figure 10: Multi-dimensional radar chart comparing Q-TensorFormer against standard dense baselines across parameter size, memory footprint, bus traffic, cold-start time, and energy efficiency.