Spaces:
Running on Zero
Running on Zero
Premchandyadav369
feat: QTF shortname branding, reasoning complexity studio tab, and unified CLI
cb99c21 |
Download README.md from Premchan369/Q-TensorFormer: direct link, hf CLI and curl.
- Browser
- Download file 9.09 kB
-
https://huggingface.co/spaces/Premchan369/Q-TensorFormer/resolve/main/README.md
- Command line
-
hf download hf://spaces/Premchan369/Q-TensorFormer/README.md
-
curl -L -o README.md https://huggingface.co/spaces/Premchan369/Q-TensorFormer/resolve/main/README.md
9.09 kB
| title: Q-TensorFormer | |
| emoji: ⚛️ | |
| colorFrom: indigo | |
| colorTo: purple | |
| sdk: gradio | |
| sdk_version: 6.30.0 | |
| python_version: '3.12' | |
| app_file: app.py | |
| pinned: false | |
| license: apache-2.0 | |
| short_description: Real-Time High-Throughput Edge LLM Inference | |
| models: | |
| - Premchan369/Q-TensorFormer | |
| tags: | |
| - qtf | |
| - edge-ai | |
| - reasoning | |
| - deepseek-style | |
| - openai-api | |
| - tensor-train | |
| - hardware-adaptive | |
| - green-ai | |
| - high-throughput | |
| - vit-ap-university | |
| # ⚛️ Q-TensorFormer (QTF): Real-Time High-Throughput Edge LLM Inference | |
| [](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/paper/q_tensorformer_ieee_paper.pdf) | |
| [](https://premchan369.github.io/Q-TensorFormer/) | |
| [](https://huggingface.co/Premchan369/Q-TensorFormer) | |
| [](https://colab.research.google.com/github/Premchan369/Q-TensorFormer/blob/main/notebooks/quickstart.ipynb) | |
| [](LICENSE) | |
| [](https://huggingface.co/Premchan369/Q-TensorFormer) | |
| Welcome to the live interactive research showcase of **Q-TensorFormer**, a real-time high-throughput edge LLM inference architecture developed by **Dr. Madala Guru Brahmam** (Assistant Professor) and **V C Premchand Yadav** (Final Year B.Tech CSE AIML) at **VIT-AP University**, Amaravati, Andhra Pradesh, India. | |
| > 📄 **Instant Paper Download:** [Download Camera-Ready 6-Page IEEE Conference Paper (PDF)](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/paper/q_tensorformer_ieee_paper.pdf) | |
| > 📦 **Pretrained Weights & Code:** [Premchan369/Q-TensorFormer on Hugging Face Hub](https://huggingface.co/Premchan369/Q-TensorFormer) | |
| ### What This Space Demonstrates: | |
| 1. **📖 Beginner & Layman Interactive Guide:** Test drive the automatic transmission AI car across driving conditions and explore the 5 core inventions with zero math. | |
| 2. **💬 Live Dynamic Tensor-Train Slicing:** Type any prompt to watch the model slice Tensor-Train rank cores ($r \in [4, 16]$) token-by-token based on marginal informational value and target energy budgets with live lookahead horizon predictions. | |
| 3. **🔮 Edge Digital Twin & Predictive Future Forecaster:** Simulate real-time physical dynamics into the future (5 to 60 minutes) across 5 hardware platforms (Raspberry Pi 5, Jetson Orin Nano, Apple M3/M4, Drone MCU, Intel Core Ultra 7) forecasting junction temperature ($T_j$), thermal throttling cliffs, battery depletion, and zero-chattering rank evolution. | |
| 4. **🌐 Autonomous Swarm & Global Fleet ROI Calculator:** Predict fleet survival across 10-100 edge nodes, and compute annual MWh saved, tons of CO2 avoided, and cloud hosting cost reductions. | |
| 5. **🎛️ Real-Time Dynamic SLA Controller Sandbox:** Test closed-loop PID disturbance rejection with live phase portraits and zero limit-cycle chattering ($\Delta V_t \le 0$). | |
| 6. **⚡ Zero-SVD Speculative Self-Drafting Engine:** Single-model self-speculative decoding simulator ($r_{\text{draft}}=1 \to r_{\text{target}}=4$) delivering $1.42\times$ acceleration with 0.00 MB extra DRAM allocation. | |
| 7. **🔬 Core Slicing & Singular Spectrum Inspector:** Direct mathematical inspection of TT core shapes ($\mathcal{G}^{(1)}, \mathcal{G}^{(2)}, \mathcal{G}^{(3)}$) and interactive singular value energy waterfall chart ($\sum_{i=1}^r \sigma_i^2 / \sum \sigma_i^2$), validating Theorem 1. | |
| 8. **👁️ Multimodal Vision Projection:** Spatial-variance-guided visual patch ranking (Rank 16 for foreground, Rank 2 for background, 65.2% FLOP reduction). | |
| 9. **⚡ LLM Compression Studio:** Interactive parameter savings calculator for open-weight models (Llama-3, Qwen, Mistral). | |
| 10. **🎓 NeurIPS & IEEE Research Deep-Dive & Paper Download:** 1-click download of the camera-ready 6-page IEEE conference paper PDF, plus interactive verification of Theorems 1-4 and 50-trial empirical hypothesis testing. | |
| - **Official Repository:** [https://huggingface.co/Premchan369/Q-TensorFormer](https://huggingface.co/Premchan369/Q-TensorFormer) | |
| - **Technical Documentation Portal:** [https://premchan369.github.io/Q-TensorFormer/](https://premchan369.github.io/Q-TensorFormer/) | |
| --- | |
| ## 📖 Beginner & Layman's Guide: How Q-TensorFormer Works (No Math Required) | |
| ### 1. The Big Problem: Why Today's AI Overheats Your Devices | |
| Imagine driving a car where the gas pedal is permanently glued to the floor at 150 mph - even when stopped at a red light. That is how standard LLMs run today: they spend the exact same massive computation and memory on simple filler words (like *"the"*, *"and"*) as on complex reasoning problems. | |
| ### 2. The Solution: The Smart Reader with Adaptive Gears | |
| **Q-TensorFormer** turns the model into an **adaptive reader equipped with automatic transmission, cruise control, and selective memory**. | |
|  | |
| *Figure 1: Complete end-to-end flow of Q-TensorFormer. Tokens enter, are measured by lightweight sensors, routed through compressed tensor layers, and regulated by closed-loop hardware feedback.* | |
| #### 🟢 Step 1: The Sensor Gauge | |
| Before spending compute, a lightweight sensor checks: *Is this word difficult? Is the battery low? Is memory running out?* | |
|  | |
| *Figure 2: Relative importance of the 8 real-time sensor signals.* | |
| #### 🟢 Step 2: Russian Nesting Dolls (Nested Tensor-Train Weights) | |
| Instead of huge flat matrices, weights are stored as compact tensor cores. For easy words, we open only the small inner doll (Rank 1 or 2); for hard words, we open the full doll (Rank 8 or 16). Slicing happens instantly in micro-seconds! | |
|  | |
| *Figure 3: Representation quality across nested Tensor-Train ranks.* | |
|  | |
| *Figure 4: Active rank directly controls latency and DRAM bus traffic.* | |
| #### 🟢 Step 3: Cruise Control with Anti-Pothole Buffer (Hysteresis PID Controller) | |
| A closed-loop PID controller prevents erratic speed spikes. A 0.15 hysteresis deadband ensures the model doesn't flicker between consecutive words, eliminating jitter. | |
|  | |
| *Figure 5: Closed-loop PID controller tracking target latency budgets.* | |
|  | |
| *Figure 6: Rank-switching chatter drops from 25.40% down to 0.00%.* | |
| #### 🟢 Step 4: The Selective Highlighter & Attic Trunk (Semantic Anchor KV Cache) | |
| Instead of keeping every past word in full memory, we keep the first 4 anchor words (attention sinks) and critical facts in crisp full fidelity, and compress the rest to 8-bit/4-bit integers. | |
|  | |
| *Figure 7: Memory vs attention fidelity curve.* | |
|  | |
| *Figure 8: Attention memory traffic reduction across context lengths.* | |
| #### 🟢 Step 5: The In-Kitchen Chef (Fused Hardware Contraction Engine) | |
| Instead of sending numbers back and forth to slow DRAM memory, our fused kernel keeps numbers directly in the processor's fast registers, cutting memory bus traffic by 96.90%! | |
|  | |
| *Figure 9: Arithmetic intensity and memory bandwidth roofline.* | |
| #### 🟢 Step 6: Multi-Dimensional Comparison Radar | |
|  | |
| *Figure 10: Multi-dimensional radar chart comparing Q-TensorFormer against standard dense baselines across parameter size, memory footprint, bus traffic, cold-start time, and energy efficiency.* | |