Instructions to use Premchan369/Q-TensorFormer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Premchan369/Q-TensorFormer with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Premchan369/Q-TensorFormer", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Premchan369/Q-TensorFormer", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Premchan369/Q-TensorFormer with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Premchan369/Q-TensorFormer"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Premchan369/Q-TensorFormer",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Premchan369/Q-TensorFormer

SGLang

How to use Premchan369/Q-TensorFormer with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Premchan369/Q-TensorFormer" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Premchan369/Q-TensorFormer",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Premchan369/Q-TensorFormer" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Premchan369/Q-TensorFormer",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Premchan369/Q-TensorFormer with Docker Model Runner:
```
docker model run hf.co/Premchan369/Q-TensorFormer
```

Q-TensorFormer / SPACE_README.md

Premchandyadav369

feat: SEO and discoverability expansion, viral launch kit, FAQ schema, and trending tags

7d566f9 9 days ago

preview code

Raw History Blame Contribute Delete

9.07 kB

---
title: Q-TensorFormer
emoji: ⚛️
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 6.30.0
python_version: '3.12'
app_file: app.py
pinned: false
license: apache-2.0
short_description: Real-Time High-Throughput Edge LLM Inference
models:
- Premchan369/Q-TensorFormer
tags:
- edge-ai
- reasoning
- deepseek-style
- openai-api
- tensor-train
- hardware-adaptive
- green-ai
- high-throughput
- vit-ap-university
---
# ⚛️ Q-TensorFormer: Real-Time High-Throughput Edge LLM Inference

[![Download Camera-Ready Paper PDF](https://img.shields.io/badge/%F0%9F%93%84%20Download%20Paper-IEEE%206--Page%20PDF-red?style=for-the-badge)](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/paper/q_tensorformer_ieee_paper.pdf)
[![Documentation](https://img.shields.io/badge/Docs-premchan369.github.io%2FQ--TensorFormer-blue?style=for-the-badge)](https://premchan369.github.io/Q-TensorFormer/)
[![Model Hub](https://img.shields.io/badge/%F0%9F%A4%97%20Model%20Hub-Premchan369%2FQ--TensorFormer-yellow?style=for-the-badge)](https://huggingface.co/Premchan369/Q-TensorFormer)
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Premchan369/Q-TensorFormer/blob/main/notebooks/quickstart.ipynb)
[![License: Apache 2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
[![Tests: 117 Passed](https://img.shields.io/badge/Pytest-117%20Passed%20%7C%200%20Failed-brightgreen)](https://huggingface.co/Premchan369/Q-TensorFormer)

Welcome to the live interactive research showcase of **Q-TensorFormer**, a real-time high-throughput edge LLM inference architecture developed by **Dr. Madala Guru Brahmam** (Assistant Professor) and **V C Premchand Yadav** (Final Year B.Tech CSE AIML) at **VIT-AP University**, Amaravati, Andhra Pradesh, India.

> 📄 **Instant Paper Download:** [Download Camera-Ready 6-Page IEEE Conference Paper (PDF)](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/paper/q_tensorformer_ieee_paper.pdf)  
> 📦 **Pretrained Weights & Code:** [Premchan369/Q-TensorFormer on Hugging Face Hub](https://huggingface.co/Premchan369/Q-TensorFormer)

### What This Space Demonstrates:
1. **📖 Beginner & Layman Interactive Guide:** Test drive the automatic transmission AI car across driving conditions and explore the 5 core inventions with zero math.
2. **💬 Live Dynamic Tensor-Train Slicing:** Type any prompt to watch the model slice Tensor-Train rank cores ($r \in [4, 16]$) token-by-token based on marginal informational value and target energy budgets with live lookahead horizon predictions.
3. **🔮 Edge Digital Twin & Predictive Future Forecaster:** Simulate real-time physical dynamics into the future (5 to 60 minutes) across 5 hardware platforms (Raspberry Pi 5, Jetson Orin Nano, Apple M3/M4, Drone MCU, Intel Core Ultra 7) forecasting junction temperature ($T_j$), thermal throttling cliffs, battery depletion, and zero-chattering rank evolution.
4. **🌐 Autonomous Swarm & Global Fleet ROI Calculator:** Predict fleet survival across 10-100 edge nodes, and compute annual MWh saved, tons of CO2 avoided, and cloud hosting cost reductions.
5. **🎛️ Real-Time Dynamic SLA Controller Sandbox:** Test closed-loop PID disturbance rejection with live phase portraits and zero limit-cycle chattering ($\Delta V_t \le 0$).
6. **⚡ Zero-SVD Speculative Self-Drafting Engine:** Single-model self-speculative decoding simulator ($r_{\text{draft}}=1 \to r_{\text{target}}=4$) delivering $1.42\times$ acceleration with 0.00 MB extra DRAM allocation.
7. **🔬 Core Slicing & Singular Spectrum Inspector:** Direct mathematical inspection of TT core shapes ($\mathcal{G}^{(1)}, \mathcal{G}^{(2)}, \mathcal{G}^{(3)}$) and interactive singular value energy waterfall chart ($\sum_{i=1}^r \sigma_i^2 / \sum \sigma_i^2$), validating Theorem 1.
8. **👁️ Multimodal Vision Projection:** Spatial-variance-guided visual patch ranking (Rank 16 for foreground, Rank 2 for background, 65.2% FLOP reduction).
9. **⚡ LLM Compression Studio:** Interactive parameter savings calculator for open-weight models (Llama-3, Qwen, Mistral).
10. **🎓 NeurIPS & IEEE Research Deep-Dive & Paper Download:** 1-click download of the camera-ready 6-page IEEE conference paper PDF, plus interactive verification of Theorems 1-4 and 50-trial empirical hypothesis testing.
- **Official Repository:** [https://huggingface.co/Premchan369/Q-TensorFormer](https://huggingface.co/Premchan369/Q-TensorFormer)
- **Technical Documentation Portal:** [https://premchan369.github.io/Q-TensorFormer/](https://premchan369.github.io/Q-TensorFormer/)
---
## 📖 Beginner & Layman's Guide: How Q-TensorFormer Works (No Math Required)
### 1. The Big Problem: Why Today's AI Overheats Your Devices
Imagine driving a car where the gas pedal is permanently glued to the floor at 150 mph - even when stopped at a red light. That is how standard LLMs run today: they spend the exact same massive computation and memory on simple filler words (like *"the"*, *"and"*) as on complex reasoning problems.
### 2. The Solution: The Smart Reader with Adaptive Gears
**Q-TensorFormer** turns the model into an **adaptive reader equipped with automatic transmission, cruise control, and selective memory**.
![Q-TensorFormer Architecture Overview](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure1_architecture.png)
*Figure 1: Complete end-to-end flow of Q-TensorFormer. Tokens enter, are measured by lightweight sensors, routed through compressed tensor layers, and regulated by closed-loop hardware feedback.*
#### 🟢 Step 1: The Sensor Gauge
Before spending compute, a lightweight sensor checks: *Is this word difficult? Is the battery low? Is memory running out?*
![8D Information State Feature Importance](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure3_8d_lofo_importance.png)
*Figure 2: Relative importance of the 8 real-time sensor signals.*
#### 🟢 Step 2: Russian Nesting Dolls (Nested Tensor-Train Weights)
Instead of huge flat matrices, weights are stored as compact tensor cores. For easy words, we open only the small inner doll (Rank 1 or 2); for hard words, we open the full doll (Rank 8 or 16). Slicing happens instantly in micro-seconds!
![Nested Tensor-Train Slicing Suboptimality](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure4_nested_tt_suboptimality.png)
*Figure 3: Representation quality across nested Tensor-Train ranks.*
![Adaptive Rank vs Latency & Memory Traffic](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure5_adaptive_rank_latency_traffic.png)
*Figure 4: Active rank directly controls latency and DRAM bus traffic.*
#### 🟢 Step 3: Cruise Control with Anti-Pothole Buffer (Hysteresis PID Controller)
A closed-loop PID controller prevents erratic speed spikes. A 0.15 hysteresis deadband ensures the model doesn't flicker between consecutive words, eliminating jitter.
![PID Controller Convergence](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure8_controller_convergence.png)
*Figure 5: Closed-loop PID controller tracking target latency budgets.*
![Hysteresis Anti-Chattering Stability](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure9_hysteresis_churn_jitter.png)
*Figure 6: Rank-switching chatter drops from 25.40% down to 0.00%.*
#### 🟢 Step 4: The Selective Highlighter & Attic Trunk (Semantic Anchor KV Cache)
Instead of keeping every past word in full memory, we keep the first 4 anchor words (attention sinks) and critical facts in crisp full fidelity, and compress the rest to 8-bit/4-bit integers.
![KV Cache Memory vs Quality Rate Distortion](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure7_kv_rate_distortion.png)
*Figure 7: Memory vs attention fidelity curve.*
![GQA & Attention Memory Traffic](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure6_gqa_memory_traffic.png)
*Figure 8: Attention memory traffic reduction across context lengths.*
#### 🟢 Step 5: The In-Kitchen Chef (Fused Hardware Contraction Engine)
Instead of sending numbers back and forth to slow DRAM memory, our fused kernel keeps numbers directly in the processor's fast registers, cutting memory bus traffic by 96.90%!
![Hardware Roofline Intensity Analysis](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure12_hardware_roofline.png)
*Figure 9: Arithmetic intensity and memory bandwidth roofline.*
#### 🟢 Step 6: Multi-Dimensional Comparison Radar
![Multi-Dimensional Baseline Improvement Radar](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure16_baseline_improvement_radar.png)
*Figure 10: Multi-dimensional radar chart comparing Q-TensorFormer against standard dense baselines across parameter size, memory footprint, bus traffic, cold-start time, and energy efficiency.*

	---
	title: Q-TensorFormer
	emoji: ⚛️
	colorFrom: indigo
	colorTo: purple
	sdk: gradio
	sdk_version: 6.30.0
	python_version: '3.12'
	app_file: app.py
	pinned: false
	license: apache-2.0
	short_description: Real-Time High-Throughput Edge LLM Inference
	models:
	- Premchan369/Q-TensorFormer
	tags:
	- edge-ai
	- reasoning
	- deepseek-style
	- openai-api
	- tensor-train
	- hardware-adaptive
	- green-ai
	- high-throughput
	- vit-ap-university
	---

	# ⚛️ Q-TensorFormer: Real-Time High-Throughput Edge LLM Inference

	[![Download Camera-Ready Paper PDF](https://img.shields.io/badge/%F0%9F%93%84%20Download%20Paper-IEEE%206--Page%20PDF-red?style=for-the-badge)](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/paper/q_tensorformer_ieee_paper.pdf)
	[![Documentation](https://img.shields.io/badge/Docs-premchan369.github.io%2FQ--TensorFormer-blue?style=for-the-badge)](https://premchan369.github.io/Q-TensorFormer/)
	[![Model Hub](https://img.shields.io/badge/%F0%9F%A4%97%20Model%20Hub-Premchan369%2FQ--TensorFormer-yellow?style=for-the-badge)](https://huggingface.co/Premchan369/Q-TensorFormer)
	[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Premchan369/Q-TensorFormer/blob/main/notebooks/quickstart.ipynb)
	[![License: Apache 2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
	[![Tests: 117 Passed](https://img.shields.io/badge/Pytest-117%20Passed%20%7C%200%20Failed-brightgreen)](https://huggingface.co/Premchan369/Q-TensorFormer)

	Welcome to the live interactive research showcase of Q-TensorFormer, a real-time high-throughput edge LLM inference architecture developed by Dr. Madala Guru Brahmam (Assistant Professor) and V C Premchand Yadav (Final Year B.Tech CSE AIML) at VIT-AP University, Amaravati, Andhra Pradesh, India.

	> 📄 Instant Paper Download: [Download Camera-Ready 6-Page IEEE Conference Paper (PDF)](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/paper/q_tensorformer_ieee_paper.pdf)
	> 📦 Pretrained Weights & Code: [Premchan369/Q-TensorFormer on Hugging Face Hub](https://huggingface.co/Premchan369/Q-TensorFormer)

	### What This Space Demonstrates:
	1. 📖 Beginner & Layman Interactive Guide: Test drive the automatic transmission AI car across driving conditions and explore the 5 core inventions with zero math.
	2. 💬 Live Dynamic Tensor-Train Slicing: Type any prompt to watch the model slice Tensor-Train rank cores ($r \in [4, 16]$) token-by-token based on marginal informational value and target energy budgets with live lookahead horizon predictions.
	3. 🔮 Edge Digital Twin & Predictive Future Forecaster: Simulate real-time physical dynamics into the future (5 to 60 minutes) across 5 hardware platforms (Raspberry Pi 5, Jetson Orin Nano, Apple M3/M4, Drone MCU, Intel Core Ultra 7) forecasting junction temperature ($T_j$), thermal throttling cliffs, battery depletion, and zero-chattering rank evolution.
	4. 🌐 Autonomous Swarm & Global Fleet ROI Calculator: Predict fleet survival across 10-100 edge nodes, and compute annual MWh saved, tons of CO2 avoided, and cloud hosting cost reductions.
	5. 🎛️ Real-Time Dynamic SLA Controller Sandbox: Test closed-loop PID disturbance rejection with live phase portraits and zero limit-cycle chattering ($\Delta V_t \le 0$).
	6. ⚡ Zero-SVD Speculative Self-Drafting Engine: Single-model self-speculative decoding simulator ($r_{\text{draft}}=1 \to r_{\text{target}}=4$) delivering $1.42\times$ acceleration with 0.00 MB extra DRAM allocation.
	7. 🔬 Core Slicing & Singular Spectrum Inspector: Direct mathematical inspection of TT core shapes ($\mathcal{G}^{(1)}, \mathcal{G}^{(2)}, \mathcal{G}^{(3)}$) and interactive singular value energy waterfall chart ($\sum_{i=1}^r \sigma_i^2 / \sum \sigma_i^2$), validating Theorem 1.
	8. 👁️ Multimodal Vision Projection: Spatial-variance-guided visual patch ranking (Rank 16 for foreground, Rank 2 for background, 65.2% FLOP reduction).
	9. ⚡ LLM Compression Studio: Interactive parameter savings calculator for open-weight models (Llama-3, Qwen, Mistral).
	10. 🎓 NeurIPS & IEEE Research Deep-Dive & Paper Download: 1-click download of the camera-ready 6-page IEEE conference paper PDF, plus interactive verification of Theorems 1-4 and 50-trial empirical hypothesis testing.

	- Official Repository: [https://huggingface.co/Premchan369/Q-TensorFormer](https://huggingface.co/Premchan369/Q-TensorFormer)
	- Technical Documentation Portal: [https://premchan369.github.io/Q-TensorFormer/](https://premchan369.github.io/Q-TensorFormer/)


	---

	## 📖 Beginner & Layman's Guide: How Q-TensorFormer Works (No Math Required)

	### 1. The Big Problem: Why Today's AI Overheats Your Devices
	Imagine driving a car where the gas pedal is permanently glued to the floor at 150 mph - even when stopped at a red light. That is how standard LLMs run today: they spend the exact same massive computation and memory on simple filler words (like "the", "and") as on complex reasoning problems.

	### 2. The Solution: The Smart Reader with Adaptive Gears
	Q-TensorFormer turns the model into an adaptive reader equipped with automatic transmission, cruise control, and selective memory.

	![Q-TensorFormer Architecture Overview](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure1_architecture.png)
	Figure 1: Complete end-to-end flow of Q-TensorFormer. Tokens enter, are measured by lightweight sensors, routed through compressed tensor layers, and regulated by closed-loop hardware feedback.

	#### 🟢 Step 1: The Sensor Gauge
	Before spending compute, a lightweight sensor checks: Is this word difficult? Is the battery low? Is memory running out?
	![8D Information State Feature Importance](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure3_8d_lofo_importance.png)
	Figure 2: Relative importance of the 8 real-time sensor signals.

	#### 🟢 Step 2: Russian Nesting Dolls (Nested Tensor-Train Weights)
	Instead of huge flat matrices, weights are stored as compact tensor cores. For easy words, we open only the small inner doll (Rank 1 or 2); for hard words, we open the full doll (Rank 8 or 16). Slicing happens instantly in micro-seconds!
	![Nested Tensor-Train Slicing Suboptimality](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure4_nested_tt_suboptimality.png)
	Figure 3: Representation quality across nested Tensor-Train ranks.
	![Adaptive Rank vs Latency & Memory Traffic](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure5_adaptive_rank_latency_traffic.png)
	Figure 4: Active rank directly controls latency and DRAM bus traffic.

	#### 🟢 Step 3: Cruise Control with Anti-Pothole Buffer (Hysteresis PID Controller)
	A closed-loop PID controller prevents erratic speed spikes. A 0.15 hysteresis deadband ensures the model doesn't flicker between consecutive words, eliminating jitter.
	![PID Controller Convergence](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure8_controller_convergence.png)
	Figure 5: Closed-loop PID controller tracking target latency budgets.
	![Hysteresis Anti-Chattering Stability](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure9_hysteresis_churn_jitter.png)
	Figure 6: Rank-switching chatter drops from 25.40% down to 0.00%.

	#### 🟢 Step 4: The Selective Highlighter & Attic Trunk (Semantic Anchor KV Cache)
	Instead of keeping every past word in full memory, we keep the first 4 anchor words (attention sinks) and critical facts in crisp full fidelity, and compress the rest to 8-bit/4-bit integers.
	![KV Cache Memory vs Quality Rate Distortion](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure7_kv_rate_distortion.png)
	Figure 7: Memory vs attention fidelity curve.
	![GQA & Attention Memory Traffic](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure6_gqa_memory_traffic.png)
	Figure 8: Attention memory traffic reduction across context lengths.

	#### 🟢 Step 5: The In-Kitchen Chef (Fused Hardware Contraction Engine)
	Instead of sending numbers back and forth to slow DRAM memory, our fused kernel keeps numbers directly in the processor's fast registers, cutting memory bus traffic by 96.90%!
	![Hardware Roofline Intensity Analysis](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure12_hardware_roofline.png)
	Figure 9: Arithmetic intensity and memory bandwidth roofline.

	#### 🟢 Step 6: Multi-Dimensional Comparison Radar
	![Multi-Dimensional Baseline Improvement Radar](https://huggingface.co/Premchan369/Q-TensorFormer/resolve/main/outputs/figures/figure16_baseline_improvement_radar.png)
	Figure 10: Multi-dimensional radar chart comparing Q-TensorFormer against standard dense baselines across parameter size, memory footprint, bus traffic, cold-start time, and energy efficiency.