Text Generation
Transformers
Safetensors
PyTorch
English
qtensorformer
q-tensorformer
high-throughput-inference
tensor-train
tensor-networks
tensorrt-llm
vllm
continuous-batching
paged-attention
speculative-decoding
zero-svd
adaptive-computation
dynamic-neural-networks
kv-cache
kv-cache-quantization
attention-sinks
lyapunov-stability
control-theory
contextual-bandits
edge-ai
edge-inference
on-device-ai
model-compression
hopper-h100
blackwell-b200
jetson-orin
jetson-thor
grouped-query-attention
rotary-position-embeddings
custom_code
academic-paper
ieee
neurips
vit-ap-university
File size: 4,644 Bytes
7a61b0c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 | {
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# ⚛️ Q-TensorFormer: 1-Click Interactive Google Colab Quickstart\n",
"\n",
"[](https://colab.research.google.com/github/Premchan369/Q-TensorFormer/blob/main/notebooks/quickstart.ipynb)\n",
"[](https://huggingface.co/Premchan369/Q-TensorFormer)\n",
"\n",
"Welcome to **Q-TensorFormer**, an arXiv-grade, hardware-aware Transformer architecture that dynamically allocates model capacity, tensor rank ($r \\in [4, 16]$), and KV cache precision on a per-token basis using a closed-loop PID controller.\n",
"\n",
"In this notebook, you will:\n",
"1. Load Q-TensorFormer directly with `trust_remote_code=True`.\n",
"2. Observe token-by-token dynamic rank shifting and live energy telemetry in action.\n",
"3. Learn how to compress any dense Hugging Face model (LLaMA, Qwen) into a low-rank Q-TensorFormer model."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 1: Install Dependencies"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install -q \"transformers>=4.38.0\" \"torch>=2.0.0\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 2: Load Q-TensorFormer Directly via Hugging Face AutoModel"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer\n",
"\n",
"device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
"print(f\"Using device: {device}\")\n",
"\n",
"# Load turnkey model from Hugging Face Hub\n",
"model_id = \"Premchan369/Q-TensorFormer\"\n",
"config = AutoConfig.from_pretrained(model_id, trust_remote_code=True)\n",
"model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True).to(device)\n",
"model.eval()\n",
"\n",
"print(f\"✓ Successfully loaded Q-TensorFormer ({sum(p.numel() for p in model.parameters()):,} parameters)!\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 3: Autoregressive Text Generation with Live Dynamic Telemetry"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Example prompt\n",
"prompt = \"Quantum tensor network architectures optimize energy by\"\n",
"print(f\"Input Prompt: {prompt}\\n\")\n",
"\n",
"# Tokenize prompt\n",
"tokens = [ord(c) % config.vocab_size for c in prompt]\n",
"input_ids = torch.tensor([tokens], dtype=torch.long, device=device)\n",
"\n",
"# Generate with live closed-loop PID rank tracking\n",
"curr_ids = input_ids\n",
"past_key_values = None\n",
"\n",
"print(\"--- Streaming Generated Output with Live Tensor-Train Rank ---\")\n",
"for step in range(32):\n",
" with torch.no_grad():\n",
" outputs = model(curr_ids[:, -1:], past_key_values=past_key_values, use_cache=True)\n",
" logits = outputs.logits[:, -1, :]\n",
" past_key_values = outputs.past_key_values\n",
" next_tok = logits.argmax(dim=-1, keepdim=True)\n",
" \n",
" curr_ids = torch.cat([curr_ids, next_tok], dim=1)\n",
" tok_id = int(next_tok[0, 0].item())\n",
" char = chr(tok_id) if 32 <= tok_id < 127 else \" \"\n",
" \n",
" # Query live PID controller rank\n",
" pid_ctrl = model.model.pid_controller\n",
" active_rank = int(pid_ctrl.rank_levels[int(pid_ctrl.active_rank_idx.item())])\n",
" \n",
" print(f\"{char}\", end=\"\", flush=True)\n",
"\n",
"print(\"\\n\\n✓ Generation Complete!\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 4: Drop-In LLM Compressor\n",
"\n",
"You can also use Q-TensorFormer to compress existing popular dense models (like LLaMA-3.2-1B, Qwen-2.5, or Mistral) down by 50%:\n",
"\n",
"```bash\n",
"!python qtensorformer_compress.py \\\n",
" --model-name-or-path meta-llama/Llama-3.2-1B \\\n",
" --output-dir ./Llama-3.2-1B-QTensor \\\n",
" --tt-rank 16\n",
"```"
]
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
|