Text Generation
Transformers
English
qtensorformer
tensor-networks
model-compression
adaptive-computation
kv-cache-compression
hardware-aware
energy-aware
quantum-machine-learning
green-ai
Instructions to use Premchan369/Q-TensorFormer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Premchan369/Q-TensorFormer with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Premchan369/Q-TensorFormer")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Premchan369/Q-TensorFormer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Premchan369/Q-TensorFormer with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Premchan369/Q-TensorFormer" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Premchan369/Q-TensorFormer", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Premchan369/Q-TensorFormer
- SGLang
How to use Premchan369/Q-TensorFormer with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Premchan369/Q-TensorFormer" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Premchan369/Q-TensorFormer", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Premchan369/Q-TensorFormer" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Premchan369/Q-TensorFormer", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Premchan369/Q-TensorFormer with Docker Model Runner:
docker model run hf.co/Premchan369/Q-TensorFormer
File size: 2,278 Bytes
eaeea8f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | """
Tests for Information-Value Resource Allocator.
"""
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import torch
import pytest
from src.resource_allocator import InformationValueAllocator, AllocationBudget
def test_resource_allocator_decisions():
allocator = InformationValueAllocator(info_dim=8, hidden_dim=16)
B, T = 2, 8
z_t = torch.rand(B, T, 8)
decisions, diagnostics = allocator(z_t, preset="balanced")
assert decisions["rank"] in [1, 2, 4, 8]
assert decisions["attn_mode_idx"].shape == (B, T)
assert decisions["depth_mode"] in ["skip", "partial", "full"]
assert decisions["kv_precision"] in ["fp16", "int8", "int4"]
assert "chosen_rank" in diagnostics
assert "routing_churn_rate" in diagnostics
print("✓ test_resource_allocator_decisions passed")
def test_resource_allocator_hysteresis():
allocator = InformationValueAllocator(info_dim=8, hidden_dim=16, hysteresis_tau=0.5)
B, T = 1, 4
z_t_1 = torch.full((B, T, 8), 0.2)
decisions_1, _ = allocator(z_t_1)
rank_1 = decisions_1["rank"]
# Small perturbation that should NOT break hysteresis threshold
z_t_2 = torch.full((B, T, 8), 0.22)
decisions_2, _ = allocator(z_t_2)
rank_2 = decisions_2["rank"]
assert rank_1 == rank_2, f"Hysteresis should preserve rank on small delta: {rank_1} vs {rank_2}"
print("✓ test_resource_allocator_hysteresis passed")
def test_resource_allocator_presets():
allocator = InformationValueAllocator(info_dim=8)
z_t = torch.rand(2, 4, 8)
# Edge preset should force classical
decisions_edge, diag_edge = allocator(z_t, preset="edge")
assert not decisions_edge["is_quantum_token"].any(), "Edge preset should disable quantum tokens"
# Classical-only preset should force classical
decisions_class, _ = allocator(z_t, preset="classical_only")
assert not decisions_class["is_quantum_token"].any(), "Classical-only preset should disable quantum tokens"
print("✓ test_resource_allocator_presets passed")
if __name__ == "__main__":
test_resource_allocator_decisions()
test_resource_allocator_hysteresis()
test_resource_allocator_presets()
print("All Resource Allocator tests passed!")
|