Documentation & API Reference

1. Installation

Clone the repository and install dependencies:

git clone https://huggingface.co/Premchan369/Q-TensorFormer
cd Q-TensorFormer
pip install -r requirements.txt

Dependencies: torch >= 2.0.0, transformers >= 4.35.0, numpy >= 1.24.0, pytest >= 7.0.0.

2. PyTorch Research API (src/)

Load the verified canonical checkpoint and perform inference:

import torch
from src.checkpoint import load_canonical_checkpoint
from src.tokenizer import QTensorFormerTokenizer

# 1. Load canonical tokenizer
tokenizer = QTensorFormerTokenizer.load("checkpoints/tokenizer/tokenizer.json")

# 2. Load canonical checkpoint with strict verification
model, info = load_canonical_checkpoint(
    "checkpoints/qtensorformer_canonical_trained.pt",
    strict=True,
    require_trained=True,
    tokenizer=tokenizer
)

# 3. Tokenize input text
prompt = "Transformer attention"
input_ids = torch.tensor([tokenizer.encode(prompt, add_bos=True)])

# 4. Generate with KV cache and deterministic greedy decoding
output_ids = model.generate(
    input_ids,
    max_new_tokens=15,
    do_sample=False,
    use_cache=True,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id
)

print("Decoded Output:", tokenizer.decode(output_ids[0]))

3. Hugging Face Transformers API

Use Q-TensorFormer with the standard Hugging Face pipeline:

from transformers import AutoConfig, AutoModelForCausalLM

# Load model from the canonical checkpoint directory
model_path = "checkpoints/hf_canonical_trained"
config = AutoConfig.from_pretrained(model_path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True)

# Generate using Hugging Face .generate()
import torch
prompt_ids = torch.tensor([[1, 25, 42, 10]])
generated = model.generate(prompt_ids, max_new_tokens=10, do_sample=False)
print("Generated Token IDs:", generated)

4. Deployment & Export Tools

python qtensorformer_to_gguf.py \
  --checkpoint checkpoints/qtensorformer_canonical_trained.pt \
  --output qtensorformer.gguf \
  --strict

python export_onnx.py \
  --output qtensorformer.onnx \
  --batch_size 1 \
  --seq_len 32

5. Running Unit Tests & Smoke Tests

Verify that your environment matches our benchmark findings:

# Run full pytest suite (94 passing tests)
pytest -v

# Run deterministic smoke test
python scripts/smoke_test.py