Text Generation
Transformers
PyTorch
English
experimental
research
bit-level
transformer
reversible
safety
telemetry
language-modeling
Instructions to use WCNegentropy/BitTransformerLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use WCNegentropy/BitTransformerLM with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="WCNegentropy/BitTransformerLM")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("WCNegentropy/BitTransformerLM", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use WCNegentropy/BitTransformerLM with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "WCNegentropy/BitTransformerLM" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "WCNegentropy/BitTransformerLM", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/WCNegentropy/BitTransformerLM
- SGLang
How to use WCNegentropy/BitTransformerLM with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "WCNegentropy/BitTransformerLM" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "WCNegentropy/BitTransformerLM", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "WCNegentropy/BitTransformerLM" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "WCNegentropy/BitTransformerLM", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use WCNegentropy/BitTransformerLM with Docker Model Runner:
docker model run hf.co/WCNegentropy/BitTransformerLM
File size: 1,992 Bytes
36c78b1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 | import matplotlib.pyplot as plt
from typing import Dict, List, Tuple
def plot_telemetry(
metrics_log: Dict[str, List[float]],
k_floor: float = 0.5,
c_floor: float = 0.3,
s_floor: float = 0.5,
) -> Tuple[plt.Figure, List[plt.Axes]]:
"""Plot K, C, S metrics over time with cluster transitions.
Args:
metrics_log: Dictionary with keys ``negentropy``, ``lz_complexity``,
``symbiosis_score`` and optional ``clusters`` listing cluster
assignments per step.
k_floor: Threshold for negentropy (K).
c_floor: Threshold for LZ complexity (C).
s_floor: Threshold for symbiosis score (S).
Returns:
(figure, axes) tuple for further customization or saving.
"""
steps = list(range(len(metrics_log.get("negentropy", []))))
fig, axes = plt.subplots(3, 1, sharex=True, figsize=(10, 6))
metrics = [
("negentropy", k_floor, "K"),
("lz_complexity", c_floor, "C"),
("symbiosis_score", s_floor, "S"),
]
for ax, (key, floor, label) in zip(axes, metrics):
values = metrics_log.get(key, [])
ax.plot(steps, values, label=label)
ax.axhline(floor, color="r", linestyle="--", linewidth=1)
violations = [i for i, v in enumerate(values) if v < floor]
if violations:
ax.scatter(
[steps[i] for i in violations],
[values[i] for i in violations],
color="r",
zorder=5,
label="violation",
)
ax.set_ylabel(label)
ax.legend(loc="upper right")
clusters = metrics_log.get("clusters")
if clusters is not None:
prev = clusters[0]
for t, c in enumerate(clusters):
if t > 0 and c != prev:
for ax in axes:
ax.axvline(t, color="gray", linestyle=":", alpha=0.5)
prev = c
axes[-1].set_xlabel("step")
plt.tight_layout()
return fig, axes
|