Text Generation
Transformers
PyTorch
English
experimental
research
bit-level
transformer
reversible
safety
telemetry
language-modeling
Instructions to use WCNegentropy/BitTransformerLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use WCNegentropy/BitTransformerLM with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="WCNegentropy/BitTransformerLM")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("WCNegentropy/BitTransformerLM", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use WCNegentropy/BitTransformerLM with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "WCNegentropy/BitTransformerLM" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "WCNegentropy/BitTransformerLM", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/WCNegentropy/BitTransformerLM
- SGLang
How to use WCNegentropy/BitTransformerLM with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "WCNegentropy/BitTransformerLM" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "WCNegentropy/BitTransformerLM", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "WCNegentropy/BitTransformerLM" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "WCNegentropy/BitTransformerLM", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use WCNegentropy/BitTransformerLM with Docker Model Runner:
docker model run hf.co/WCNegentropy/BitTransformerLM
| import matplotlib.pyplot as plt | |
| from typing import Dict, List, Tuple | |
| def plot_telemetry( | |
| metrics_log: Dict[str, List[float]], | |
| k_floor: float = 0.5, | |
| c_floor: float = 0.3, | |
| s_floor: float = 0.5, | |
| ) -> Tuple[plt.Figure, List[plt.Axes]]: | |
| """Plot K, C, S metrics over time with cluster transitions. | |
| Args: | |
| metrics_log: Dictionary with keys ``negentropy``, ``lz_complexity``, | |
| ``symbiosis_score`` and optional ``clusters`` listing cluster | |
| assignments per step. | |
| k_floor: Threshold for negentropy (K). | |
| c_floor: Threshold for LZ complexity (C). | |
| s_floor: Threshold for symbiosis score (S). | |
| Returns: | |
| (figure, axes) tuple for further customization or saving. | |
| """ | |
| steps = list(range(len(metrics_log.get("negentropy", [])))) | |
| fig, axes = plt.subplots(3, 1, sharex=True, figsize=(10, 6)) | |
| metrics = [ | |
| ("negentropy", k_floor, "K"), | |
| ("lz_complexity", c_floor, "C"), | |
| ("symbiosis_score", s_floor, "S"), | |
| ] | |
| for ax, (key, floor, label) in zip(axes, metrics): | |
| values = metrics_log.get(key, []) | |
| ax.plot(steps, values, label=label) | |
| ax.axhline(floor, color="r", linestyle="--", linewidth=1) | |
| violations = [i for i, v in enumerate(values) if v < floor] | |
| if violations: | |
| ax.scatter( | |
| [steps[i] for i in violations], | |
| [values[i] for i in violations], | |
| color="r", | |
| zorder=5, | |
| label="violation", | |
| ) | |
| ax.set_ylabel(label) | |
| ax.legend(loc="upper right") | |
| clusters = metrics_log.get("clusters") | |
| if clusters is not None: | |
| prev = clusters[0] | |
| for t, c in enumerate(clusters): | |
| if t > 0 and c != prev: | |
| for ax in axes: | |
| ax.axvline(t, color="gray", linestyle=":", alpha=0.5) | |
| prev = c | |
| axes[-1].set_xlabel("step") | |
| plt.tight_layout() | |
| return fig, axes | |