Instructions to use jpacifico/Aramis-2B-BitNet-bf16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jpacifico/Aramis-2B-BitNet-bf16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jpacifico/Aramis-2B-BitNet-bf16", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("jpacifico/Aramis-2B-BitNet-bf16", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("jpacifico/Aramis-2B-BitNet-bf16", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use jpacifico/Aramis-2B-BitNet-bf16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jpacifico/Aramis-2B-BitNet-bf16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jpacifico/Aramis-2B-BitNet-bf16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/jpacifico/Aramis-2B-BitNet-bf16

SGLang

How to use jpacifico/Aramis-2B-BitNet-bf16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jpacifico/Aramis-2B-BitNet-bf16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jpacifico/Aramis-2B-BitNet-bf16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jpacifico/Aramis-2B-BitNet-bf16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jpacifico/Aramis-2B-BitNet-bf16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use jpacifico/Aramis-2B-BitNet-bf16 with Docker Model Runner:
```
docker model run hf.co/jpacifico/Aramis-2B-BitNet-bf16
```

jpacifico commited on Aug 17, 2025

Commit

5fa3cea

verified ·

1 Parent(s): 4495094

Update README.md

Browse files

Files changed (1) hide show

README.md +7 -0

README.md CHANGED Viewed

@@ -32,6 +32,7 @@ Runs natively as BitNet 1.58-bit (ternary) and is available in GGUF 1.58-bit, lo
 - jpacifico/bitnet-dpo-merged-modelstock7 (this repo): Contains the retrainable weights in BF16 format
 - [jpacifico/bitnet-dpo-fr-i2s-2](https://huggingface.co/jpacifico/bitnet-dpo-fr-i2s-2) : Quantized 1.58-bit GGUF version, you can use with [bitnet.cpp](https://github.com/microsoft/BitNet)
 # Training Recipe
 Base model : [microsoft/bitnet-b1.58-2B-4T-bf16](https://huggingface.co/microsoft/bitnet-b1.58-2B-4T-bf16)
@@ -43,6 +44,7 @@ Iterative DPO + Model merging :
   [Intel/orca_dpo_pairs](https://huggingface.co/datasets/Intel/orca_dpo_pairs)
 - Model merging (ModelStock and TIES methods, via [Mergekit](https://github.com/cg123/mergekit) to combine complementary strengths of bilingual models (FR-centric + EN-centric), improving robustness across reasoning and comprehension tasks while maintaining stability.
 # First benchmarks
 **Interpretation:** Significant gains on language understanding & pragmatic reasoning (ARC-C/E, Wino, BoolQ, HellaSwag, TriviaQA) with stability on other skills. Math/code are not the optimization target; GSM8K stays essentially stable relative to the BitNet 1.58-bit quantized baseline (58,38).
@@ -99,6 +101,8 @@ HF_ALLOW_CODE_EVAL=1 lm-eval --model hf \
 - Randomness (e.g. seeds, batch sizes) may cause slight variations in results
 - The same procedure was used to evaluate all tasks presented in the benchmark tables
 # Usage with `bitnet.cpp`
 You can run this model using my demo [Colab notebook](https://github.com/jpacifico/) TBD
@@ -106,6 +110,7 @@ You can run this model using my demo [Colab notebook](https://github.com/jpacifi
 Please refer to the [bitnet.cpp](https://github.com/microsoft/BitNet) GitHub repository for detailed compilation steps, usage examples, and command-line options.
 # Last checkpoint
 ### Merge Method
@@ -138,6 +143,7 @@ tokenizer_source: jpacifico/bitnet-dpo-merged-modelstock-retrain
 ```
 # Limitations
 Not tuned for coding or formal math; prefer specialized variants if those are critical.
@@ -147,6 +153,7 @@ No explicit chain-of-thought training; improvements come from bilingual DPO + me
 This model is intended for research and development purposes only and should not be used in commercial or real-world applications without further testing. While the Microsoft Research team has applied SFT and DPO to align the BitNet base model, it may still produce unexpected, biased, or inaccurate outputs. Please use responsibly.
 - **Developed by:** Jonathan Pacifico, 2025
 - **Model type:** LLM
 - **Language(s) (NLP):** French, English

 - jpacifico/bitnet-dpo-merged-modelstock7 (this repo): Contains the retrainable weights in BF16 format
 - [jpacifico/bitnet-dpo-fr-i2s-2](https://huggingface.co/jpacifico/bitnet-dpo-fr-i2s-2) : Quantized 1.58-bit GGUF version, you can use with [bitnet.cpp](https://github.com/microsoft/BitNet)
 # Training Recipe
 Base model : [microsoft/bitnet-b1.58-2B-4T-bf16](https://huggingface.co/microsoft/bitnet-b1.58-2B-4T-bf16)
   [Intel/orca_dpo_pairs](https://huggingface.co/datasets/Intel/orca_dpo_pairs)
 - Model merging (ModelStock and TIES methods, via [Mergekit](https://github.com/cg123/mergekit) to combine complementary strengths of bilingual models (FR-centric + EN-centric), improving robustness across reasoning and comprehension tasks while maintaining stability.
 # First benchmarks
 **Interpretation:** Significant gains on language understanding & pragmatic reasoning (ARC-C/E, Wino, BoolQ, HellaSwag, TriviaQA) with stability on other skills. Math/code are not the optimization target; GSM8K stays essentially stable relative to the BitNet 1.58-bit quantized baseline (58,38).
 - Randomness (e.g. seeds, batch sizes) may cause slight variations in results
 - The same procedure was used to evaluate all tasks presented in the benchmark tables
 # Usage with `bitnet.cpp`
 You can run this model using my demo [Colab notebook](https://github.com/jpacifico/) TBD
 Please refer to the [bitnet.cpp](https://github.com/microsoft/BitNet) GitHub repository for detailed compilation steps, usage examples, and command-line options.
 # Last checkpoint
 ### Merge Method
 ```
 # Limitations
 Not tuned for coding or formal math; prefer specialized variants if those are critical.
 This model is intended for research and development purposes only and should not be used in commercial or real-world applications without further testing. While the Microsoft Research team has applied SFT and DPO to align the BitNet base model, it may still produce unexpected, biased, or inaccurate outputs. Please use responsibly.
 - **Developed by:** Jonathan Pacifico, 2025
 - **Model type:** LLM
 - **Language(s) (NLP):** French, English