Instructions to use ControlLLM/Control-LLM-Llama3.1-8B-Math16-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ControlLLM/Control-LLM-Llama3.1-8B-Math16-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ControlLLM/Control-LLM-Llama3.1-8B-Math16-Instruct")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("ControlLLM/Control-LLM-Llama3.1-8B-Math16-Instruct", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ControlLLM/Control-LLM-Llama3.1-8B-Math16-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ControlLLM/Control-LLM-Llama3.1-8B-Math16-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ControlLLM/Control-LLM-Llama3.1-8B-Math16-Instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ControlLLM/Control-LLM-Llama3.1-8B-Math16-Instruct

SGLang

How to use ControlLLM/Control-LLM-Llama3.1-8B-Math16-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ControlLLM/Control-LLM-Llama3.1-8B-Math16-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ControlLLM/Control-LLM-Llama3.1-8B-Math16-Instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ControlLLM/Control-LLM-Llama3.1-8B-Math16-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ControlLLM/Control-LLM-Llama3.1-8B-Math16-Instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ControlLLM/Control-LLM-Llama3.1-8B-Math16-Instruct with Docker Model Runner:
```
docker model run hf.co/ControlLLM/Control-LLM-Llama3.1-8B-Math16-Instruct
```

Control-LLM-Llama3.1-8B-Math16-Instruct / README.md

hawei

Improve model card (#1)

677843e verified over 1 year ago

preview code

Raw

History Blame Contribute Delete

4.22 kB

	---
	license: llama3.1
	datasets:
	- nvidia/OpenMathInstruct-2
	language:
	- en
	base_model:
	- meta-llama/Llama-3.1-8B-Instruct
	model-index:
	- name: Control-LLM-Llama3.1-8B-Math16
	results:
	- task:
	type: math-evaluation
	dataset:
	type: parquet
	name: Math, Math Hard, GSM8K
	dataset_kwargs:
	data_files: "https://github.com/linkedin/ControlLLM/blob/main/src/controlllm/inference/llm_eval_harness/additional_tasks/math/joined_math.parquet"
	metrics:
	- name: exact_match,none
	type: exact_match
	value: 0.6205678398534606
	stderr: 0.005249520342473376
	verified: false
	- name: exact_match,none (gsm8k_0shot_instruct)
	type: exact_match
	value: 0.8968915845337376
	stderr: 0.008376436987507811
	verified: false
	- name: exact_match,none (meta_math_0shot_instruct)
	type: exact_match
	value: 0.6166
	stderr: 0.006876797660918556
	verified: false
	- name: exact_match,none (meta_math_hard_0shot_instruct)
	type: exact_match
	value: 0.36027190332326287
	stderr: 0.013198755610252931
	verified: false
	- task:
	type: original-capability
	dataset:
	type: meta/Llama-3.1-8B-Instruct-evals
	name: Llama-3.1-8B-Instruct-evals Dataset
	dataset_path: "meta-llama/llama-3.1-8_b-instruct-evals"
	dataset_name: "Llama-3.1-8B-Instruct-evals__arc_challenge__details"
	metrics:
	- name: exact_match,strict-match
	type: exact_match
	value: 0.6001372485281902
	stderr: 0.002821514831773572
	verified: false
	- name: exact_match,strict-match (meta_arc_0shot_instruct)
	type: exact_match
	value: 0.8248927038626609
	stderr: 0.011139722235859526
	verified: false
	- name: exact_match,strict-match (meta_gpqa_0shot_cot_instruct)
	type: exact_match
	value: 0.3080357142857143
	stderr: 0.021836780796366417
	verified: false
	- name: exact_match,strict-match (meta_mmlu_0shot_instruct)
	type: exact_match
	value: 0.7159948725252813
	stderr: 0.00380556397209409
	verified: false
	- name: exact_match,strict-match (meta_mmlu_pro_5shot_instruct)
	type: exact_match
	value: 0.45403922872340424
	stderr: 0.004539171007529716
	verified: false
	library_name: transformers
	pipeline_tag: text-generation
	---

	# Control-LLM-Llama3.1-8B-Math16
	This is a fine-tuned model of Llama-3.1-8B-Instruct for mathematical tasks on OpenMath2 dataset.

	## Linked Paper
	This model is associated with the paper: [Control-LLM](https://huggingface.co/papers/2501.10979).

	## Linked Open Source code - training, eval and benchmark
	This model is associated with the github: [Control-LLM](https://github.com/linkedin/ControlLLM).

	## Evaluation Results
	Here is an overview of the evaluation results and findings:

	### Benchmark Results Table
	The table below summarizes evaluation results across mathematical tasks and original capabilities.

	\| Model \| MH \| M \| G8K \| M-Avg \| ARC \| GPQA \| MLU \| MLUP \| O-Avg \| Overall \|
	\|-------------------\|--------\|--------\|---------\|-----------\|---------\|----------\|---------\|----------\|-----------\|-------------\|
	\| Llama3.1-8B-Inst \| 23.7 \| 50.9 \| 85.6 \| 52.1 \| 83.4 \| 29.9 \| 72.4 \| 46.7 \| 60.5 \| 56.3 \|
	\| Control LLM* \| 36.0 \| 61.7 \| 89.7\| 62.5 \| 82.5 \| 30.8 \| 71.6\| 45.4 \| 57.6 \| 60.0 \|

	---
	### Explanation:
	- MH: MathHard
	- M: Math
	- G8K: GSM8K
	- M-Avg: Math - Average across MathHard, Math, and GSM8K
	- ARC: ARC benchmark
	- GPQA: General knowledge QA
	- MLU: MMLU (Massive Multitask Language Understanding)
	- MLUP: MMLU Pro
	- O-Avg: Original Capability - Average across ARC, GPQA, MMLU, and MLUP
	- Overall: Combined average across all tasks

	### Catastrophic Forgetting on OpenMath
	The following plot illustrates and compares catastrophic forgetting mitigation during training

	![Catastrophic Forgetting](plots/ControlLLM_CF_Plot_Math.png)

	### Alignment Result
	The plot below highlights the alignment result of the model trained with Control LLM.

	![Alignment](plots/alignment_best.png)