Instructions to use Multi-Domain-Expert-Learning/given-mpt-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Multi-Domain-Expert-Learning/given-mpt-7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Multi-Domain-Expert-Learning/given-mpt-7b", trust_remote_code=True)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Multi-Domain-Expert-Learning/given-mpt-7b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("Multi-Domain-Expert-Learning/given-mpt-7b", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Multi-Domain-Expert-Learning/given-mpt-7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Multi-Domain-Expert-Learning/given-mpt-7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Multi-Domain-Expert-Learning/given-mpt-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Multi-Domain-Expert-Learning/given-mpt-7b

SGLang

How to use Multi-Domain-Expert-Learning/given-mpt-7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Multi-Domain-Expert-Learning/given-mpt-7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Multi-Domain-Expert-Learning/given-mpt-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Multi-Domain-Expert-Learning/given-mpt-7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Multi-Domain-Expert-Learning/given-mpt-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Multi-Domain-Expert-Learning/given-mpt-7b with Docker Model Runner:
```
docker model run hf.co/Multi-Domain-Expert-Learning/given-mpt-7b
```

given-mpt-7b / README.md

Ontocord.AI

Update README.md

2cd41a9 about 3 years ago

preview code

Raw

History Blame Contribute Delete

12.1 kB

	---
	license: apache-2.0
	inference: false
	---
	# Given-MPT-7B

	This is a merge of the following MPT-7B models:

	- gorilla-llm/gorilla-mpt-7b-hf-v0
	- ibm/mpt-7b-instruct2
	- TehVenom/MPT-7b-WizardLM_Uncensored-Storywriter-Merge
	- emozilla/mpt-7b-storysummarizer
	- nomic-ai/gpt4all-mpt


	## Model License

	Apache 2.0


	## Purpose

	This model is for experimenting with merging and routing to expert layers.

	# Test eval on only 10% of eval set

	hf-causal (pretrained=Multi-Domain-Expert-Layers/given-mpt-7b,dtype=bfloat16,trust_remote_code=True), limit: 0.1, provide_description: False, num_fewshot: 0, batch_size: None
	\| Task \|Version\| Metric \| Value \| \|Stderr\|
	\|-------------------------------------------------\|------:\|-----------\|------:\|---\|-----:\|
	\|arc_challenge \| 0\|acc \| 0.4274\|± \|0.0459\|
	\| \| \|acc_norm \| 0.3846\|± \|0.0452\|
	\|arc_easy \| 0\|acc \| 0.7863\|± \|0.0381\|
	\| \| \|acc_norm \| 0.7350\|± \|0.0410\|
	\|hellaswag \| 0\|acc \| 0.5556\|± \|0.0461\|
	\| \| \|acc_norm \| 0.8120\|± \|0.0363\|
	\|hendrycksTest-college_chemistry \| 0\|acc \| 0.3600\|± \|0.0482\|
	\| \| \|acc_norm \| 0.3700\|± \|0.0485\|
	\|hendrycksTest-college_computer_science \| 0\|acc \| 0.3400\|± \|0.0476\|
	\| \| \|acc_norm \| 0.3600\|± \|0.0482\|
	\|hendrycksTest-college_mathematics \| 0\|acc \| 0.2500\|± \|0.0435\|
	\| \| \|acc_norm \| 0.2900\|± \|0.0456\|
	\|hendrycksTest-college_medicine \| 0\|acc \| 0.3675\|± \|0.0448\|
	\| \| \|acc_norm \| 0.3162\|± \|0.0432\|
	\|hendrycksTest-college_physics \| 0\|acc \| 0.2451\|± \|0.0428\|
	\| \| \|acc_norm \| 0.2941\|± \|0.0453\|
	\|hendrycksTest-computer_security \| 0\|acc \| 0.4800\|± \|0.0502\|
	\| \| \|acc_norm \| 0.4400\|± \|0.0499\|
	\|hendrycksTest-conceptual_physics \| 0\|acc \| 0.2051\|± \|0.0375\|
	\| \| \|acc_norm \| 0.1709\|± \|0.0350\|
	\|hendrycksTest-econometrics \| 0\|acc \| 0.2982\|± \|0.0430\|
	\| \| \|acc_norm \| 0.2368\|± \|0.0400\|
	\|hendrycksTest-electrical_engineering \| 0\|acc \| 0.3248\|± \|0.0435\|
	\| \| \|acc_norm \| 0.3590\|± \|0.0445\|
	\|hendrycksTest-elementary_mathematics \| 0\|acc \| 0.3333\|± \|0.0438\|
	\| \| \|acc_norm \| 0.3162\|± \|0.0432\|
	\|hendrycksTest-formal_logic \| 0\|acc \| 0.3077\|± \|0.0429\|
	\| \| \|acc_norm \| 0.3248\|± \|0.0435\|
	\|hendrycksTest-global_facts \| 0\|acc \| 0.3000\|± \|0.0461\|
	\| \| \|acc_norm \| 0.2700\|± \|0.0446\|
	\|hendrycksTest-high_school_biology \| 0\|acc \| 0.3675\|± \|0.0448\|
	\| \| \|acc_norm \| 0.3077\|± \|0.0429\|
	\|hendrycksTest-high_school_chemistry \| 0\|acc \| 0.2564\|± \|0.0405\|
	\| \| \|acc_norm \| 0.2906\|± \|0.0422\|
	\|hendrycksTest-high_school_computer_science \| 0\|acc \| 0.4100\|± \|0.0494\|
	\| \| \|acc_norm \| 0.4400\|± \|0.0499\|
	\|hendrycksTest-high_school_european_history \| 0\|acc \| 0.4359\|± \|0.0460\|
	\| \| \|acc_norm \| 0.3590\|± \|0.0445\|
	\|hendrycksTest-high_school_geography \| 0\|acc \| 0.3248\|± \|0.0435\|
	\| \| \|acc_norm \| 0.3675\|± \|0.0448\|
	\|hendrycksTest-high_school_government_and_politics\| 0\|acc \| 0.3932\|± \|0.0454\|
	\| \| \|acc_norm \| 0.3932\|± \|0.0454\|
	\|hendrycksTest-high_school_macroeconomics \| 0\|acc \| 0.3333\|± \|0.0438\|
	\| \| \|acc_norm \| 0.3248\|± \|0.0435\|
	\|hendrycksTest-high_school_mathematics \| 0\|acc \| 0.2051\|± \|0.0375\|
	\| \| \|acc_norm \| 0.2564\|± \|0.0405\|
	\|hendrycksTest-high_school_microeconomics \| 0\|acc \| 0.3504\|± \|0.0443\|
	\| \| \|acc_norm \| 0.4188\|± \|0.0458\|
	\|hendrycksTest-high_school_physics \| 0\|acc \| 0.2650\|± \|0.0410\|
	\| \| \|acc_norm \| 0.2906\|± \|0.0422\|
	\|hendrycksTest-high_school_psychology \| 0\|acc \| 0.3761\|± \|0.0450\|
	\| \| \|acc_norm \| 0.3419\|± \|0.0440\|
	\|hendrycksTest-high_school_statistics \| 0\|acc \| 0.3077\|± \|0.0429\|
	\| \| \|acc_norm \| 0.3504\|± \|0.0443\|
	\|hendrycksTest-high_school_us_history \| 0\|acc \| 0.3333\|± \|0.0438\|
	\| \| \|acc_norm \| 0.3333\|± \|0.0438\|
	\|hendrycksTest-high_school_world_history \| 0\|acc \| 0.3333\|± \|0.0438\|
	\| \| \|acc_norm \| 0.3419\|± \|0.0440\|
	\|hendrycksTest-human_aging \| 0\|acc \| 0.3761\|± \|0.0450\|
	\| \| \|acc_norm \| 0.3162\|± \|0.0432\|
	\|hendrycksTest-human_sexuality \| 0\|acc \| 0.4274\|± \|0.0459\|
	\| \| \|acc_norm \| 0.3761\|± \|0.0450\|
	\|hendrycksTest-international_law \| 0\|acc \| 0.4188\|± \|0.0458\|
	\| \| \|acc_norm \| 0.4957\|± \|0.0464\|
	\|hendrycksTest-jurisprudence \| 0\|acc \| 0.3148\|± \|0.0449\|
	\| \| \|acc_norm \| 0.4815\|± \|0.0483\|
	\|hendrycksTest-logical_fallacies \| 0\|acc \| 0.3504\|± \|0.0443\|
	\| \| \|acc_norm \| 0.3675\|± \|0.0448\|
	\|hendrycksTest-machine_learning \| 0\|acc \| 0.3214\|± \|0.0443\|
	\| \| \|acc_norm \| 0.2946\|± \|0.0433\|
	\|hendrycksTest-management \| 0\|acc \| 0.3786\|± \|0.0480\|
	\| \| \|acc_norm \| 0.3495\|± \|0.0472\|
	\|hendrycksTest-marketing \| 0\|acc \| 0.5043\|± \|0.0464\|
	\| \| \|acc_norm \| 0.4188\|± \|0.0458\|
	\|hendrycksTest-medical_genetics \| 0\|acc \| 0.3200\|± \|0.0469\|
	\| \| \|acc_norm \| 0.4100\|± \|0.0494\|
	\|hendrycksTest-miscellaneous \| 0\|acc \| 0.5299\|± \|0.0463\|
	\| \| \|acc_norm \| 0.4872\|± \|0.0464\|
	\|hendrycksTest-moral_disputes \| 0\|acc \| 0.3248\|± \|0.0435\|
	\| \| \|acc_norm \| 0.3162\|± \|0.0432\|
	\|hendrycksTest-moral_scenarios \| 0\|acc \| 0.3248\|± \|0.0435\|
	\| \| \|acc_norm \| 0.2479\|± \|0.0401\|
	\|hendrycksTest-nutrition \| 0\|acc \| 0.3675\|± \|0.0448\|
	\| \| \|acc_norm \| 0.3932\|± \|0.0454\|
	\|hendrycksTest-philosophy \| 0\|acc \| 0.2991\|± \|0.0425\|
	\| \| \|acc_norm \| 0.3504\|± \|0.0443\|
	\|hendrycksTest-prehistory \| 0\|acc \| 0.2821\|± \|0.0418\|
	\| \| \|acc_norm \| 0.3248\|± \|0.0435\|
	\|hendrycksTest-professional_accounting \| 0\|acc \| 0.2137\|± \|0.0381\|
	\| \| \|acc_norm \| 0.2222\|± \|0.0386\|
	\|hendrycksTest-professional_law \| 0\|acc \| 0.3077\|± \|0.0429\|
	\| \| \|acc_norm \| 0.2735\|± \|0.0414\|
	\|hendrycksTest-professional_medicine \| 0\|acc \| 0.2991\|± \|0.0425\|
	\| \| \|acc_norm \| 0.2650\|± \|0.0410\|
	\|hendrycksTest-professional_psychology \| 0\|acc \| 0.3248\|± \|0.0435\|
	\| \| \|acc_norm \| 0.3419\|± \|0.0440\|
	\|hendrycksTest-public_relations \| 0\|acc \| 0.3909\|± \|0.0467\|
	\| \| \|acc_norm \| 0.3545\|± \|0.0458\|
	\|hendrycksTest-security_studies \| 0\|acc \| 0.3419\|± \|0.0440\|
	\| \| \|acc_norm \| 0.2906\|± \|0.0422\|
	\|hendrycksTest-sociology \| 0\|acc \| 0.3761\|± \|0.0450\|
	\| \| \|acc_norm \| 0.3162\|± \|0.0432\|
	\|hendrycksTest-us_foreign_policy \| 0\|acc \| 0.5000\|± \|0.0503\|
	\| \| \|acc_norm \| 0.4100\|± \|0.0494\|
	\|hendrycksTest-virology \| 0\|acc \| 0.3932\|± \|0.0454\|
	\| \| \|acc_norm \| 0.3248\|± \|0.0435\|
	\|hendrycksTest-world_religions \| 0\|acc \| 0.5299\|± \|0.0463\|
	\| \| \|acc_norm \| 0.5128\|± \|0.0464\|
	\|truthfulqa_gen \| 1\|bleurt_max \|-0.8551\|± \|0.0501\|
	\| \| \|bleurt_acc \| 0.3590\|± \|0.0445\|
	\| \| \|bleurt_diff\|-0.1292\|± \|0.0483\|
	\| \| \|bleu_max \|19.3738\|± \|1.8461\|
	\| \| \|bleu_acc \| 0.3932\|± \|0.0454\|
	\| \| \|bleu_diff \|-4.3883\|± \|2.1748\|
	\| \| \|rouge1_max \|41.8428\|± \|2.6156\|
	\| \| \|rouge1_acc \| 0.3162\|± \|0.0432\|
	\| \| \|rouge1_diff\|-8.8583\|± \|2.7745\|
	\| \| \|rouge2_max \|26.3956\|± \|2.8311\|
	\| \| \|rouge2_acc \| 0.2137\|± \|0.0381\|
	\| \| \|rouge2_diff\|-9.5287\|± \|3.3258\|
	\| \| \|rougeL_max \|39.5215\|± \|2.5620\|
	\| \| \|rougeL_acc \| 0.3162\|± \|0.0432\|
	\| \| \|rougeL_diff\|-8.5753\|± \|2.8259\|

	---
	license: apache-2.0
	inference: false
	---
	# Given-MPT-7B

	This is a merge of the following MPT-7B models:

	- gorilla-llm/gorilla-mpt-7b-hf-v0
	- ibm/mpt-7b-instruct2
	- TehVenom/MPT-7b-WizardLM_Uncensored-Storywriter-Merge
	- emozilla/mpt-7b-storysummarizer
	- nomic-ai/gpt4all-mpt


	## Model License

	Apache 2.0


	## Purpose

	This model is for experimenting with merging and routing to expert layers.

	# Test eval on only 10% of eval set

	hf-causal (pretrained=Multi-Domain-Expert-Layers/given-mpt-7b,dtype=bfloat16,trust_remote_code=True), limit: 0.1, provide_description: False, num_fewshot: 0, batch_size: None
	\| Task \|Version\| Metric \| Value \| \|Stderr\|
	\|-------------------------------------------------\|------:\|-----------\|------:\|---\|-----:\|
	\|arc_challenge \| 0\|acc \| 0.4274\|± \|0.0459\|
	\| \| \|acc_norm \| 0.3846\|± \|0.0452\|
	\|arc_easy \| 0\|acc \| 0.7863\|± \|0.0381\|
	\| \| \|acc_norm \| 0.7350\|± \|0.0410\|
	\|hellaswag \| 0\|acc \| 0.5556\|± \|0.0461\|
	\| \| \|acc_norm \| 0.8120\|± \|0.0363\|
	\|hendrycksTest-college_chemistry \| 0\|acc \| 0.3600\|± \|0.0482\|
	\| \| \|acc_norm \| 0.3700\|± \|0.0485\|
	\|hendrycksTest-college_computer_science \| 0\|acc \| 0.3400\|± \|0.0476\|
	\| \| \|acc_norm \| 0.3600\|± \|0.0482\|
	\|hendrycksTest-college_mathematics \| 0\|acc \| 0.2500\|± \|0.0435\|
	\| \| \|acc_norm \| 0.2900\|± \|0.0456\|
	\|hendrycksTest-college_medicine \| 0\|acc \| 0.3675\|± \|0.0448\|
	\| \| \|acc_norm \| 0.3162\|± \|0.0432\|
	\|hendrycksTest-college_physics \| 0\|acc \| 0.2451\|± \|0.0428\|
	\| \| \|acc_norm \| 0.2941\|± \|0.0453\|
	\|hendrycksTest-computer_security \| 0\|acc \| 0.4800\|± \|0.0502\|
	\| \| \|acc_norm \| 0.4400\|± \|0.0499\|
	\|hendrycksTest-conceptual_physics \| 0\|acc \| 0.2051\|± \|0.0375\|
	\| \| \|acc_norm \| 0.1709\|± \|0.0350\|
	\|hendrycksTest-econometrics \| 0\|acc \| 0.2982\|± \|0.0430\|
	\| \| \|acc_norm \| 0.2368\|± \|0.0400\|
	\|hendrycksTest-electrical_engineering \| 0\|acc \| 0.3248\|± \|0.0435\|
	\| \| \|acc_norm \| 0.3590\|± \|0.0445\|
	\|hendrycksTest-elementary_mathematics \| 0\|acc \| 0.3333\|± \|0.0438\|
	\| \| \|acc_norm \| 0.3162\|± \|0.0432\|
	\|hendrycksTest-formal_logic \| 0\|acc \| 0.3077\|± \|0.0429\|
	\| \| \|acc_norm \| 0.3248\|± \|0.0435\|
	\|hendrycksTest-global_facts \| 0\|acc \| 0.3000\|± \|0.0461\|
	\| \| \|acc_norm \| 0.2700\|± \|0.0446\|
	\|hendrycksTest-high_school_biology \| 0\|acc \| 0.3675\|± \|0.0448\|
	\| \| \|acc_norm \| 0.3077\|± \|0.0429\|
	\|hendrycksTest-high_school_chemistry \| 0\|acc \| 0.2564\|± \|0.0405\|
	\| \| \|acc_norm \| 0.2906\|± \|0.0422\|
	\|hendrycksTest-high_school_computer_science \| 0\|acc \| 0.4100\|± \|0.0494\|
	\| \| \|acc_norm \| 0.4400\|± \|0.0499\|
	\|hendrycksTest-high_school_european_history \| 0\|acc \| 0.4359\|± \|0.0460\|
	\| \| \|acc_norm \| 0.3590\|± \|0.0445\|
	\|hendrycksTest-high_school_geography \| 0\|acc \| 0.3248\|± \|0.0435\|
	\| \| \|acc_norm \| 0.3675\|± \|0.0448\|
	\|hendrycksTest-high_school_government_and_politics\| 0\|acc \| 0.3932\|± \|0.0454\|
	\| \| \|acc_norm \| 0.3932\|± \|0.0454\|
	\|hendrycksTest-high_school_macroeconomics \| 0\|acc \| 0.3333\|± \|0.0438\|
	\| \| \|acc_norm \| 0.3248\|± \|0.0435\|
	\|hendrycksTest-high_school_mathematics \| 0\|acc \| 0.2051\|± \|0.0375\|
	\| \| \|acc_norm \| 0.2564\|± \|0.0405\|
	\|hendrycksTest-high_school_microeconomics \| 0\|acc \| 0.3504\|± \|0.0443\|
	\| \| \|acc_norm \| 0.4188\|± \|0.0458\|
	\|hendrycksTest-high_school_physics \| 0\|acc \| 0.2650\|± \|0.0410\|
	\| \| \|acc_norm \| 0.2906\|± \|0.0422\|
	\|hendrycksTest-high_school_psychology \| 0\|acc \| 0.3761\|± \|0.0450\|
	\| \| \|acc_norm \| 0.3419\|± \|0.0440\|
	\|hendrycksTest-high_school_statistics \| 0\|acc \| 0.3077\|± \|0.0429\|
	\| \| \|acc_norm \| 0.3504\|± \|0.0443\|
	\|hendrycksTest-high_school_us_history \| 0\|acc \| 0.3333\|± \|0.0438\|
	\| \| \|acc_norm \| 0.3333\|± \|0.0438\|
	\|hendrycksTest-high_school_world_history \| 0\|acc \| 0.3333\|± \|0.0438\|
	\| \| \|acc_norm \| 0.3419\|± \|0.0440\|
	\|hendrycksTest-human_aging \| 0\|acc \| 0.3761\|± \|0.0450\|
	\| \| \|acc_norm \| 0.3162\|± \|0.0432\|
	\|hendrycksTest-human_sexuality \| 0\|acc \| 0.4274\|± \|0.0459\|
	\| \| \|acc_norm \| 0.3761\|± \|0.0450\|
	\|hendrycksTest-international_law \| 0\|acc \| 0.4188\|± \|0.0458\|
	\| \| \|acc_norm \| 0.4957\|± \|0.0464\|
	\|hendrycksTest-jurisprudence \| 0\|acc \| 0.3148\|± \|0.0449\|
	\| \| \|acc_norm \| 0.4815\|± \|0.0483\|
	\|hendrycksTest-logical_fallacies \| 0\|acc \| 0.3504\|± \|0.0443\|
	\| \| \|acc_norm \| 0.3675\|± \|0.0448\|
	\|hendrycksTest-machine_learning \| 0\|acc \| 0.3214\|± \|0.0443\|
	\| \| \|acc_norm \| 0.2946\|± \|0.0433\|
	\|hendrycksTest-management \| 0\|acc \| 0.3786\|± \|0.0480\|
	\| \| \|acc_norm \| 0.3495\|± \|0.0472\|
	\|hendrycksTest-marketing \| 0\|acc \| 0.5043\|± \|0.0464\|
	\| \| \|acc_norm \| 0.4188\|± \|0.0458\|
	\|hendrycksTest-medical_genetics \| 0\|acc \| 0.3200\|± \|0.0469\|
	\| \| \|acc_norm \| 0.4100\|± \|0.0494\|
	\|hendrycksTest-miscellaneous \| 0\|acc \| 0.5299\|± \|0.0463\|
	\| \| \|acc_norm \| 0.4872\|± \|0.0464\|
	\|hendrycksTest-moral_disputes \| 0\|acc \| 0.3248\|± \|0.0435\|
	\| \| \|acc_norm \| 0.3162\|± \|0.0432\|
	\|hendrycksTest-moral_scenarios \| 0\|acc \| 0.3248\|± \|0.0435\|
	\| \| \|acc_norm \| 0.2479\|± \|0.0401\|
	\|hendrycksTest-nutrition \| 0\|acc \| 0.3675\|± \|0.0448\|
	\| \| \|acc_norm \| 0.3932\|± \|0.0454\|
	\|hendrycksTest-philosophy \| 0\|acc \| 0.2991\|± \|0.0425\|
	\| \| \|acc_norm \| 0.3504\|± \|0.0443\|
	\|hendrycksTest-prehistory \| 0\|acc \| 0.2821\|± \|0.0418\|
	\| \| \|acc_norm \| 0.3248\|± \|0.0435\|
	\|hendrycksTest-professional_accounting \| 0\|acc \| 0.2137\|± \|0.0381\|
	\| \| \|acc_norm \| 0.2222\|± \|0.0386\|
	\|hendrycksTest-professional_law \| 0\|acc \| 0.3077\|± \|0.0429\|
	\| \| \|acc_norm \| 0.2735\|± \|0.0414\|
	\|hendrycksTest-professional_medicine \| 0\|acc \| 0.2991\|± \|0.0425\|
	\| \| \|acc_norm \| 0.2650\|± \|0.0410\|
	\|hendrycksTest-professional_psychology \| 0\|acc \| 0.3248\|± \|0.0435\|
	\| \| \|acc_norm \| 0.3419\|± \|0.0440\|
	\|hendrycksTest-public_relations \| 0\|acc \| 0.3909\|± \|0.0467\|
	\| \| \|acc_norm \| 0.3545\|± \|0.0458\|
	\|hendrycksTest-security_studies \| 0\|acc \| 0.3419\|± \|0.0440\|
	\| \| \|acc_norm \| 0.2906\|± \|0.0422\|
	\|hendrycksTest-sociology \| 0\|acc \| 0.3761\|± \|0.0450\|
	\| \| \|acc_norm \| 0.3162\|± \|0.0432\|
	\|hendrycksTest-us_foreign_policy \| 0\|acc \| 0.5000\|± \|0.0503\|
	\| \| \|acc_norm \| 0.4100\|± \|0.0494\|
	\|hendrycksTest-virology \| 0\|acc \| 0.3932\|± \|0.0454\|
	\| \| \|acc_norm \| 0.3248\|± \|0.0435\|
	\|hendrycksTest-world_religions \| 0\|acc \| 0.5299\|± \|0.0463\|
	\| \| \|acc_norm \| 0.5128\|± \|0.0464\|
	\|truthfulqa_gen \| 1\|bleurt_max \|-0.8551\|± \|0.0501\|
	\| \| \|bleurt_acc \| 0.3590\|± \|0.0445\|
	\| \| \|bleurt_diff\|-0.1292\|± \|0.0483\|
	\| \| \|bleu_max \|19.3738\|± \|1.8461\|
	\| \| \|bleu_acc \| 0.3932\|± \|0.0454\|
	\| \| \|bleu_diff \|-4.3883\|± \|2.1748\|
	\| \| \|rouge1_max \|41.8428\|± \|2.6156\|
	\| \| \|rouge1_acc \| 0.3162\|± \|0.0432\|
	\| \| \|rouge1_diff\|-8.8583\|± \|2.7745\|
	\| \| \|rouge2_max \|26.3956\|± \|2.8311\|
	\| \| \|rouge2_acc \| 0.2137\|± \|0.0381\|
	\| \| \|rouge2_diff\|-9.5287\|± \|3.3258\|
	\| \| \|rougeL_max \|39.5215\|± \|2.5620\|
	\| \| \|rougeL_acc \| 0.3162\|± \|0.0432\|
	\| \| \|rougeL_diff\|-8.5753\|± \|2.8259\|