Instructions to use solintellegence/Sol-Lite-2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use solintellegence/Sol-Lite-2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="solintellegence/Sol-Lite-2", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("solintellegence/Sol-Lite-2", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use solintellegence/Sol-Lite-2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "solintellegence/Sol-Lite-2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "solintellegence/Sol-Lite-2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/solintellegence/Sol-Lite-2

SGLang

How to use solintellegence/Sol-Lite-2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "solintellegence/Sol-Lite-2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "solintellegence/Sol-Lite-2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "solintellegence/Sol-Lite-2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "solintellegence/Sol-Lite-2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use solintellegence/Sol-Lite-2 with Docker Model Runner:
```
docker model run hf.co/solintellegence/Sol-Lite-2
```

Sol-Lite-2 / evaluation /index.json

j0no12

Document Sol Lite 2 architecture, 20B training, and final evaluation

b209c37 verified 29 days ago

Raw History Blame Contribute Delete

1 kB

{
  "scores": {
    "hellaswag": 28.689504082852025,
    "arc_easy": 36.27946127946128,
    "arc_challenge": 24.914675767918087,
    "piqa": 56.80087051142546,
    "arithmark3": 35.5
  },
  "counts": {
    "hellaswag": 10042,
    "arc_easy": 2376,
    "arc_challenge": 1172,
    "piqa": 1838,
    "arithmark3": 1000
  },
  "intelligence_index": 9.612010364814575,
  "checkpoint_sha256": "7b4ec38fb123d32e867df7f65dfcbd49f65b9daef78e1255d0328cc7955a6438",
  "tokenizer_sha256": "ed6b75f7fad6994e75932c98cd1bc8dfdcaa88d4be8bd905bab970963f5c10b0",
  "arithmark_sha256": "bf8ab1a5193d52cdf0e05ff0b3ca226bdfcf416cb6e75562dcbe72e7e4559435",
  "tokens_seen": 20000000000,
  "zero_shot": true,
  "dtype": "float32",
  "lm_eval": "0.4.12",
  "max_context": 2048,
  "truncated_requests": 0,
  "seconds": 426.45051527023315,
  "methodology": "https://huggingface.co/spaces/AxiomicLabs/Open_SLM_Leaderboard/blob/main/index.html",
  "stage": "final",
  "development_proxy": false,
  "development_protocol": null
}

	{
	"scores": {
	"hellaswag": 28.689504082852025,
	"arc_easy": 36.27946127946128,
	"arc_challenge": 24.914675767918087,
	"piqa": 56.80087051142546,
	"arithmark3": 35.5
	},
	"counts": {
	"hellaswag": 10042,
	"arc_easy": 2376,
	"arc_challenge": 1172,
	"piqa": 1838,
	"arithmark3": 1000
	},
	"intelligence_index": 9.612010364814575,
	"checkpoint_sha256": "7b4ec38fb123d32e867df7f65dfcbd49f65b9daef78e1255d0328cc7955a6438",
	"tokenizer_sha256": "ed6b75f7fad6994e75932c98cd1bc8dfdcaa88d4be8bd905bab970963f5c10b0",
	"arithmark_sha256": "bf8ab1a5193d52cdf0e05ff0b3ca226bdfcf416cb6e75562dcbe72e7e4559435",
	"tokens_seen": 20000000000,
	"zero_shot": true,
	"dtype": "float32",
	"lm_eval": "0.4.12",
	"max_context": 2048,
	"truncated_requests": 0,
	"seconds": 426.45051527023315,
	"methodology": "https://huggingface.co/spaces/AxiomicLabs/Open_SLM_Leaderboard/blob/main/index.html",
	"stage": "final",
	"development_proxy": false,
	"development_protocol": null
	}