Instructions to use webbigdata/ALMA-7B-Ja-V2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use webbigdata/ALMA-7B-Ja-V2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="webbigdata/ALMA-7B-Ja-V2")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("webbigdata/ALMA-7B-Ja-V2")
model = AutoModelForCausalLM.from_pretrained("webbigdata/ALMA-7B-Ja-V2", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use webbigdata/ALMA-7B-Ja-V2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "webbigdata/ALMA-7B-Ja-V2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "webbigdata/ALMA-7B-Ja-V2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/webbigdata/ALMA-7B-Ja-V2

SGLang

How to use webbigdata/ALMA-7B-Ja-V2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "webbigdata/ALMA-7B-Ja-V2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "webbigdata/ALMA-7B-Ja-V2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "webbigdata/ALMA-7B-Ja-V2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "webbigdata/ALMA-7B-Ja-V2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use webbigdata/ALMA-7B-Ja-V2 with Docker Model Runner:
```
docker model run hf.co/webbigdata/ALMA-7B-Ja-V2
```

ALMA-7B-Ja-V2

Commit History

Nov 1, 2024

18:41
Adding `safetensors` variant of this model⁠

SFconvertbot

Verified signature
845e085

Mar 4, 2024

08:50
Update README.md

dahara1

Verified signature
b110495

Nov 12, 2023

14:16
Update README.md

dahara1

c249758

Nov 6, 2023

12:46
Update README.md

dahara1

21ad81c
10:13
Update README.md

dahara1

6cbacb2
09:20
Update README.md

dahara1

7b909e9
05:15
Update README.md

dahara1

a36a1b1
04:54
Update README.md

dahara1

3cc36e1

Nov 3, 2023

02:40
Update README.md

dahara1

67a66cc
02:35
Update README.md

dahara1

6ad4d23
02:31
Update README.md

dahara1

ed8a82a
02:13
Update README.md

dahara1

207616e
02:08
Update README.md

dahara1

6bf75d8

Nov 2, 2023

12:14
Update README.md

dahara1

7607a32
12:00
Create README.md

dahara1

ffa99bc

Oct 24, 2023

03:27
1085change

dahara1

f5c7e31

Oct 21, 2023

07:51
first commit

dahara1

dc27e3e
07:49
initial commit

dahara1

598a8c9

Commit History

Nov 1, 2024

Adding `safetensors` variant of this model⁠

Mar 4, 2024

Update README.md

Nov 12, 2023

Update README.md

Nov 6, 2023

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Nov 3, 2023

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Nov 2, 2023

Update README.md

Create README.md

Oct 24, 2023

1085change

Oct 21, 2023

first commit

initial commit