Instructions to use internlm/internlm2-base-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use internlm/internlm2-base-7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="internlm/internlm2-base-7b", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("internlm/internlm2-base-7b", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use internlm/internlm2-base-7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "internlm/internlm2-base-7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "internlm/internlm2-base-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/internlm/internlm2-base-7b

SGLang

How to use internlm/internlm2-base-7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "internlm/internlm2-base-7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "internlm/internlm2-base-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "internlm/internlm2-base-7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "internlm/internlm2-base-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use internlm/internlm2-base-7b with Docker Model Runner:
```
docker model run hf.co/internlm/internlm2-base-7b
```

internlm2-base-7b

Commit History

Jan 13, 2025

19:55
Adding `safetensors` variant of this model⁠

SFconvertbot

Verified signature
48cc8d0

Aug 20, 2024

06:51
update modeling file to newest

x54-729

4d75220

Jul 3, 2024

06:26
update opencompass url

x54-729

7639c22

Jul 2, 2024

12:27
transforer version

x54-729

7c14023
08:34
update opencompass leaderboard url

x54-729

ac444c2

Jun 21, 2024

12:50
fix flash attention import

x54-729

67275c5

Jun 19, 2024

07:26
small update

x54-729

fff3636

Jun 17, 2024

07:56
update for new version

x54-729

b198e5b

May 14, 2024

12:11
fix eos & update README for tech report

x54-729

438e4c8

Mar 19, 2024

11:58
upload new weights

x54-729

Verified signature
b3a9f51
11:57
upload new weights

x54-729

Verified signature
1cb5714
11:45
upload new weights

x54-729

Verified signature
9e85766

Feb 28, 2024

05:29
fix no white space when using stream_chat with fast tokenizer

x54-729

8c7f44e

Jan 24, 2024

11:57
small

x54-729

62f4666
08:34
fast tokenizer and stream_chat fix (#1)⁠

RangiLyu +1: x54-729

Verified signature
bff2446

Jan 19, 2024

12:46
update rope_scaling

x54-729

ffd2f5b
12:41
remove unnecessary attention_drop

x54-729

c2efc37
12:40
update chat template

x54-729

74d2c19

Jan 18, 2024

12:04
update readme info

x54-729

ec25627
09:05
Update README.md

ZwwWayne

Verified signature
6aa78ed

Jan 17, 2024

04:21
fix example prompt

x54-729

9dfc044
04:01
update readme

x54-729

7edacce

Jan 16, 2024

14:33
fix import error

x54-729

6032db7
11:37
support flash attn 2

x54-729

328cd15

Jan 15, 2024

09:59
Create README.md

ZwwWayne

Verified signature
93039e2

Jan 12, 2024

06:39
update model weights

ZwwWayne

2fb05a1
06:34
update .gitattributes

ZwwWayne

7a4f010
06:18
initial commit

ZwwWayne

Verified signature
15b3423

Commit History

Jan 13, 2025

Adding `safetensors` variant of this model⁠

Aug 20, 2024

update modeling file to newest

Jul 3, 2024

update opencompass url

Jul 2, 2024

transforer version

update opencompass leaderboard url

Jun 21, 2024

fix flash attention import

Jun 19, 2024

small update

Jun 17, 2024

update for new version

May 14, 2024

fix eos & update README for tech report

Mar 19, 2024

upload new weights

upload new weights

upload new weights

Feb 28, 2024

fix no white space when using stream_chat with fast tokenizer

Jan 24, 2024

small

fast tokenizer and stream_chat fix (#1)⁠

Jan 19, 2024

update rope_scaling

remove unnecessary attention_drop

update chat template

Jan 18, 2024

update readme info

Update README.md

Jan 17, 2024

fix example prompt

update readme

Jan 16, 2024

fix import error

support flash attn 2

Jan 15, 2024

Create README.md

Jan 12, 2024

update model weights

update .gitattributes

initial commit