Instructions to use MoonstoneF/kosm-checkpoint with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MoonstoneF/kosm-checkpoint with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="MoonstoneF/kosm-checkpoint")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("MoonstoneF/kosm-checkpoint")
model = AutoModelForMultimodalLM.from_pretrained("MoonstoneF/kosm-checkpoint", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use MoonstoneF/kosm-checkpoint with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MoonstoneF/kosm-checkpoint"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MoonstoneF/kosm-checkpoint",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/MoonstoneF/kosm-checkpoint

SGLang

How to use MoonstoneF/kosm-checkpoint with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "MoonstoneF/kosm-checkpoint" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MoonstoneF/kosm-checkpoint",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "MoonstoneF/kosm-checkpoint" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MoonstoneF/kosm-checkpoint",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use MoonstoneF/kosm-checkpoint with Docker Model Runner:
```
docker model run hf.co/MoonstoneF/kosm-checkpoint
```

MoonstoneF commited on Jul 29, 2024

Commit

40ffd3e

verified ·

1 Parent(s): 3d45273

Actually updated text prompt to Q-A

Browse files

Files changed (3) hide show

README.md +6 -6
model-00001-of-00002.safetensors +1 -1
model-00002-of-00002.safetensors +1 -1

README.md CHANGED Viewed

@@ -15,7 +15,7 @@ should probably proofread and complete it, then remove this comment. -->
 This model is a fine-tuned version of [microsoft/kosmos-2-patch14-224](https://huggingface.co/microsoft/kosmos-2-patch14-224) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.0501
 ## Model description
@@ -49,11 +49,11 @@ The following hyperparameters were used during training:
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
-| No log        | 0.0497 | 200  | 0.1044          |
-| 1.0039        | 0.0993 | 400  | 0.0760          |
-| 0.0801        | 0.1490 | 600  | 0.0624          |
-| 0.0728        | 0.1986 | 800  | 0.0538          |
-| 0.0637        | 0.2483 | 1000 | 0.0501          |
 ### Framework versions

 This model is a fine-tuned version of [microsoft/kosmos-2-patch14-224](https://huggingface.co/microsoft/kosmos-2-patch14-224) on the None dataset.
 It achieves the following results on the evaluation set:
+- Loss: 0.0420
 ## Model description
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
+| No log        | 0.0497 | 200  | 0.0700          |
+| 0.0802        | 0.0993 | 400  | 0.0581          |
+| 0.0676        | 0.1490 | 600  | 0.0496          |
+| 0.0584        | 0.1986 | 800  | 0.0450          |
+| 0.0525        | 0.2483 | 1000 | 0.0420          |
 ### Framework versions

model-00001-of-00002.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a680370aa4eec4d4001e6728f8c9235a49bc9c73aeb28c915ad062eef88c1b24
 size 4999738624

 version https://git-lfs.github.com/spec/v1
+oid sha256:34bd24e2a0fe242bc370ddcf0ea8dc04f38da941dde875204acb625e2d35b8fc
 size 4999738624

model-00002-of-00002.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8e39d2f9b6d736af3a1260814896aab55b775b0ec1d5acfb70f3a31eaba29bc4
 size 1658313704

 version https://git-lfs.github.com/spec/v1
+oid sha256:b09da7b3546a2422b851d877a6202d86fecc612a4e03121fc6ec0e8903ae229f
 size 1658313704