Instructions to use OpenRussianAI/andrey with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpenRussianAI/andrey with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OpenRussianAI/andrey", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelWithLMHead
model = AutoModelWithLMHead.from_pretrained("OpenRussianAI/andrey", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use OpenRussianAI/andrey with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpenRussianAI/andrey"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenRussianAI/andrey",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/OpenRussianAI/andrey

SGLang

How to use OpenRussianAI/andrey with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpenRussianAI/andrey" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenRussianAI/andrey",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpenRussianAI/andrey" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenRussianAI/andrey",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use OpenRussianAI/andrey with Docker Model Runner:
```
docker model run hf.co/OpenRussianAI/andrey
```

XBFDI commited on 23 days ago

Commit

bea1dcb

verified ·

1 Parent(s): 888fc1b

Update tokenizer.json

Browse files

Files changed (1) hide show

tokenizer.json +78 -133

tokenizer.json CHANGED Viewed

@@ -1,135 +1,80 @@
 {
-  "stoi": {
-    "\n": 0,
-    " ": 1,
-    ",": 2,
-    "-": 3,
-    ".": 4,
-    ":": 5,
-    "B": 6,
-    "E": 7,
-    "F": 8,
-    "U": 9,
-    "a": 10,
-    "b": 11,
-    "c": 12,
-    "d": 13,
-    "e": 14,
-    "f": 15,
-    "g": 16,
-    "h": 17,
-    "i": 18,
-    "k": 19,
-    "l": 20,
-    "m": 21,
-    "n": 22,
-    "o": 23,
-    "p": 24,
-    "r": 25,
-    "s": 26,
-    "t": 27,
-    "u": 28,
-    "y": 29,
-    "А": 30,
-    "И": 31,
-    "а": 32,
-    "б": 33,
-    "в": 34,
-    "г": 35,
-    "д": 36,
-    "е": 37,
-    "ж": 38,
-    "з": 39,
-    "и": 40,
-    "й": 41,
-    "к": 42,
-    "л": 43,
-    "м": 44,
-    "н": 45,
-    "о": 46,
-    "п": 47,
-    "р": 48,
-    "с": 49,
-    "т": 50,
-    "у": 51,
-    "ф": 52,
-    "х": 53,
-    "ц": 54,
-    "ч": 55,
-    "ш": 56,
-    "щ": 57,
-    "ы": 58,
-    "ь": 59,
-    "э": 60,
-    "ю": 61,
-    "я": 62,
-    "ё": 63
-  },
-  "itos": {
-    "0": "\n",
-    "1": " ",
-    "2": ",",
-    "3": "-",
-    "4": ".",
-    "5": ":",
-    "6": "B",
-    "7": "E",
-    "8": "F",
-    "9": "U",
-    "10": "a",
-    "11": "b",
-    "12": "c",
-    "13": "d",
-    "14": "e",
-    "15": "f",
-    "16": "g",
-    "17": "h",
-    "18": "i",
-    "19": "k",
-    "20": "l",
-    "21": "m",
-    "22": "n",
-    "23": "o",
-    "24": "p",
-    "25": "r",
-    "26": "s",
-    "27": "t",
-    "28": "u",
-    "29": "y",
-    "30": "А",
-    "31": "И",
-    "32": "а",
-    "33": "б",
-    "34": "в",
-    "35": "г",
-    "36": "д",
-    "37": "е",
-    "38": "ж",
-    "39": "з",
-    "40": "и",
-    "41": "й",
-    "42": "к",
-    "43": "л",
-    "44": "м",
-    "45": "н",
-    "46": "о",
-    "47": "п",
-    "48": "р",
-    "49": "с",
-    "50": "т",
-    "51": "у",
-    "52": "ф",
-    "53": "х",
-    "54": "ц",
-    "55": "ч",
-    "56": "ш",
-    "57": "щ",
-    "58": "ы",
-    "59": "ь",
-    "60": "э",
-    "61": "ю",
-    "62": "я",
-    "63": "ё"
-  },
-  "added_tokens": []
 }

 {
+  "version": "1.0",
+  "truncation": null,
+  "padding": null,
+  "added_tokens": [],
+  "normalizer": null,
+  "pre_tokenizer": null,
+  "post_processor": null,
+  "decoder": null,
+  "model": {
+    "type": "WordLevel",
+    "vocab": {
+      "\n": 0,
+      " ": 1,
+      ",": 2,
+      "-": 3,
+      ".": 4,
+      ":": 5,
+      "B": 6,
+      "E": 7,
+      "F": 8,
+      "U": 9,
+      "a": 10,
+      "b": 11,
+      "c": 12,
+      "d": 13,
+      "e": 14,
+      "f": 15,
+      "g": 16,
+      "h": 17,
+      "i": 18,
+      "k": 19,
+      "l": 20,
+      "m": 21,
+      "n": 22,
+      "o": 23,
+      "p": 24,
+      "r": 25,
+      "s": 26,
+      "t": 27,
+      "u": 28,
+      "y": 29,
+      "А": 30,
+      "И": 31,
+      "а": 32,
+      "б": 33,
+      "в": 34,
+      "г": 35,
+      "д": 36,
+      "е": 37,
+      "ж": 38,
+      "з": 39,
+      "и": 40,
+      "й": 41,
+      "к": 42,
+      "л": 43,
+      "м": 44,
+      "н": 45,
+      "о": 46,
+      "п": 47,
+      "р": 48,
+      "с": 49,
+      "т": 50,
+      "у": 51,
+      "ф": 52,
+      "х": 53,
+      "ц": 54,
+      "ч": 55,
+      "ш": 56,
+      "щ": 57,
+      "ы": 58,
+      "ь": 59,
+      "э": 60,
+      "ю": 61,
+      "я": 62,
+      "ё": 63
+    },
+    "unk_token": "<unk>"
+  }
 }