Instructions to use OpenRussianAI/andrey with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpenRussianAI/andrey with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OpenRussianAI/andrey", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelWithLMHead
model = AutoModelWithLMHead.from_pretrained("OpenRussianAI/andrey", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use OpenRussianAI/andrey with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpenRussianAI/andrey"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenRussianAI/andrey",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/OpenRussianAI/andrey

SGLang

How to use OpenRussianAI/andrey with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpenRussianAI/andrey" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenRussianAI/andrey",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpenRussianAI/andrey" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenRussianAI/andrey",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use OpenRussianAI/andrey with Docker Model Runner:
```
docker model run hf.co/OpenRussianAI/andrey
```

XBFDI commited on Sep 20

Commit

888fc1b

verified ·

1 Parent(s): 3336636

Update tokenizer.json

Browse files

Files changed (1) hide show

tokenizer.json +135 -1

tokenizer.json CHANGED Viewed

	@@ -1 +1,135 @@
1	- {"stoi": {"\n": 0, " ": 1, ",": 2, "-": 3, ".": 4, ":": 5, "B": 6, "E": 7, "F": 8, "U": 9, "a": 10, "b": 11, "c": 12, "d": 13, "e": 14, "f": 15, "g": 16, "h": 17, "i": 18, "k": 19, "l": 20, "m": 21, "n": 22, "o": 23, "p": 24, "r": 25, "s": 26, "t": 27, "u": 28, "y": 29, "А": 30, "И": 31, "а": 32, "б": 33, "в": 34, "г": 35, "д": 36, "е": 37, "ж": 38, "з": 39, "и": 40, "й": 41, "к": 42, "л": 43, "м": 44, "н": 45, "о": 46, "п": 47, "р": 48, "с": 49, "т": 50, "у": 51, "ф": 52, "х": 53, "ц": 54, "ч": 55, "ш": 56, "щ": 57, "ы": 58, "ь": 59, "э": 60, "ю": 61, "я": 62, "ё": 63}, "itos": {"0": "\n", "1": " ", "2": ",", "3": "-", "4": ".", "5": ":", "6": "B", "7": "E", "8": "F", "9": "U", "10": "a", "11": "b", "12": "c", "13": "d", "14": "e", "15": "f", "16": "g", "17": "h", "18": "i", "19": "k", "20": "l", "21": "m", "22": "n", "23": "o", "24": "p", "25": "r", "26": "s", "27": "t", "28": "u", "29": "y", "30": "А", "31": "И", "32": "а", "33": "б", "34": "в", "35": "г", "36": "д", "37": "е", "38": "ж", "39": "з", "40": "и", "41": "й", "42": "к", "43": "л", "44": "м", "45": "н", "46": "о", "47": "п", "48": "р", "49": "с", "50": "т", "51": "у", "52": "ф", "53": "х", "54": "ц", "55": "ч", "56": "ш", "57": "щ", "58": "ы", "59": "ь", "60": "э", "61": "ю", "62": "я", "63": "ё"}}

+{
+  "stoi": {
+    "\n": 0,
+    " ": 1,
+    ",": 2,
+    "-": 3,
+    ".": 4,
+    ":": 5,
+    "B": 6,
+    "E": 7,
+    "F": 8,
+    "U": 9,
+    "a": 10,
+    "b": 11,
+    "c": 12,
+    "d": 13,
+    "e": 14,
+    "f": 15,
+    "g": 16,
+    "h": 17,
+    "i": 18,
+    "k": 19,
+    "l": 20,
+    "m": 21,
+    "n": 22,
+    "o": 23,
+    "p": 24,
+    "r": 25,
+    "s": 26,
+    "t": 27,
+    "u": 28,
+    "y": 29,
+    "А": 30,
+    "И": 31,
+    "а": 32,
+    "б": 33,
+    "в": 34,
+    "г": 35,
+    "д": 36,
+    "е": 37,
+    "ж": 38,
+    "з": 39,
+    "и": 40,
+    "й": 41,
+    "к": 42,
+    "л": 43,
+    "м": 44,
+    "н": 45,
+    "о": 46,
+    "п": 47,
+    "р": 48,
+    "с": 49,
+    "т": 50,
+    "у": 51,
+    "ф": 52,
+    "х": 53,
+    "ц": 54,
+    "ч": 55,
+    "ш": 56,
+    "щ": 57,
+    "ы": 58,
+    "ь": 59,
+    "э": 60,
+    "ю": 61,
+    "я": 62,
+    "ё": 63
+  },
+  "itos": {
+    "0": "\n",
+    "1": " ",
+    "2": ",",
+    "3": "-",
+    "4": ".",
+    "5": ":",
+    "6": "B",
+    "7": "E",
+    "8": "F",
+    "9": "U",
+    "10": "a",
+    "11": "b",
+    "12": "c",
+    "13": "d",
+    "14": "e",
+    "15": "f",
+    "16": "g",
+    "17": "h",
+    "18": "i",
+    "19": "k",
+    "20": "l",
+    "21": "m",
+    "22": "n",
+    "23": "o",
+    "24": "p",
+    "25": "r",
+    "26": "s",
+    "27": "t",
+    "28": "u",
+    "29": "y",
+    "30": "А",
+    "31": "И",
+    "32": "а",
+    "33": "б",
+    "34": "в",
+    "35": "г",
+    "36": "д",
+    "37": "е",
+    "38": "ж",
+    "39": "з",
+    "40": "и",
+    "41": "й",
+    "42": "к",
+    "43": "л",
+    "44": "м",
+    "45": "н",
+    "46": "о",
+    "47": "п",
+    "48": "р",
+    "49": "с",
+    "50": "т",
+    "51": "у",
+    "52": "ф",
+    "53": "х",
+    "54": "ц",
+    "55": "ч",
+    "56": "ш",
+    "57": "щ",
+    "58": "ы",
+    "59": "ь",
+    "60": "э",
+    "61": "ю",
+    "62": "я",
+    "63": "ё"
+  },
+  "added_tokens": []
+}