Instructions to use SEE7/Astheral with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SEE7/Astheral with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SEE7/Astheral")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("SEE7/Astheral")
model = AutoModelForCausalLM.from_pretrained("SEE7/Astheral", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SEE7/Astheral with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SEE7/Astheral"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SEE7/Astheral",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/SEE7/Astheral

SGLang

How to use SEE7/Astheral with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SEE7/Astheral" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SEE7/Astheral",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SEE7/Astheral" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SEE7/Astheral",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use SEE7/Astheral with Docker Model Runner:
```
docker model run hf.co/SEE7/Astheral
```

Astheral

File size: 2,408 Bytes

{
  "version": "3",
  "training": {
    "status": "complete",
    "parent_revision": "d9aa899722be4788f57107c2951c72d0ec6614ee",
    "hardware": "NVIDIA A10G",
    "job_timeout_seconds": 5400,
    "listed_usd_per_hour": 1.0,
    "training_seed": 20261113,
    "baseline_passed": 40,
    "elapsed_seconds": 622.09,
    "training_examples": 1713,
    "trainable_parameters": 41943040,
    "training": {
      "train_runtime": 390.2404,
      "train_samples_per_second": 1.845,
      "train_steps_per_second": 0.461,
      "total_flos": 3960249019342848.0,
      "train_loss": 0.2867420093880759,
      "epoch": 0.4203152364273205
    },
    "optimizer_steps": 180,
    "candidate_passed": 49,
    "reloaded_passed": 43,
    "groups": {
      "integer": {
        "baseline": 0,
        "candidate": 3,
        "total": 20
      },
      "exact": {
        "baseline": 20,
        "candidate": 20,
        "total": 20
      },
      "code": {
        "baseline": 10,
        "candidate": 10,
        "total": 10
      },
      "json": {
        "baseline": 10,
        "candidate": 10,
        "total": 10
      }
    },
    "quality_gate_passed": true,
    "peak_gpu_bytes": 18991949312
  },
  "matched_bf16_and_independent_tests": {
    "comparison": "V2 versus V3, both BF16",
    "v2_revision": "d9aa899722be4788f57107c2951c72d0ec6614ee",
    "paired_bf16_scores": {
      "base": 41,
      "astheral": 43
    },
    "suite": "GSM8K test rows 231-238; HumanEval selected tasks 0,3,7,12,18,25,36,45",
    "scores": {
      "base": {
        "code": {
          "passed": 6,
          "total": 8,
          "token_limited": 0
        },
        "number": {
          "passed": 6,
          "total": 8,
          "token_limited": 0
        }
      },
      "astheral": {
        "code": {
          "passed": 6,
          "total": 8,
          "token_limited": 0
        },
        "number": {
          "passed": 6,
          "total": 8,
          "token_limited": 0
        }
      }
    },
    "no_material_regression_gate": true,
    "astheral_revision": "8d553fe747fd6e12fd650c97b712e7009a8b4f9e",
    "settings": {
      "sampling": false,
      "code_max_new_tokens": 768,
      "math_max_new_tokens": 1536
    },
    "hardware": "NVIDIA A10G",
    "source_revisions": {
      "gsm8k": "740312add88f781978c0658806c59bc2815b9866",
      "humaneval": "7dce6050a7d6d172f3cc5c32aa97f52fa1a2e544"
    }
  }
}