Text Generation
Transformers
Safetensors
raon1_mini
raon
mxfp8
fp8
w8a8
calibrated-kv-cache
custom-code
conversational
custom_code
Instructions to use cbert33/Raon-1-mini-preview-Calibrated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use cbert33/Raon-1-mini-preview-Calibrated with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="cbert33/Raon-1-mini-preview-Calibrated", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("cbert33/Raon-1-mini-preview-Calibrated", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use cbert33/Raon-1-mini-preview-Calibrated with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "cbert33/Raon-1-mini-preview-Calibrated" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "cbert33/Raon-1-mini-preview-Calibrated", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/cbert33/Raon-1-mini-preview-Calibrated
- SGLang
How to use cbert33/Raon-1-mini-preview-Calibrated with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "cbert33/Raon-1-mini-preview-Calibrated" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "cbert33/Raon-1-mini-preview-Calibrated", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "cbert33/Raon-1-mini-preview-Calibrated" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "cbert33/Raon-1-mini-preview-Calibrated", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use cbert33/Raon-1-mini-preview-Calibrated with Docker Model Runner:
docker model run hf.co/cbert33/Raon-1-mini-preview-Calibrated
Download artifact-validation.json from cbert33/Raon-1-mini-preview-Calibrated: direct link, hf CLI and curl.
- Browser
- Download file 1.75 kB
-
https://huggingface.co/cbert33/Raon-1-mini-preview-Calibrated/resolve/main/artifact-validation.json
- Command line
-
hf download hf://cbert33/Raon-1-mini-preview-Calibrated/artifact-validation.json
-
curl -L -o artifact-validation.json https://huggingface.co/cbert33/Raon-1-mini-preview-Calibrated/resolve/main/artifact-validation.json
1.75 kB
| { | |
| "status": "offline_calibration_and_artifact_checks_passed", | |
| "source_weight_shards_identical": 7, | |
| "source_tensors_preserved": 19434, | |
| "added_scalar_scales": 14, | |
| "final_index_tensor_count": 19448, | |
| "source_quant_method": "mxfp8", | |
| "FP8_KV_scale_scheme": "per-MLA-layer normalized512latent+raw64tail scalar absmax/448", | |
| "source_weights_hardlinked": true, | |
| "native_runtime_patch_required": true, | |
| "native_GPU_consumer": "unexecuted", | |
| "full_calibration_samples": 512, | |
| "full_calibration_tokens": 437514, | |
| "derivative_class_readback": { | |
| "passed": true, | |
| "derivative_class_state_keys": 19448, | |
| "original_state_keys": 19434, | |
| "calibrated_scalars_readback": 14, | |
| "original_weight_bytes_loaded": 0, | |
| "source_tensor_map_unchanged": true, | |
| "CUDA_initialized": false, | |
| "native_GPU_consumer": "unexecuted" | |
| }, | |
| "native_CPU_scalar_readback": { | |
| "passed": true, | |
| "actual_calibrated_layers": [ | |
| 3, | |
| 7, | |
| 11, | |
| 15, | |
| 19, | |
| 23, | |
| 24 | |
| ], | |
| "loaded_scalars": 14, | |
| "native_scalar_finalization_and_Raon_absorption": "passed", | |
| "inverse_scale": "passed", | |
| "whole_native_GPU_model": "unexecuted" | |
| }, | |
| "source_provenance": { | |
| "passed": true, | |
| "source_revision": "7bf87c45fc7b658d3c9532305f3aaed106bbac4f", | |
| "source_payload_files_verified_against_pinned_remote": 24, | |
| "source_payload_file_count": 24, | |
| "protected_derivative_files_equal": 21, | |
| "intentional_derivative_metadata_changes": [ | |
| "config.json", | |
| "model.safetensors.index.json", | |
| "modeling_raon1_mini.py" | |
| ], | |
| "source_weight_shards_hardlink_and_remote_sha256_match": 7, | |
| "all_source_tensors": 19434, | |
| "added_FP32_KV_scalars": 14, | |
| "runtime_GPU_serving": "not_tested" | |
| } | |
| } | |