Instructions to use skt/A.X-K1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use skt/A.X-K1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="skt/A.X-K1") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("skt/A.X-K1") model = AutoModelForCausalLM.from_pretrained("skt/A.X-K1", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use skt/A.X-K1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "skt/A.X-K1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "skt/A.X-K1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/skt/A.X-K1
- SGLang
How to use skt/A.X-K1 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "skt/A.X-K1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "skt/A.X-K1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "skt/A.X-K1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "skt/A.X-K1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use skt/A.X-K1 with Docker Model Runner:
docker model run hf.co/skt/A.X-K1
Commit History
13 days ago
-
Add IFBench evaluation result (#10)
- Add IFBench evaluation result (66f3e3bfa6d23f25d5e67aa6d8d72b91f61467b5) Co-authored-by: Nathan Habib <SaylorTwift@users.noreply.huggingface.co>de35cff
-
Update config.json
Verified signaturebb56f50 -
Convert to fused experts for native transformers support (drop remote code) (#9)
- Convert to fused experts for native transformers support (drop remote code) (7321fe87ec2c1ebd90cc4b99ec966ca2f4afc642)72f54ff
-
Update config.json
Verified signature5c7dc16
-
Add evaluation results (#8)
- Add evaluation results (c00abd23e8aed702f4326b311a9d692e901e2bd4) Co-authored-by: Nathan Habib <SaylorTwift@users.noreply.huggingface.co>f796bf7
-
update README
Kim, Jine923e63
-
update README
Kim, Jin6a0491c -
update README
Kim, Jinc6fef76 -
update README
Kim, Jin938962a
-
update technical report url
Kim, Jin3b90b1e
-
update HLE score
Kim, Jinc3bf4ac
-
update config.json
Kim, Jin4006aa3 -
update license
Kim, Jina637b85
-
Update generation_config.json (#5)
Verified signature- Update generation_config.json (803afc987866ed021624fc6058c6e9546b7aff3e) Co-authored-by: Nicholas Broad <nbroad@users.noreply.huggingface.co>799062e
-
update README
Kim, Jine65d579 -
update README
Kim, Jin57e9c8c
-
add message.content check to prevent UndefinedError (#4)
Verified signature- add message.content check to prevent UndefinedError (dda4e09a838e4e51365381ddccadd38c99b86afc) Co-authored-by: Harish K <togetherhaiku@users.noreply.huggingface.co>3d41848
-
update README
Kim, Jind6fac98
-
update README
Kim, Jin4aa9815 -
Upload README.md with huggingface_hub
Verified signatureebbbc4b
-
Adding `transformers` as the library name (#3)
Verified signature- Adding `transformers` as the library name (468281548028c20c1c3dd910cd864281cc241308) Co-authored-by: Aritra Roy Gosthipaty <ariG23498@users.noreply.huggingface.co>4d3c0ba
-
update README
Kim, Jinbc936d7
-
update README for dark mode
Kim, Jindc2faa5 -
Rename A.X_K1_BI_Negative.png to assets/A.X_K1_BI_Negative.png
Verified signature202a48e -
Upload A.X_K1_BI_Negative.png with huggingface_hub
Verified signaturee210423
-
Upload modeling_axk1.py with huggingface_hub
Verified signature7ba2e80 -
Upload model-00174-of-00260.safetensors with huggingface_hub
Verified signaturee9c94b6 -
Upload shard 260/260
Verified signature6fd9f5a -
Upload shard 259/260
Verified signaturefa05030 -
Upload shard 258/260
Verified signature8f282ab -
Upload shard 257/260
Verified signature7ae74c5 -
Upload shard 256/260
Verified signatureb7ce9b1 -
Upload shard 255/260
Verified signatureaa56376 -
Upload shard 254/260
Verified signature30494d4 -
Upload shard 253/260
Verified signaturea7e9146 -
Upload shard 252/260
Verified signature16ebf69 -
Upload shard 251/260
Verified signatureeff7871 -
Upload shard 250/260
Verified signaturec806cff -
Upload shard 249/260
Verified signaturec86ebf8 -
Upload shard 248/260
Verified signature1923a4a -
Upload shard 247/260
Verified signature8ccd23e -
Upload shard 246/260
Verified signatureae3bf4b -
Upload shard 245/260
Verified signature1192510 -
Upload shard 244/260
Verified signature86c9cd5 -
Upload shard 243/260
Verified signature01ddcd0 -
Upload shard 242/260
Verified signature87056b9 -
Upload shard 241/260
Verified signature0422a9c -
Upload shard 240/260
Verified signature54df264 -
Upload shard 239/260
Verified signatureb0ed3ea -
Upload shard 238/260
Verified signature459ef4c