Instructions to use ISTA-DASLab/Kimi-K3-P48NVFP4-MoESQ with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ISTA-DASLab/Kimi-K3-P48NVFP4-MoESQ with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ISTA-DASLab/Kimi-K3-P48NVFP4-MoESQ", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("ISTA-DASLab/Kimi-K3-P48NVFP4-MoESQ", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ISTA-DASLab/Kimi-K3-P48NVFP4-MoESQ with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ISTA-DASLab/Kimi-K3-P48NVFP4-MoESQ" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ISTA-DASLab/Kimi-K3-P48NVFP4-MoESQ", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/ISTA-DASLab/Kimi-K3-P48NVFP4-MoESQ
- SGLang
How to use ISTA-DASLab/Kimi-K3-P48NVFP4-MoESQ with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ISTA-DASLab/Kimi-K3-P48NVFP4-MoESQ" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ISTA-DASLab/Kimi-K3-P48NVFP4-MoESQ", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ISTA-DASLab/Kimi-K3-P48NVFP4-MoESQ" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ISTA-DASLab/Kimi-K3-P48NVFP4-MoESQ", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use ISTA-DASLab/Kimi-K3-P48NVFP4-MoESQ with Docker Model Runner:
docker model run hf.co/ISTA-DASLab/Kimi-K3-P48NVFP4-MoESQ
12 indexed weight shards missing / 404 EntryNotFound (revision 57ee129)
The published checkpoint appears incomplete. The safetensors index references 94 shards, but the public repository lists only 82. The 12 absent files return 404 EntryNotFound, preventing a complete model load.
Checked on 2026-10-30, with main still at revision:57ee1297912fdf6c9b7de70e6f530a570213228d.
Missing shards
model-00020-of-00094.safetensors
model-00026-of-00094.safetensors
model-00027-of-00094.safetensors
model-00032-of-00094.safetensors
model-00033-of-00094.safetensors
model-00059-of-00094.safetensors
model-00066-of-00094.safetensors
model-00068-of-00094.safetensors
model-00078-of-00094.safetensors
model-00090-of-00094.safetensors
model-00091-of-00094.safetensors
model-00092-of-00094.safetensors
These filenames occur in both the pinned safetensors index and the published SHA256 manifest.
The recursive tree API with limit=1000 fits in one page. Model metadata reports 101 sibling entries, including 82 safetensors files, so this does not appear to be a pagination issue.
Actual GET reproduction and positive control
curl -sS -L --range 0-1023 --max-filesize 1048576 \
-D /tmp/k3-missing.headers -o /tmp/k3-missing.body \
'https://huggingface.co/ISTA-DASLab/Kimi-K3-P48NVFP4-MoESQ/resolve/57ee1297912fdf6c9b7de70e6f530a570213228d/model-00020-of-00094.safetensors'
Result: HTTP 404, X-Error-Code: EntryNotFound, body Entry not found.
The same request for model-00003-of-00094.safetensors succeeds with HTTP 206 / 1024 bytes. Its prefix matches the same file after full verification against the published SHA256.
All 12 missing files were checked using GET, not only HEAD. Three rounds through HF directly and HF Mirror produced 72/72 404 EntryNotFound responses for missing-file requests; all six positive controls succeeded. Later retries included no-cache headers and unique query parameters. Network/TLS failures seen from another machine were recorded separately and are not counted as missing-file evidence.
Xget's documented /hf/ route also returned an upstream 404 for all 12 pinned URLs, while an existing shard was readable. The missing shard-20 URL on main likewise returned 404.
Why these files are required
The index maps these shards to zero-based decoder layers 18, 24, 25, 30, 31, 57, 64, 66, 76, 88, 89 and 90. Together they contain 161,610 indexed tensor entries, including mandatory attention/router tensors and all 896 routed experts for those layers. They cannot simply be omitted when loading the complete model.
Could you please upload the 12 shards matching the published hashes, or point us to a complete source for this same checkpoint? If the index/manifest is intentionally stale, please clarify the intended compatible weight layout.
Cross-posted technical report, including all 12 published SHA256 values: https://github.com/IST-DASLab/MoESQ/issues/1
Thank you for releasing the weights and for checking this.
thanks for checking this! part of the shard was not uploaded due to the organization limits, and is now uploaded.
please re-download the checkpoint and let me know if anything is missing