Userb1az/AliceAI-Foundation-80B-A3B-Base-NVFP4

AliceAI-Foundation-80B-A3B-Base — NVFP4

NVFP4 quantized version of yandex/AliceAI-Foundation-80B-A3B-Base with working Multi-Token Prediction (MTP).

Attribution

• Original model: Yandex
• Quantization: voves
• This repository: re-upload of the quantized weights + verified MTP shard

All rights to the original model belong to Yandex. This repository redistributes the quantized weights under the same Apache-2.0 license.

Quick Start (recommended)

Use the custom vLLM image built for NVIDIA Blackwell (sm_120 / RTX 5090) with working MTP:

docker pull userb1az/alice-vllm-sm120:latest

Run on 2× RTX 5090

docker run --name alice-vllm --rm \
  --gpus '"device=0,1"' --ipc=host \
  -v /path/to/models:/models:ro \
  -p 8000:8000 \
  userb1az/alice-vllm-sm120:latest \
  /models/AliceAI-Foundation-80B-A3B-Base-NVFP4 \
  --served-model-name alice-80b-base \
  --tensor-parallel-size 2 \
  --max-model-len 4096 \
  --max-num-seqs 2 \
  --gpu-memory-utilization 0.85 \
  --attention-backend FLASHINFER \
  --no-enable-flashinfer-autotune \
  --enable-prefix-caching \
  --speculative-config '{"method":"mtp","num_speculative_tokens":2}'

Replace /path/to/models with the directory that contains the model folder.

Test

curl -s http://127.0.0.1:8000/v1/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "alice-80b-base",
    "prompt": "Столица Франции — город",
    "max_tokens": 32,
    "temperature": 0
  }'

MTP acceptance rate on this checkpoint is typically ~95%.

Notes

• Approximate size: ~50 GB
• Context: tested with 4k (can be increased if you have more VRAM)
• Official quality numbers and detailed benchmarks: see the original quantizer repo linked above