Instructions to use basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 # Run inference directly in the terminal: llama cli -hf basecompute/Qwen3.6-35B-A3B-BaseQuant-V1
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 # Run inference directly in the terminal: llama cli -hf basecompute/Qwen3.6-35B-A3B-BaseQuant-V1
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 # Run inference directly in the terminal: ./build/bin/llama cli -hf basecompute/Qwen3.6-35B-A3B-BaseQuant-V1
Use Docker
docker model run hf.co/basecompute/Qwen3.6-35B-A3B-BaseQuant-V1
- Llama new
- LM Studio
- Jan
- Ollama
How to use basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 with Ollama:
ollama run hf.co/basecompute/Qwen3.6-35B-A3B-BaseQuant-V1
- Unsloth Desktop
- Pi
How to use basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf basecompute/Qwen3.6-35B-A3B-BaseQuant-V1
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "basecompute/Qwen3.6-35B-A3B-BaseQuant-V1" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 with Docker Model Runner:
docker model run hf.co/basecompute/Qwen3.6-35B-A3B-BaseQuant-V1
- Lemonade
How to use basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull basecompute/Qwen3.6-35B-A3B-BaseQuant-V1
Run and chat with the model
lemonade run user.Qwen3.6-35B-A3B-BaseQuant-V1-{{QUANT_TAG}}List all available models
lemonade list
- Hermes Agent
How to use basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf basecompute/Qwen3.6-35B-A3B-BaseQuant-V1
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default basecompute/Qwen3.6-35B-A3B-BaseQuant-V1
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf basecompute/Qwen3.6-35B-A3B-BaseQuant-V1
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "basecompute/Qwen3.6-35B-A3B-BaseQuant-V1" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Qwen3.6-35B-A3B-BaseQuant-V1
BaseQuant V1 build of Qwen/Qwen3.6-35B-A3B, as a .base bundle for baseRT and a GGUF for llama.cpp.
| Size | baseRT bundle | GGUF (llama.cpp) |
|---|---|---|
| Medium | Qwen3.6-35B-A3B-BaseQuant-V1-Medium.base (23.08 GB) |
Qwen3.6-35B-A3B-BaseQuant-V1-Medium.gguf (22.13 GB) |
The .base bundle also carries the vision tower and the MTP head for speculative decoding. It needs baseRT 0.5.2 or later.
Usage
# baseRT
hf download basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 Qwen3.6-35B-A3B-BaseQuant-V1-Medium.base --local-dir .
basert serve ./Qwen3.6-35B-A3B-BaseQuant-V1-Medium.base
# SuperFluid
curl -fsSL https://superfluid.sh/install.sh | sh
superfluid serve basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 --runtime basert --pull-file Qwen3.6-35B-A3B-BaseQuant-V1-Medium.base
# llama.cpp
llama-server -hf basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 -hff Qwen3.6-35B-A3B-BaseQuant-V1-Medium.gguf -c 32768 -ngl 99
Accuracy
KL divergence of the next-token distribution from the BF16 model, on held-out text: 58 prompts in ten categories, 32,507 scored tokens. None of it was used to build these files. Lower is closer to BF16. The .base bundle is scored in baseRT, and the GGUFs in llama.cpp. unsloth's UD-Q4_K_M, the same size, is shown for reference.
| Category | Tokens | BaseQuant .base |
BaseQuant GGUF | unsloth UD-Q4_K_M |
|---|---|---|---|---|
| Agentic coding | 2,228 | 0.00465 | 0.00490 | 0.00515 |
| Chat | 3,342 | 0.00648 | 0.00634 | 0.00645 |
| Code | 2,883 | 0.00427 | 0.00434 | 0.00450 |
| Long context | 2,048 | 0.00427 | 0.00438 | 0.00536 |
| Math | 4,608 | 0.00304 | 0.00318 | 0.00265 |
| Multilingual | 4,232 | 0.00222 | 0.00213 | 0.00233 |
| Prose | 3,994 | 0.00624 | 0.00587 | 0.00628 |
| Research | 2,123 | 0.00737 | 0.00848 | 0.00584 |
| Science | 4,382 | 0.00467 | 0.00510 | 0.00490 |
| Tool use | 2,667 | 0.00474 | 0.00517 | 0.00519 |
| All | 32,507 | 0.00462 | 0.00476 | 0.00467 |
- Downloads last month
- 244
We're not able to determine the quantization variants.
Model tree for basecompute/Qwen3.6-35B-A3B-BaseQuant-V1
Base model
Qwen/Qwen3.6-35B-A3B