Instructions to use DevJadhav/deepseek-v3.2_512M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use DevJadhav/deepseek-v3.2_512M with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf DevJadhav/deepseek-v3.2_512M:Q8_0 # Run inference directly in the terminal: llama cli -hf DevJadhav/deepseek-v3.2_512M:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf DevJadhav/deepseek-v3.2_512M:Q8_0 # Run inference directly in the terminal: llama cli -hf DevJadhav/deepseek-v3.2_512M:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf DevJadhav/deepseek-v3.2_512M:Q8_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf DevJadhav/deepseek-v3.2_512M:Q8_0
Use Docker
docker model run hf.co/DevJadhav/deepseek-v3.2_512M:Q8_0
- Llama new
- LM Studio
- Jan
- vLLM
How to use DevJadhav/deepseek-v3.2_512M with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "DevJadhav/deepseek-v3.2_512M" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DevJadhav/deepseek-v3.2_512M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/DevJadhav/deepseek-v3.2_512M:Q8_0
- Ollama
How to use DevJadhav/deepseek-v3.2_512M with Ollama:
ollama run hf.co/DevJadhav/deepseek-v3.2_512M:Q8_0
- Unsloth Desktop
- Docker Model Runner
How to use DevJadhav/deepseek-v3.2_512M with Docker Model Runner:
docker model run hf.co/DevJadhav/deepseek-v3.2_512M:Q8_0
- Lemonade
How to use DevJadhav/deepseek-v3.2_512M with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull DevJadhav/deepseek-v3.2_512M:Q8_0
Run and chat with the model
lemonade run user.deepseek-v3.2_512M-Q8_0
List all available models
lemonade list
- Atomic Chat
How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf DevJadhav/deepseek-v3.2_512M:Q8_0# Run inference directly in the terminal:
llama cli -hf DevJadhav/deepseek-v3.2_512M:Q8_0Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf DevJadhav/deepseek-v3.2_512M:Q8_0# Run inference directly in the terminal:
./build/bin/llama cli -hf DevJadhav/deepseek-v3.2_512M:Q8_0Use Docker
docker model run hf.co/DevJadhav/deepseek-v3.2_512M:Q8_0Quick Links
deepseek-v3.2_512M
A 512M parameter DeepSeek-style language model trained from scratch, exported to GGUF format.
Model Details
- Architecture: DeepSeek V3.2 style (MLA attention, MoE, MTP)
- Parameters: ~1.34B (512M active)
- Training: 9,900 steps on Modal (8x A100-40GB)
- Final Loss: 10.385
- Format: GGUF (Q8_0 quantization)
Architecture Configuration
| Parameter | Value |
|---|---|
| d_model | 2048 |
| n_heads | 32 |
| n_layers | 24 |
| vocab_size | 32000 |
| d_ff | 8192 |
| max_seq_len | 1024 |
Usage
With llama.cpp
./main -m deepseek-512M-q8_0.gguf -p "Once upon a time" -n 128
With Ollama
Create a Modelfile:
FROM ./deepseek-512M-q8_0.gguf
TEMPLATE "{{.Prompt}}"
PARAMETER temperature 0.7
PARAMETER top_p 0.9
Then run:
ollama create deepseek-512m -f Modelfile
ollama run deepseek-512m
With LM Studio
- Download the GGUF file
- Import into LM Studio
- Start chatting!
Training Details
This model was trained as part of the DeepSeek-From-Scratch project, implementing DeepSeek V3 architecture from scratch.
Training Infrastructure
- Platform: Modal Cloud
- GPUs: 8x NVIDIA A100-40GB
- Parallelism: TP=2, PP=2, DP=2 (5D parallelism)
- Precision: BF16
Limitations
This is a research/educational model trained on synthetic data. It is not intended for production use and may generate nonsensical or harmful content.
License
MIT License - See the repository for details.
- Downloads last month
- 13
Hardware compatibility
Log In to add your hardware
8-bit
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf DevJadhav/deepseek-v3.2_512M:Q8_0# Run inference directly in the terminal: llama cli -hf DevJadhav/deepseek-v3.2_512M:Q8_0