Text Generation
GGUF
Safetensors
English
mark_slm_v2
llama.cpp
deepseek2
mixture-of-experts
multi-head-latent-attention
iq4_nl
mark-ai
systems-engineering
custom_code
conversational
Instructions to use miceworks/Mark-SLM-V2-302M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use miceworks/Mark-SLM-V2-302M with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf miceworks/Mark-SLM-V2-302M:IQ4_NL # Run inference directly in the terminal: llama cli -hf miceworks/Mark-SLM-V2-302M:IQ4_NL
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf miceworks/Mark-SLM-V2-302M:IQ4_NL # Run inference directly in the terminal: llama cli -hf miceworks/Mark-SLM-V2-302M:IQ4_NL
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf miceworks/Mark-SLM-V2-302M:IQ4_NL # Run inference directly in the terminal: ./build/bin/llama cli -hf miceworks/Mark-SLM-V2-302M:IQ4_NL
Use Docker
docker model run hf.co/miceworks/Mark-SLM-V2-302M:IQ4_NL
- Llama new
- LM Studio
- Jan
- vLLM
How to use miceworks/Mark-SLM-V2-302M with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "miceworks/Mark-SLM-V2-302M" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "miceworks/Mark-SLM-V2-302M", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/miceworks/Mark-SLM-V2-302M:IQ4_NL
- Ollama
How to use miceworks/Mark-SLM-V2-302M with Ollama:
ollama run hf.co/miceworks/Mark-SLM-V2-302M:IQ4_NL
- Unsloth Desktop
- Docker Model Runner
How to use miceworks/Mark-SLM-V2-302M with Docker Model Runner:
docker model run hf.co/miceworks/Mark-SLM-V2-302M:IQ4_NL
- Lemonade
How to use miceworks/Mark-SLM-V2-302M with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull miceworks/Mark-SLM-V2-302M:IQ4_NL
Run and chat with the model
lemonade run user.Mark-SLM-V2-302M-IQ4_NL
List all available models
lemonade list
- Atomic Chat
Commit History
11 days ago
-
docs: sync test-verified README.md
Verified signature3f7bda6 -
Update verified artifact: mark_slm_v2_302m_iq4_nl.gguf
Verified signature0ce5c43 -
Update verified artifact: gguf_verification_manifest.json
Verified signatureb06716f -
Update verified artifact: config.json
Verified signature6925372 -
Update verified artifact: README.md
Verified signatureee38854
14 days ago
-
fix(audit): update Modelfile with verified forensic metadata
Verified signature7d2464f -
fix(audit): update gguf_verification_manifest.json with verified forensic metadata
Verified signature4925e94 -
fix(audit): update config.json with verified forensic metadata
Verified signaturec83405e -
fix(audit): update README.md with verified forensic metadata
Verified signatureb1e9934 -
Audit fix: update tokenizer_config.json with verified benchmarks, PB-MoE clans, and GGUF metadata
Verified signature6c0cfa8 -
Audit fix: update README.md with verified benchmarks, PB-MoE clans, and GGUF metadata
Verified signatureb201d5f -
Audit fix: update gguf_verification_manifest.json with verified benchmarks, PB-MoE clans, and GGUF metadata
Verified signature9de41b2 -
Audit fix: update README.md with verified benchmarks, PB-MoE clans, and GGUF metadata
Verified signature01d419c
18 days ago
-
Upload model.safetensors for HF Transformers compatibility
Verified signaturecf37717 -
Upload tokenizer_config.json for HF Transformers compatibility
Verified signature1d08795 -
Upload modeling_mark_slm_v2.py for HF Transformers compatibility
Verified signaturec0f543a -
Upload configuration_mark_slm_v2.py for HF Transformers compatibility
Verified signaturede4581f -
Synchronize README.md with GGUF binary single source of truth
Verified signature4db56dc -
Synchronize README.md with GGUF binary single source of truth
Verified signature81cd811 -
Synchronize README.md with GGUF binary single source of truth
Verified signature2ae63e0 -
Synchronize gguf_verification_manifest.json with GGUF binary single source of truth
Verified signature63617d7 -
Synchronize config.json with GGUF binary single source of truth
Verified signatureab63eb4 -
Synchronize README.md with GGUF binary single source of truth
Verified signature9ad6b87 -
Restore trained 10k steps checkpoint in IQ4_NL GGUF
Verified signature0a99859 -
Synchronize gguf_verification_manifest.json with GGUF binary single source of truth
Verified signature208f8ae -
Synchronize Modelfile with GGUF binary single source of truth
Verified signature4070820 -
Synchronize tokenizer.json with GGUF binary single source of truth
Verified signatureb05e98f -
Synchronize tokenizer_config.json with GGUF binary single source of truth
Verified signature0583e7a -
Synchronize config.json with GGUF binary single source of truth
Verified signaturedfdedfe -
Synchronize README.md with GGUF binary single source of truth
Verified signature3b02489 -
Update config.json with verified GGUF metadata
Verified signature3ec7e51 -
Update Modelfile with verified GGUF metadata
Verified signaturec4b670a -
Update README.md with verified GGUF metadata
Verified signatureae105a0 -
Update Mark-SLM-V2-302M native deepseek2 IQ4_NL GGUF container
Verified signature1ff54df -
feat(weights): replace legacy container with verified deepseek2 native GGUF
Verified signature5733f69 -
docs/config: update README.md for deepseek2 native container
Verified signature90b3c57 -
docs/config: update tokenizer_config.json for deepseek2 native container
Verified signaturec3a9db2 -
docs/config: update tokenizer.json for deepseek2 native container
Verified signature17e41ae -
docs/config: update Modelfile for deepseek2 native container
Verified signature1175aaa -
docs/config: update config.json for deepseek2 native container
Verified signature49cb27e -
initial commit
Verified signature046b4a4