How to use from
llama.cpp
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf voperl/PsychQwen3_v4:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf voperl/PsychQwen3_v4:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf voperl/PsychQwen3_v4:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf voperl/PsychQwen3_v4:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf voperl/PsychQwen3_v4:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf voperl/PsychQwen3_v4:Q4_K_M
Use Docker
docker model run hf.co/voperl/PsychQwen3_v4:Q4_K_M
Quick Links

PsychQwen3_v4 : GGUF

This model was finetuned and converted to GGUF format using Unsloth.

This model was created for fun, when experimenting with datasets and reasoning in LLMs, This model is a LoRA SFT of Qwen3 8B, which is already an instruct model though not specialized in psychiatry psychqwen3_v4 was trained on a dataset consisting of 20k pairs of question and answer, thinking blocks were added into the answer because psychqwen3_v3 which was trained on the same dataset without thinking blocks couldn't think after the fine-tuning at all.

you are free to deploy and test this model or test it until i stop tunneling my local server via ngrok to the internet connecting this model in inference endpoints and server part with all the tools it has

regarding the tools it has, it can search ICD-11, pull drug information, calculate and search pubmed though not always it acts according to what an optimal model would do as an assistant.

the model hasn't been benchmarked, you are free to do it yourself.

Example usage:

  • For text only LLMs: llama-cli -hf voperl/PsychQwen3_v4 --jinja
  • For multimodal models: llama-mtmd-cli -hf voperl/PsychQwen3_v4 --jinja

Available Model files:

  • Qwen3-8B.Q4_K_M.gguf

Ollama

An Ollama Modelfile is included for easy deployment. This was trained 2x faster with Unsloth

Downloads last month
50
GGUF
Model size
8B params
Architecture
qwen3
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Dataset used to train voperl/PsychQwen3_v4