GGUF Models for Ollama
Collection
Ready-to-use GGUF quantizations for Ollama, llama.cpp, and local inference. • 8 items • Updated • 1
How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf richardyoung/openbiollm# Run inference directly in the terminal:
llama cli -hf richardyoung/openbiollmBuild from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf richardyoung/openbiollm# Run inference directly in the terminal:
./build/bin/llama cli -hf richardyoung/openbiollmUse Docker
docker model run hf.co/richardyoung/openbiollmQuick Links
OpenBioLLM (Llama3-8B) GGUF
Quantized build of the OpenBioLLM Llama3-8B model for biomedical question answering, packaged for Ollama / llama.cpp runtimes. This release contains the Modelfile exported from the Ollama registry plus the matching GGUF binary.
- Base model:
aaditya/Llama3-OpenBioLLM-8B - Domain: Biomedical / life-sciences QA and reasoning
Variant
| Variant | Size | Blob |
|---|---|---|
latest |
7.95 GB | sha256-1cdfa5be309b5c9206925746aa8fa60601b3a04bc130d85f3257b65121408178 |
Usage with Ollama
ollama create openbiollm -f modelfiles/openbiollm--latest.Modelfile
ollama run openbiollm
Source
Originally published on my Ollama profile: https://ollama.com/richardyoung/openbiollm
Built & maintained by Richard Young · DeepNeuro
- Downloads last month
- 11
Hardware compatibility
Log In to add your hardware
We're not able to determine the quantization variants.
Model tree for richardyoung/openbiollm
Base model
meta-llama/Meta-Llama-3-8B Finetuned
aaditya/Llama3-OpenBioLLM-8B
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf richardyoung/openbiollm# Run inference directly in the terminal: llama cli -hf richardyoung/openbiollm