How to use from
llama.cpp
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf tinyopsec/Meta-Llama-3-8B-Instruct-Jailbroken-GGUF:
# Run inference directly in the terminal:
llama cli -hf tinyopsec/Meta-Llama-3-8B-Instruct-Jailbroken-GGUF:
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf tinyopsec/Meta-Llama-3-8B-Instruct-Jailbroken-GGUF:
# Run inference directly in the terminal:
llama cli -hf tinyopsec/Meta-Llama-3-8B-Instruct-Jailbroken-GGUF:
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf tinyopsec/Meta-Llama-3-8B-Instruct-Jailbroken-GGUF:
# Run inference directly in the terminal:
./build/bin/llama cli -hf tinyopsec/Meta-Llama-3-8B-Instruct-Jailbroken-GGUF:
Use Docker
docker model run hf.co/tinyopsec/Meta-Llama-3-8B-Instruct-Jailbroken-GGUF:
Quick Links

Meta-Llama-3-8B-Instruct-Jailbroken β€” GGUF

GGUF quantizations of cooperleong00/Meta-Llama-3-8B-Instruct-Jailbroken, a Llama-3-8B-Instruct model jailbroken via weight orthogonalization (Arditi et al., 2024), produced with llama.cpp.

Available files

File Quantization Description
model-F16.gguf F16 Full precision (16.1 GB)
model-Q8_0.gguf Q8_0 8-bit
model-Q6_K.gguf Q6_K 6-bit
model-Q5_K_M.gguf Q5_K_M 5-bit medium
model-Q5_K_S.gguf Q5_K_S 5-bit small
model-Q5_0.gguf Q5_0 5-bit
model-Q4_K_M.gguf Q4_K_M 4-bit medium (recommended)
model-Q4_K_S.gguf Q4_K_S 4-bit small
model-Q4_0.gguf Q4_0 4-bit
model-Q3_K_L.gguf Q3_K_L 3-bit large
model-Q3_K_M.gguf Q3_K_M 3-bit medium
model-Q3_K_S.gguf Q3_K_S 3-bit small
model-Q2_K.gguf Q2_K 2-bit

Usage

With llama.cpp:

./llama-cli -m model-Q4_K_M.gguf -p "Your prompt" -n 256

Or any GGUF-compatible runtime (Ollama, LM Studio, koboldcpp, etc.).

License

Meta Llama 3 Community License. Use of this derivative is bound by the original model's license. Built with Meta Llama 3.

Disclaimer

Released strictly for academic research in AI safety and model alignment. Author bears no responsibility for misuse. Comply with all applicable laws and ethical guidelines.

Downloads last month
770
GGUF
Model size
8B params
Architecture
llama
Hardware compatibility
Log In to add your hardware

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for tinyopsec/Meta-Llama-3-8B-Instruct-Jailbroken-GGUF

Collection including tinyopsec/Meta-Llama-3-8B-Instruct-Jailbroken-GGUF