Instructions to use FoolDev/Janus-35B-HERETIC with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use FoolDev/Janus-35B-HERETIC with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Use Docker

docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M

LM Studio
Jan

vLLM

How to use FoolDev/Janus-35B-HERETIC with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FoolDev/Janus-35B-HERETIC"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FoolDev/Janus-35B-HERETIC",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M

Ollama
How to use FoolDev/Janus-35B-HERETIC with Ollama:
```
ollama run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
```
Unsloth Desktop

How to use FoolDev/Janus-35B-HERETIC with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "FoolDev/Janus-35B-HERETIC:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use FoolDev/Janus-35B-HERETIC with Docker Model Runner:
```
docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
```

Lemonade

How to use FoolDev/Janus-35B-HERETIC with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull FoolDev/Janus-35B-HERETIC:Q4_K_M

Run and chat with the model

lemonade run user.Janus-35B-HERETIC-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use FoolDev/Janus-35B-HERETIC with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default FoolDev/Janus-35B-HERETIC:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use FoolDev/Janus-35B-HERETIC with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "FoolDev/Janus-35B-HERETIC:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Janus-35B-HERETIC / Makefile

FoolDev

Claude Opus 5 (1M context)

release 0.9.13: the tag check explained itself wrongly and could pass having compared nothing

9c0773c 30 days ago

Raw History Blame Contribute Delete

3.29 kB

# Janus-35B convenience Makefile.
#
# All work is delegated to scripts/* — this file just gives common
# operations short, discoverable names.
#
# Variables you can override on the command line:
#   QUANT     GGUF quant suffix       (default: Q4_K_M)
#   TAG       Ollama model tag        (default: janus)
#   GGUF_PATH path to existing GGUF   (skip the download)
#   GGUF      GGUF for live-check     (default: GGUF_PATH, else the bundled file)
#   MODEL     model tag for smoke     (default: $(TAG))
#
# Examples:
#   make build                          # Q4_K_M from the Heretic GGUF repo (the MTP strip no-ops on these quants)
#   make build QUANT=Q5_K_M             # higher-quality quant
#   make build GGUF_PATH=~/models/Qwen3.6-35B-A3B-uncensored-heretic-Q4_K_M.gguf
#   make load-bundle                    # this repo's bundled GGUF -> local Ollama tag (smudge LFS if needed)
#   make smoke
#   make check
#   make clean

QUANT ?= Q4_K_M
TAG   ?= janus
MODEL ?= $(TAG)
CTX   ?= 4096

.DEFAULT_GOAL := help

PRECISION ?= BF16

.PHONY: help build load-bundle cap-ctx smoke smoke-tools bench check hooks mmproj verify-arch clean live-check check-tag

help:  ## Show this help.
	@awk 'BEGIN {FS = ":.*##"; printf "Targets:\n"} /^[a-zA-Z_-]+:.*?##/ { printf "  \033[36m%-12s\033[0m %s\n", $$1, $$2 }' $(MAKEFILE_LIST)
	@echo
	@echo "Current settings:"
	@echo "  QUANT=$(QUANT) TAG=$(TAG) PRECISION=$(PRECISION)"
ifdef GGUF_PATH
	@echo "  GGUF_PATH=$(GGUF_PATH)"
endif

build:  ## Download a Heretic GGUF, run the MTP strip (a no-op on these quants), and run 'ollama create'.
	GGUF_PATH=$(GGUF_PATH) TAG=$(TAG) ./scripts/build.sh $(QUANT)

load-bundle:  ## Load THIS repo's bundled GGUF into a local Ollama tag (smudge LFS + ollama create).
	TAG=$(TAG) ./scripts/load_bundle.sh

cap-ctx:  ## Bake a small-num_ctx local tag for OpenAI /v1 clients (CTX default 4096).
	CTX=$(CTX) TAG=$(TAG) ./scripts/cap_ctx.sh

smoke:  ## Verify the model is reachable and round-trips.
	MODEL=$(MODEL) ./scripts/smoke_test.sh

smoke-tools:  ## smoke + end-to-end tool-call round-trip (~10s extra).
	MODEL=$(MODEL) TOOLS_TEST=1 ./scripts/smoke_test.sh

bench:  ## Measure tok/s using Ollama's eval timing (3 prompts).
	MODEL=$(MODEL) ./scripts/bench.sh

mmproj:  ## Fetch the vision projector for llama.cpp (Ollama vision is broken upstream).
	./scripts/fetch_vision.sh $(PRECISION)

check:  ## Lint, bridge-file sync, the Go-template guard, and that the Modelfile's FROM is the GGUF this repo ships.
	./scripts/check.sh

live-check:  ## Live end-to-end checks in an isolated Ollama (GGUF=path or GGUF_PATH=path; tens of minutes on CPU).
	./scripts/live_check.sh "$(or $(GGUF),$(GGUF_PATH))"

check-tag:  ## Check an installed Ollama tag still carries this repo's template (TAG=name).
	./scripts/check_tag_sync.sh $(TAG)

verify-arch:  ## Cross-check README "Architecture" bullets against GGUF metadata.
	./scripts/verify_arch.py

hooks:  ## Install scripts/check.sh as the git pre-commit hook.
	./scripts/install-hooks.sh

clean:  ## Remove local GGUF copies and ephemeral caches in this repo.
	@echo "[*] removing local GGUFs and ephemeral caches in $$PWD"
	@rm -f ./Qwen3.6-35B-A3B-uncensored-*.gguf ./mmproj-*.gguf ./*.stripped.gguf
	@rm -rf ./.cache __pycache__ examples/__pycache__
	@echo "[+] clean"

	# Janus-35B convenience Makefile.
	#
	# All work is delegated to scripts/* — this file just gives common
	# operations short, discoverable names.
	#
	# Variables you can override on the command line:
	# QUANT GGUF quant suffix (default: Q4_K_M)
	# TAG Ollama model tag (default: janus)
	# GGUF_PATH path to existing GGUF (skip the download)
	# GGUF GGUF for live-check (default: GGUF_PATH, else the bundled file)
	# MODEL model tag for smoke (default: $(TAG))
	#
	# Examples:
	# make build # Q4_K_M from the Heretic GGUF repo (the MTP strip no-ops on these quants)
	# make build QUANT=Q5_K_M # higher-quality quant
	# make build GGUF_PATH=~/models/Qwen3.6-35B-A3B-uncensored-heretic-Q4_K_M.gguf
	# make load-bundle # this repo's bundled GGUF -> local Ollama tag (smudge LFS if needed)
	# make smoke
	# make check
	# make clean

	QUANT ?= Q4_K_M
	TAG ?= janus
	MODEL ?= $(TAG)
	CTX ?= 4096

	.DEFAULT_GOAL := help

	PRECISION ?= BF16

	.PHONY: help build load-bundle cap-ctx smoke smoke-tools bench check hooks mmproj verify-arch clean live-check check-tag

	help: ## Show this help.
	@awk 'BEGIN {FS = ":.##"; printf "Targets:\n"} /^[a-zA-Z_-]+:.?##/ { printf " \033[36m%-12s\033[0m %s\n", $$1, $$2 }' $(MAKEFILE_LIST)
	@echo
	@echo "Current settings:"
	@echo " QUANT=$(QUANT) TAG=$(TAG) PRECISION=$(PRECISION)"
	ifdef GGUF_PATH
	@echo " GGUF_PATH=$(GGUF_PATH)"
	endif

	build: ## Download a Heretic GGUF, run the MTP strip (a no-op on these quants), and run 'ollama create'.
	GGUF_PATH=$(GGUF_PATH) TAG=$(TAG) ./scripts/build.sh $(QUANT)

	load-bundle: ## Load THIS repo's bundled GGUF into a local Ollama tag (smudge LFS + ollama create).
	TAG=$(TAG) ./scripts/load_bundle.sh

	cap-ctx: ## Bake a small-num_ctx local tag for OpenAI /v1 clients (CTX default 4096).
	CTX=$(CTX) TAG=$(TAG) ./scripts/cap_ctx.sh

	smoke: ## Verify the model is reachable and round-trips.
	MODEL=$(MODEL) ./scripts/smoke_test.sh

	smoke-tools: ## smoke + end-to-end tool-call round-trip (~10s extra).
	MODEL=$(MODEL) TOOLS_TEST=1 ./scripts/smoke_test.sh

	bench: ## Measure tok/s using Ollama's eval timing (3 prompts).
	MODEL=$(MODEL) ./scripts/bench.sh

	mmproj: ## Fetch the vision projector for llama.cpp (Ollama vision is broken upstream).
	./scripts/fetch_vision.sh $(PRECISION)

	check: ## Lint, bridge-file sync, the Go-template guard, and that the Modelfile's FROM is the GGUF this repo ships.
	./scripts/check.sh

	live-check: ## Live end-to-end checks in an isolated Ollama (GGUF=path or GGUF_PATH=path; tens of minutes on CPU).
	./scripts/live_check.sh "$(or $(GGUF),$(GGUF_PATH))"

	check-tag: ## Check an installed Ollama tag still carries this repo's template (TAG=name).
	./scripts/check_tag_sync.sh $(TAG)

	verify-arch: ## Cross-check README "Architecture" bullets against GGUF metadata.
	./scripts/verify_arch.py

	hooks: ## Install scripts/check.sh as the git pre-commit hook.
	./scripts/install-hooks.sh

	clean: ## Remove local GGUF copies and ephemeral caches in this repo.
	@echo "[*] removing local GGUFs and ephemeral caches in $$PWD"
	@rm -f ./Qwen3.6-35B-A3B-uncensored-.gguf ./mmproj-.gguf ./*.stripped.gguf
	@rm -rf ./.cache __pycache__ examples/__pycache__
	@echo "[+] clean"