Text Generation
GGUF
qwen36
Mixture of Experts
conversational
multimodal
agent
ollama
heretic
uncensored
reasoning
distillation
Instructions to use FoolDev/Janus-35B-HERETIC with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use FoolDev/Janus-35B-HERETIC with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M # Run inference directly in the terminal: llama cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M # Run inference directly in the terminal: llama cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf FoolDev/Janus-35B-HERETIC:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf FoolDev/Janus-35B-HERETIC:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Use Docker
docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use FoolDev/Janus-35B-HERETIC with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "FoolDev/Janus-35B-HERETIC" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FoolDev/Janus-35B-HERETIC", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
- Ollama
How to use FoolDev/Janus-35B-HERETIC with Ollama:
ollama run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
- Unsloth Desktop
- Pi
How to use FoolDev/Janus-35B-HERETIC with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "FoolDev/Janus-35B-HERETIC:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use FoolDev/Janus-35B-HERETIC with Docker Model Runner:
docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
- Lemonade
How to use FoolDev/Janus-35B-HERETIC with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull FoolDev/Janus-35B-HERETIC:Q4_K_M
Run and chat with the model
lemonade run user.Janus-35B-HERETIC-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use FoolDev/Janus-35B-HERETIC with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default FoolDev/Janus-35B-HERETIC:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use FoolDev/Janus-35B-HERETIC with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "FoolDev/Janus-35B-HERETIC:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Commit History
-
release 0.9.13: the tag check explained itself wrongly and could pass having compared nothing
An adversarial audit of today's 0.9.11 and 0.9.12 turned up three defects in the material they added. All three are fixed here. 1. The trailing-newline explanation was false, and the awk scrape it justified was never needed. Both releases said "this repo's `template` has no trailing newline and `ollama show` adds one". `ollama show` adds nothing; the newline came from the awk in the scrape itself. `ollama show --template <tag>` returns the template verbatim -- 2570 bytes ending `}`, byte-identical to the file, cmp-clean. The snippet and the script now use --template and compare directly. The scrape and the paragraph rationalising it are gone. 2. The check could print "up to date" having compared nothing. `curl -s` is silent on a DNS failure and exits 0 on a 404, so an unreachable reference plus an unreadable tag left both sides empty, and equal. Now `curl -fsS`, with an explicit refusal to compare when either side is empty. 0.9.12 verified only the match…9c0773c
-
release 0.9.0: back to the Qwen 3.6 35B-A3B MoE
Third base change today, and it closes the loop: 0.7.0 restored this MoE, 0.8.0 moved to the dense Qwen 3.8 27B for the newer Qwen generation, 0.9.0 comes back here. Anyone who pulled on 2026-09-18 may hold either model and should pull again. Qwen ships no 3.8 at 35B — its 3.8 line is the dense 27B, Flash-Next and 2.4T-A95B — and every Hub model named Qwen3.8-35B-A3B-* is this same 3.6 35B-A3B with a 3.8 teacher distilled in (empero-ai's Distill and Lord-H4D3ZS's abliterated Coder variant both declare base_model Qwen/Qwen3.6-35B-A3B). So the choice was 35B or 3.8, never both. The non-Heretic 3.8 distill was rejected because shipping it would mean this card could no longer claim an abliterated base. Restored from tag v0.7.0 rather than rewritten, so what ships is the card that passed the pre-publish review: MoE architecture bullets asserted against the blob's own header, moe-routing.svg, the derived KV math, the JANUS-35B artwork, and no chat_template.jinja (the GGUF carries Qwen 3.6'…f48fa07 -
release 0.8.0: back to the dense Qwen 3.8 27B Heretic
0.7.0 reversed the original rebase and shipped the Qwen 3.6 35B-A3B MoE for the sake of the -35B- slug. 0.8.0 moves back to llmfan46's dense Qwen3.8-27B-Ultra-Uncensored-Heretic-Native-MTP-Preserved for the newer base generation. Both moves happened today, so anyone who pulled during 0.7.0 has the MoE and should re-pull. There is no Qwen 3.8 at 35B, so this was 35B or 3.8, not both: Qwen's 3.8 line is the dense 27B, Flash-Next and 2.4T-A95B, and every Hub model named Qwen3.8-35B-A3B-* is the 3.6 35B-A3B with a 3.8 teacher (both candidates declare base_model Qwen/Qwen3.6-35B-A3B). llmfan46 ships no 3.8 35B at all. The dense state is restored from 6708e8f (0.6.5) rather than rewritten, so chat_template.jinja, its render checks, verify_arch.py's dense expectations, the REQUIRED MTP strip, the measured memory figures and the 27B artwork all come back exactly as they were verified. Janus-27B.Q4_K_M.gguf (17,314,193,760 B) is a pointer-only commit — HF still held the object, so nothing was…3931271 -
release 0.7.0: reverse the rebase — back to the Qwen 3.6 35B-A3B MoE
Releases 0.2.0 through 0.6.5 shipped a dense Qwen 3.8 27B under this slug. That rebase is reversed: the model is llmfan46/Qwen3.6-35B-A3B-uncensored-heretic again — ~34.7B total / ~3B active, 40 layers (10 full-attention + 30 Gated DeltaNet), 256 experts with 8 routed per token plus a shared expert, arch qwen35moe. Display name Janus-27B -> Janus-35B, so the Hub slug names the model correctly again. - Blob: Janus-35B-A3B.Q4_K_M.gguf restored as a pointer-only commit (21,233,608,512 B, sha256 b82f9f21…), byte-identical to upstream's Q4_K_M and to what this repo shipped until da34705. HF still holds that LFS object for this repo, so nothing is re-uploaded. No re-quantization, no MTP strip (this quant has no NextN block), no template restamp. - chat_template.jinja is deleted: it was a patch of the dense base's 3.8 template, and Qwen 3.6's own template — which the GGUF carries — has no reasoning_effort, tool_call_format or preserve_reasoning to patch. The card's llama.cpp se…c847255
-
docs: LM Studio engine tested, loader claims narrowed, make help
- LM Studio's CPU runtime (llama.cpp b10519) passed the same 25 llama-server checks on both published blobs. The LM Studio app itself could not be kept off the GPU (it loaded the model with a newer CPU runtime that opened the GPU), so its API layer stays untested; both guarded attempts were stopped and every LM Studio file restored. - The card no longer promises tool calls in Jan, KoboldCpp or llama-cpp-python. - Janus: --chat-template auto removed (llama.cpp has no such mode). - make help describes what make check runs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DUJPW1e74y1meH5UePzvUfa987b5 -
release 0.3.1: JSON tool definitions on Ollama, safer live checks, doc fixes
- Go template renders tool signatures with {{ json .Function }}. Since Ollama 0.14.0 a bare {{ .Function }} printed Go struct syntax inside <tools> (rendered with Ollama's own template package at v0.33.3; unchanged in 0.34.0 and main). check 9 now fails on a bare {{ .Function }}. Template-only: no new GGUF. - live_check.sh: each run uses a fresh subdirectory of WORK_DIR and deletes only that (it used to delete WORK_DIR itself); transport errors are recorded as failures instead of aborting; relative GGUF paths work; a server that fails to start is reported; paths reach Python as arguments. It runs 17 checks. - check_go_template.py accepts the documented replay-all edit and checks the think block by nesting, not offset; make live-check honours GGUF_PATH. - Verified on Ollama 0.34.0 (17/17 on both models). Docs fixes from an adversarial review: the provenance note no longer credits the upstream base with the Teacher field and datasets; check counts, llama.cpp and Ollama …752eff3 -
release 0.3.0: effort line on Ollama, no reasoning replay on llama-server
- Go template reads Ollama's think level and adds the reasoning-effort line chat_template.jinja adds: high/xhigh/max/ultra -> xhigh, low/minimal -> low, medium and unset -> none. Verified live on Ollama 0.33.3 (live_check.sh). - chat_template.jinja ignores preserve_thinking / preserve_reasoning. Recent llama-server builds set preserve_thinking=true by default, so the old opt-in replayed every earlier turn's reasoning on a stock server. The bundled GGUF is restamped in a separate commit. - check.sh check 9 (check_go_template.py) guards Ollama's thinking detection and Qwen's stock condition; scripts/live_check.sh (make live-check) runs live checks in an isolated Ollama. - Docs: standalone llama.cpp results (master a2878d3 and b10760: one call per tool request, effort values forwarded), Ollama live results, CPU-only throughput, and the kwargs-equivalence claim narrowed to the tool-call fix. - CHANGELOG 0.3.0, CITATION.cff 0.3.0. Co-Authored-By: Claude Opus 5 (1M context…2e2b8f8
-
feat(scripts): add check.sh, install-hooks.sh and a Makefile
This repo absorbed a full base-model rebase with no automated checks at all — which is how a Modelfile whose FROM path points at a file not in the repo shipped unnoticed. The sibling has had check.sh the whole time. Ports check.sh (bash -n + shellcheck over every shell script, py_compile over every Python file, the silent-exit footgun grep, and the Modelfile <-> bridge-file sync check), install-hooks.sh, and a Makefile adapted to this repo's script set and the `janus` default tag. The sibling's dot-vs-dash filename guard is deliberately NOT ported: it is inverted here. mradermacher names heretic-ara quants with a dot, so Thanatos bans the dash form; llmfan46 names these quants WITH dashes, so that check would fire on correct filenames. Guard 5 instead blocks a FROM / BUNDLE= / GGUF= path pointing at an unstripped -MTP-Preserved- quant — this repo's real footgun, since the base keeps its MTP head and stock llama.cpp and Ollama cannot load it. rename_arch.py and heal_hf_pull.sh are no…ab7a8a6