Kev-4B (GGUF)

Credit: This is a quantized GGUF version of the original Kev-4B created by Jared Palmer. All credit for the model architecture, training, and capabilities goes to the original author.

Model Summary

Kev-4B is a highly efficient decision model. Unlike standard LLMs, it does not generate text. Instead, it reads a document (the state) and a set of typed questions, returning a calibrated probability distribution over the provided options in a single forward pass.

This repository contains the Q4_K_M GGUF quantized version of the model, making it incredibly lightweight and optimized for local CPU/GPU inference via llama.cpp or similar local inference engines.

Intended Use

It is intended for developers building AI agents that need to:

• Classify, route, triage, or check documents.
• Obtain strict mathematical probabilities that can be thresholded (e.g., to send uncertain cases to human review).
• Serve the TypeSafe public System One API (POST /v1/systemone).

Model Details

• Original Developer: Jared Palmer (jaredpalmer/kev)
• Model Type: Decision model (causal language-model backbone run prefill-only, with a pointer head over the options)
• Format: GGUF (Q4_K_M quantization)
• Backbone: Qwen3.5-4B-Base (32 layers, frozen)
• Context Length: States of up to 65,536 tokens, plus at least 8,192 tokens per question.
• Languages: English
• License: Apache-2.0