Publish Jeeves MLX 4bit (CPU diagnostics; not benchmarked)

6f4220a verified 25 days ago

6.65 kB

---
license: apache-2.0
language:
- en
base_model: PostHog/jeeves
library_name: mlx
pipeline_tag: text-classification
tags:
- mlx
- qwen3.5
- decision
- unofficial-port
- experimental
- 4bit
---
# Jeeves-9B MLX — 4bit

Unofficial CPU-converted MLX packaging of **[PostHog/jeeves](https://huggingface.co/PostHog/jeeves)**, published by **[cowWhySo](https://huggingface.co/cowWhySo)**. This is not a PostHog or Apple release, a newly trained model, or a benchmark-certified port.

**Original project: [PostHog/jeeves on GitHub](https://github.com/PostHog/jeeves)**  
**Original model: [PostHog/jeeves on Hugging Face](https://huggingface.co/PostHog/jeeves)**

> **Validation: `converted_and_small_cpu_diagnostics_passed_not_benchmarked`.** Read [validation.json](validation.json) before relying on the package. Apple Metal execution, task accuracy, post-quantization calibration, and representative latency remain unmeasured. Upstream benchmark scores and CUDA speed claims do not transfer to this conversion.
## Available packages
One repository; one complete package per branch. `main` contains **4-bit**, not BF16. Other variants are branches, not subfolders. Downloading one revision does not download the others.
| Variant | Revision | Original package size on Drive |
| --- | --- | ---: |
| 4bit | [`main`](https://huggingface.co/cowWhySo/jeeves-mlx/tree/main) | 4.73 GiB |
| 8bit | [`8bit`](https://huggingface.co/cowWhySo/jeeves-mlx/tree/8bit) | 8.90 GiB |
| bf16 | [`bf16`](https://huggingface.co/cowWhySo/jeeves-mlx/tree/bf16) | 16.71 GiB |
Sizes are actual original package bytes divided by 2^30 (GiB), including auxiliary files. Publishing adds small documentation/manifest files. These are **disk sizes**, not measured peak RAM or unified-memory requirements. This README describes the `main` revision.
## What is preserved
The Qwen3.5-9B backbone uses **4-bit MLX affine quantization, group size 64**. The original pointer head is retained as `head.pt` and exported without changing its FP32 values as `head.safetensors`. The released fitted temperature is **1.85892808437347**, retained in `export.json`; it has **not** been refitted for quantization. Exact-format helpers, the tokenizer, original licenses, source records, and diagnostic fixtures accompany the weights.
Jeeves uses a custom decision head and prompt/readout format, not an ordinary text-generation pipeline. Use the included `run_jeeves.py` for decisions; `mlx_lm.generate` alone does not implement Jeeves scoring. See [BUILD.md](BUILD.md) for build details and [provenance/original_package_README.md](provenance/original_package_README.md) for the original conversion notes.
## Download and try on Apple Silicon
These are **instructions for an unmeasured Metal path**, not a claim that this release has been tested on a Mac. Review the included Python source before running it. Each variant should have its own local directory; do not overlay variants into one directory.
```bash
python3 -m venv .venv
source .venv/bin/activate
python -m pip install "huggingface_hub==1.33.0"
hf download cowWhySo/jeeves-mlx --revision main --local-dir jeeves-mlx-4bit
cd jeeves-mlx-4bit
python -m pip install -r requirements-mac.txt
python run_jeeves.py --model . --request example_request.json --device gpu --think-tokens 0
```
To select a different variant, replace the revision and directory with `main` / `jeeves-mlx-4bit`, `8bit` / `jeeves-mlx-8bit`, or `bf16` / `jeeves-mlx-bf16`. Pin a release commit SHA instead of a branch for reproducible downstream experiments.
The runner accepts `state` and a `questions` mapping with `noul`, `choice`, or `score` types. It returns option probabilities under its own `results` schema; it is **not wire-compatible** with the upstream `/v1/systemone` HTTP service.

An optional, **unvalidated greedy-thinking path** exists:

```bash
python run_jeeves.py --model . --request example_request.json --device gpu --think-tokens 128
```

The reference runner processes questions independently, uses a separate cache per question, and re-prefills the full sequence for decision readout after optional reasoning. It does not port the speculative diffusion drafters, CUDA graphs, FP8 serving, parallel-question engine, or adaptive no-thinking threshold.

## Evidence and limits

The stored format/head checks passed for 4 format cases. The synthetic-hidden-state pointer-head comparison reported maximum probability error 0; it is **not** a backbone comparison. The stored no-thinking CPU smoke test contains **1 case(s)**. The BF16 backbone comparison is explicitly a small-fixture diagnostic with heuristic tolerances, not general equivalence.

On the recorded 1-case diagnostic, the maximum option-probability difference from MLX BF16 was 0.016297996. This is not an accuracy or calibration estimate.

No task accuracy, calibration benchmark, Metal speed, quantized quality ranking, or validated reasoning performance is claimed. CPU smoke elapsed times in the evidence are diagnostics, not representative serving benchmarks. Retaining the original temperature is not evidence of calibration on a different numerical path.

## Build provenance

- Model source: [`PostHog/jeeves@8622b7d1652a9dcb8629486b84dce9e8d690c5cd`](https://huggingface.co/PostHog/jeeves/tree/8622b7d1652a9dcb8629486b84dce9e8d690c5cd).
- Jeeves code: [`f04ec5567301450dcaae0210dd54deeb4f647f87`](https://github.com/PostHog/jeeves/tree/f04ec5567301450dcaae0210dd54deeb4f647f87).
- MLX-LM code: [`94cdcae13b266c337bcaca09b97b9c5a9c0e2cde`](https://github.com/ml-explore/mlx-lm/tree/94cdcae13b266c337bcaca09b97b9c5a9c0e2cde).
- Conversion MLX requirement: `mlx[cpu]==0.32.3`; original build identity: `c4d0ea30e584dbf0d1e672c59c2e00036cb6a1a7025833ef49c01c63432b18fa`.

[PACKAGE_MANIFEST.json](PACKAGE_MANIFEST.json) hashes the published payload except itself. [provenance/source_PACKAGE_MANIFEST.json](provenance/source_PACKAGE_MANIFEST.json) is the unmodified original manifest; its paths describe the original pre-publication package, not the newly documented payload. [RELEASE.json](RELEASE.json) distinguishes conversion evidence from packaging checks. Original `validation.json` is unchanged.

## License and attribution

Model weights follow the original **Apache-2.0** license retained in [LICENSE](LICENSE). Upstream Jeeves code is **MIT** licensed; its notice is retained in [LICENSE_JEEVES_CODE](LICENSE_JEEVES_CODE). Respect any additional notices in the included files. Credits for the original model, training, calibration, and serving research belong to PostHog and the upstream authors; this repository provides an unofficial MLX conversion/packaging.

	---
	license: apache-2.0
	language:
	- en
	base_model: PostHog/jeeves
	library_name: mlx
	pipeline_tag: text-classification
	tags:
	- mlx
	- qwen3.5
	- decision
	- unofficial-port
	- experimental
	- 4bit
	---

	# Jeeves-9B MLX — 4bit

	Unofficial CPU-converted MLX packaging of [PostHog/jeeves](https://huggingface.co/PostHog/jeeves), published by [cowWhySo](https://huggingface.co/cowWhySo). This is not a PostHog or Apple release, a newly trained model, or a benchmark-certified port.

	Original project: [PostHog/jeeves on GitHub](https://github.com/PostHog/jeeves)
	Original model: [PostHog/jeeves on Hugging Face](https://huggingface.co/PostHog/jeeves)

	> Validation: `converted_and_small_cpu_diagnostics_passed_not_benchmarked`. Read [validation.json](validation.json) before relying on the package. Apple Metal execution, task accuracy, post-quantization calibration, and representative latency remain unmeasured. Upstream benchmark scores and CUDA speed claims do not transfer to this conversion.

	## Available packages

	One repository; one complete package per branch. `main` contains 4-bit, not BF16. Other variants are branches, not subfolders. Downloading one revision does not download the others.

	\| Variant \| Revision \| Original package size on Drive \|
	\| --- \| --- \| ---: \|
	\| 4bit \| [`main`](https://huggingface.co/cowWhySo/jeeves-mlx/tree/main) \| 4.73 GiB \|
	\| 8bit \| [`8bit`](https://huggingface.co/cowWhySo/jeeves-mlx/tree/8bit) \| 8.90 GiB \|
	\| bf16 \| [`bf16`](https://huggingface.co/cowWhySo/jeeves-mlx/tree/bf16) \| 16.71 GiB \|

	Sizes are actual original package bytes divided by 2^30 (GiB), including auxiliary files. Publishing adds small documentation/manifest files. These are disk sizes, not measured peak RAM or unified-memory requirements. This README describes the `main` revision.

	## What is preserved

	The Qwen3.5-9B backbone uses 4-bit MLX affine quantization, group size 64. The original pointer head is retained as `head.pt` and exported without changing its FP32 values as `head.safetensors`. The released fitted temperature is 1.85892808437347, retained in `export.json`; it has not been refitted for quantization. Exact-format helpers, the tokenizer, original licenses, source records, and diagnostic fixtures accompany the weights.

	Jeeves uses a custom decision head and prompt/readout format, not an ordinary text-generation pipeline. Use the included `run_jeeves.py` for decisions; `mlx_lm.generate` alone does not implement Jeeves scoring. See [BUILD.md](BUILD.md) for build details and [provenance/original_package_README.md](provenance/original_package_README.md) for the original conversion notes.

	## Download and try on Apple Silicon

	These are instructions for an unmeasured Metal path, not a claim that this release has been tested on a Mac. Review the included Python source before running it. Each variant should have its own local directory; do not overlay variants into one directory.

	```bash
	python3 -m venv .venv
	source .venv/bin/activate
	python -m pip install "huggingface_hub==1.33.0"
	hf download cowWhySo/jeeves-mlx --revision main --local-dir jeeves-mlx-4bit
	cd jeeves-mlx-4bit
	python -m pip install -r requirements-mac.txt
	python run_jeeves.py --model . --request example_request.json --device gpu --think-tokens 0
	```

	To select a different variant, replace the revision and directory with `main` / `jeeves-mlx-4bit`, `8bit` / `jeeves-mlx-8bit`, or `bf16` / `jeeves-mlx-bf16`. Pin a release commit SHA instead of a branch for reproducible downstream experiments.

	The runner accepts `state` and a `questions` mapping with `noul`, `choice`, or `score` types. It returns option probabilities under its own `results` schema; it is not wire-compatible with the upstream `/v1/systemone` HTTP service.

	An optional, unvalidated greedy-thinking path exists:

	```bash
	python run_jeeves.py --model . --request example_request.json --device gpu --think-tokens 128
	```

	The reference runner processes questions independently, uses a separate cache per question, and re-prefills the full sequence for decision readout after optional reasoning. It does not port the speculative diffusion drafters, CUDA graphs, FP8 serving, parallel-question engine, or adaptive no-thinking threshold.

	## Evidence and limits

	The stored format/head checks passed for 4 format cases. The synthetic-hidden-state pointer-head comparison reported maximum probability error 0; it is not a backbone comparison. The stored no-thinking CPU smoke test contains 1 case(s). The BF16 backbone comparison is explicitly a small-fixture diagnostic with heuristic tolerances, not general equivalence.

	On the recorded 1-case diagnostic, the maximum option-probability difference from MLX BF16 was 0.016297996. This is not an accuracy or calibration estimate.

	No task accuracy, calibration benchmark, Metal speed, quantized quality ranking, or validated reasoning performance is claimed. CPU smoke elapsed times in the evidence are diagnostics, not representative serving benchmarks. Retaining the original temperature is not evidence of calibration on a different numerical path.

	## Build provenance

	- Model source: [`PostHog/jeeves@8622b7d1652a9dcb8629486b84dce9e8d690c5cd`](https://huggingface.co/PostHog/jeeves/tree/8622b7d1652a9dcb8629486b84dce9e8d690c5cd).
	- Jeeves code: [`f04ec5567301450dcaae0210dd54deeb4f647f87`](https://github.com/PostHog/jeeves/tree/f04ec5567301450dcaae0210dd54deeb4f647f87).
	- MLX-LM code: [`94cdcae13b266c337bcaca09b97b9c5a9c0e2cde`](https://github.com/ml-explore/mlx-lm/tree/94cdcae13b266c337bcaca09b97b9c5a9c0e2cde).
	- Conversion MLX requirement: `mlx[cpu]==0.32.3`; original build identity: `c4d0ea30e584dbf0d1e672c59c2e00036cb6a1a7025833ef49c01c63432b18fa`.

	[PACKAGE_MANIFEST.json](PACKAGE_MANIFEST.json) hashes the published payload except itself. [provenance/source_PACKAGE_MANIFEST.json](provenance/source_PACKAGE_MANIFEST.json) is the unmodified original manifest; its paths describe the original pre-publication package, not the newly documented payload. [RELEASE.json](RELEASE.json) distinguishes conversion evidence from packaging checks. Original `validation.json` is unchanged.

	## License and attribution

	Model weights follow the original Apache-2.0 license retained in [LICENSE](LICENSE). Upstream Jeeves code is MIT licensed; its notice is retained in [LICENSE_JEEVES_CODE](LICENSE_JEEVES_CODE). Respect any additional notices in the included files. Credits for the original model, training, calibration, and serving research belong to PostHog and the upstream authors; this repository provides an unofficial MLX conversion/packaging.