Update via HuggingBot

README.md

@@ -1,3 +1,14 @@
+---
+title: OpenJev · Zero-GPU CPU Deploy Kit
+emoji: ⚖️
+colorFrom: green
+colorTo: gray
+sdk: static
+pinned: true
+license: apache-2.0
+short_description: Zero-GPU CPU deploy kit for the OpenJev decision API
+---
+
 # OpenJev — zero-GPU CPU deploy kit
 
 Runs the [OpenJev](https://huggingface.co/openjev/openjev) decision API
@@ -15,41 +26,30 @@ output position — by pointing the **official, unmodified** `openjev-server` at
 | latency | ~125 ms | seconds (see [Benchmarks](#benchmarks)) |
 | surface | `/v1/systemone` + ops | identical (official server, unchanged) |
 
+> 👀 The live index page is at **[https://broadfield-openjev-cpu-deploy.hf.space](https://broadfield-openjev-cpu-deploy.hf.space)** — this README is the machine-readable mirror. Every file below is fetchable raw: `https://huggingface.co/spaces/broadfield/openjev-cpu-deploy/raw/main/<path>`.
+
 ## Why this works (no code hacks)
 
-`openjev-server`'s `vllm` backend is really *"any OpenAI-compatible server that
-returns logprobs"*:
+`openjev-server`'s `vllm` backend is really *"any OpenAI-compatible server that returns logprobs"*:
 
-- it sends standard `/chat/completions` JSON (`logprobs`, `top_logprobs`,
-  `allowed_token_ids`, `max_tokens: 1`, `temperature: 0`);
-- **exact readout** uses vLLM's `logprob_token_ids = [ids]` extension →
-  llama.cpp doesn't have it, so the backend's documented fallback kicks in:
-  top-K logprobs are matched **by label text** instead (see
-  `openjev_server/backends/vllm.py` → `_scores`, "old protocol").
+- it sends standard `/chat/completions` JSON (`logprobs`, `top_logprobs`, `allowed_token_ids`, `max_tokens: 1`, `temperature: 0`);
+- **exact readout** uses vLLM's `logprob_token_ids = [ids]` extension → llama.cpp doesn't have it, so the backend's documented fallback kicks in: top-K logprobs are matched **by label text** instead (see `openjev_server/backends/vllm.py` → `_scores`, "old protocol").
 
-That single fallback is the whole trick: the official code path runs unmodified,
-and `--force` only relaxes the startup probe's letter-mass check if llama.cpp's
-text-matched logprobs ever look borderline for your quant.
+That single fallback is the whole trick: the official code path runs unmodified, and `--force` only relaxes the startup probe's letter-mass check if llama.cpp's text-matched logprobs ever look borderline for your quant.
 
 ## What you get
 
 - `POST /v1/systemone` — one request, any mix of `choice` / `noul` / `score`
 - `POST /v1/prewarm`, `POST /v1/chat/completions` (passthrough)
-- `GET /v1/version` · `GET /healthz` · `GET /readyz` ·
-  `GET /metrics` (Prometheus) · `GET /docs` (OpenAPI)
+- `GET /v1/version` · `GET /healthz` · `GET /readyz` · `GET /metrics` (Prometheus) · `GET /docs` (OpenAPI)
 - optional bearer auth via `OPENJEV_TOKEN`
 - Persistent model storage (HF Spaces) — the 16.2 GB download survives restarts
 
 ## Deploy on Hugging Face (zero GPU, free)
 
-1. **Duplicate the runtime Space** (needs a free HF account):
-   <https://huggingface.co/spaces/broadfield/openjev-cpu-run?duplicate=true>
-   - SDK: **Docker** · hardware: **CPU basic (free)** · persistent storage:
-     **30 GB** ($0.50/mo, required for `/data`)
-2. Wait for the build. The Docker image compiles llama.cpp CPU + installs
-   openjev-server and, on first boot, downloads the Q4 GGUF (~16.2 GB) into
-   `/data`. **Allow ~20–50 min on the first cold boot;** subsequent restarts
-   load straight from persistent storage.
+1. **Duplicate the runtime Space** (needs a free HF account): <https://huggingface.co/spaces/broadfield/openjev-cpu-run?duplicate=true>
+   - SDK: **Docker** · hardware: **CPU basic (free)** · persistent storage: **30 GB** ($0.50/mo, required for `/data`)
+2. Wait for the build. The Docker image compiles llama.cpp CPU + installs openjev-server and, on first boot, downloads the Q4 GGUF (~16.2 GB) into `/data`. **Allow ~20–50 min on the first cold boot;** subsequent restarts load straight from persistent storage.
 3. Try the API. By default the Space is public:
 
 ```bash
@@ -83,12 +83,7 @@ curl -s "$BASE/v1/systemone" -H 'Content-Type: application/json' -d '{
 }
 ```
 
-> **Latency honesty right up front** — a 27B model on a shared free CPU is *not*
-> 125 ms. Expect **~2–6 s per question** (single reader) on a 16-core Space with
-> the Q4 quant. That's a real API — just not a high-QPS one. The reason this is
-> still interesting: **a decision API only ever generates 1 token**, so the CPU
-> is the *only* slow part, and it stays predictable. Raise `LLAMA_PARALLEL`,
-> lower `LLAMA_CTX`, or use a different quant if your workload wants less of it.
+> **Latency honesty right up front** — a 27B model on a shared free CPU is *not* 125 ms. Expect **~2–6 s per question** (single reader) on a 16-core Space with the Q4 quant. That's a real API — just not a high-QPS one. The reason this is still interesting: **a decision API only ever generates 1 token**, so the CPU is the *only* slow part, and it stays predictable. Raise `LLAMA_PARALLEL`, lower `LLAMA_CTX`, or use a different quant if your workload wants less of it.
 
 ## Deploy locally (or on any VPS)
 
@@ -124,8 +119,7 @@ docker run -d -p 7860:7860 -v openjev-data:/data -e MODEL_FILE=OpenJev-Q4_K_M.gg
 
 ## Benchmarks
 
-Expected on a **free 16 vCPU CPU Space**, `OpenJev-Q4_K_M.gguf`
-(27B, Qwen3-based → ~8-bit KV cache in the llama.cpp fork, 16.2 GB RAM):
+Expected on a **free 16 vCPU CPU Space**, `OpenJev-Q4_K_M.gguf` (27B, Qwen3-based → ~8-bit KV cache in the llama.cpp fork, 16.2 GB RAM):
 
 | quant | file size | SPE (tokens/s, 8 threads) | RAM | 1-question latency* |
 |---|---|---|---|---|
@@ -133,13 +127,9 @@ Expected on a **free 16 vCPU CPU Space**, `OpenJev-Q4_K_M.gguf`
 | Q5_K_M | 18.8 GB | 1.5–3 | ~20 GB | ~3–8 s |
 | Q6_K | 21.6 GB | 1.5–3 | ~23 GB | ~3–8 s |
 
-\* single question ≈ one forward pass: `(prompt_tokens + 1) / SPE`. A 240-token
-state ⇒ ~4 s @ 3 SPE. Latency is directly linear in prompt length.
+\* single question ≈ one forward pass: `(prompt_tokens + 1) / SPE`. A 240-token state ⇒ ~4 s @ 3 SPE. Latency is directly linear in prompt length.
 
-**The decision-API twist:** because `max_tokens: 1`, a *pipeline* of many
-questions on one state shares ~all of the prefill. For batch agents, batch the
-questions into one `/v1/systemone` call — every extra answer after the first is
-nearly free:
+**The decision-API twist:** because `max_tokens: 1`, a *pipeline* of many questions on one state shares ~all of the prefill. For batch agents, batch the questions into one `/v1/systemone` call — every extra answer after the first is nearly free:
 
 | questions in one call | est. per-call latency @ 3 SPE |
 |---|---|
@@ -180,20 +170,21 @@ const r = await fetch(`${BASE}/v1/systemone`, {
 const { answers } = await r.json();
 ```
 
-## Layout
+## Files in this kit (raw links)
 
-```
-Dockerfile          # multi-stage: llama.cpp CPU + official openjev-server
-entrypoint.sh       # GGUF → /data, llama-server, then openjev serve
-docker-compose.yml  # local/VPS one-liner
-bench.mjs           # throughput + latency smoke bench against a running endpoint
-runtime-baseline.md # the exact Docker Space spec we recommend
-README.md           # you are here
-```
+| path | role | raw |
+|---|---|---|
+| `index.html` | live landing page with all instructions | [raw](https://huggingface.co/spaces/broadfield/openjev-cpu-deploy/raw/main/index.html) |
+| `Dockerfile` | multi-stage: llama.cpp CPU + official openjev-server | [raw](https://huggingface.co/spaces/broadfield/openjev-cpu-deploy/raw/main/Dockerfile) |
+| `entrypoint.sh` | GGUF → `/data`, llama-server, then `openjev serve` | [raw](https://huggingface.co/spaces/broadfield/openjev-cpu-deploy/raw/main/entrypoint.sh) |
+| `docker-compose.yml` | local/VPS one-liner | [raw](https://huggingface.co/spaces/broadfield/openjev-cpu-deploy/raw/main/docker-compose.yml) |
+| `bench.mjs` | latency/throughput smoke bench | [raw](https://huggingface.co/spaces/broadfield/openjev-cpu-deploy/raw/main/bench.mjs) |
+| `runtime-baseline.md` | the exact runtime Space spec | [raw](https://huggingface.co/spaces/broadfield/openjev-cpu-deploy/raw/main/runtime-baseline.md) |
+
+Everything is also mirrored (cloneable) in the dataset repo [`broadfield/openjev-cpu-deploy-kit`](https://huggingface.co/datasets/broadfield/openjev-cpu-deploy-kit).
 
 ## License & credit
 
-- Model weights: **CC BY-NC 4.0** — OpenJev is an independent project, not
-  affiliated with TypeSafe (Jev is their product).
+- Model weights: **CC BY-NC 4.0** — OpenJev is an independent project, not affiliated with TypeSafe (Jev is their product).
 - `openjev-server`: Apache 2.0.
 - This kit: Apache 2.0. llama.cpp: MIT.