Text Classification
PEFT
Safetensors
MLX
decision-model
system-one
jev
agent-guard
tomato-egg-problem
apple-silicon
Instructions to use fancyboi999/ockev with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use fancyboi999/ockev with PEFT:
from peft import PeftModel from transformers import AutoModel base_model = AutoModel.from_pretrained("Qwen/Qwen2.5-1.5B") model = PeftModel.from_pretrained(base_model, "fancyboi999/ockev") - MLX
How to use fancyboi999/ockev with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download fancyboi999/ockev --local-dir ockev
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
docs: enrich model card with systems comparison and benchmark breakdown
Browse files
README.md
CHANGED
|
@@ -9,17 +9,65 @@ tags:
|
|
| 9 |
- jev
|
| 10 |
- agent-guard
|
| 11 |
- tomato-egg-problem
|
|
|
|
|
|
|
| 12 |
---
|
| 13 |
|
| 14 |
-
|
| 15 |
|
| 16 |
-
|
| 17 |
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
## Quickstart
|
| 25 |
|
|
@@ -27,3 +75,16 @@ Ship the result, not the conversation. Ending the "Tomato-Egg Problem" in AI age
|
|
| 27 |
pip install ockev
|
| 28 |
ockev check path/to/deliverable.md
|
| 29 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
- jev
|
| 10 |
- agent-guard
|
| 11 |
- tomato-egg-problem
|
| 12 |
+
- mlx
|
| 13 |
+
- apple-silicon
|
| 14 |
---
|
| 15 |
|
| 16 |
+
<div align="center">
|
| 17 |
|
| 18 |
+
# Ockev-1.5B: 35ms Discriminative Decision Engine for AI Agent Deliverables
|
| 19 |
|
| 20 |
+
### Ship the result, not the conversation. Ending the "Tomato-Egg Problem".
|
| 21 |
+
|
| 22 |
+
[](https://github.com/fancyboi999/ockev)
|
| 23 |
+
[](https://github.com/fancyboi999/ockev)
|
| 24 |
+
[]()
|
| 25 |
+
|
| 26 |
+
</div>
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
|
| 30 |
+
> *"I asked an AI agent to write a recipe for tomato scrambled eggs. It proudly delivered: 'Here is your tomato scrambled eggs (note: this dish contains no pork, no beef, no chicken, and no fish).' Why are AI agents so obsessed with telling us what they didn't do? Ship the result, not the conversation."*
|
| 31 |
+
> — **Shangyin Tan**, UC Berkeley
|
| 32 |
+
|
| 33 |
+
---
|
| 34 |
+
|
| 35 |
+
## Model Summary
|
| 36 |
+
|
| 37 |
+
**Ockev-1.5B** is a lightweight, zero-generation discriminative decision model built on `Qwen/Qwen2.5-1.5B` and fine-tuned with a **Pointer Readout Head** and Block-Causal Masking.
|
| 38 |
+
|
| 39 |
+
It intercepts conversational residue, discarded alternatives, and negative echoes in AI agent deliverables before they reach users or production codebases.
|
| 40 |
+
|
| 41 |
+
### Key Capabilities
|
| 42 |
+
|
| 43 |
+
- **Zero Generated Tokens**: Decisions are computed directly via scaled dot-product between decision query vectors and option key vectors. Zero hallucination, zero JSON repair.
|
| 44 |
+
- **35ms Latency**: 5x-7x faster than cloud decision services, 100% offline, free, and private.
|
| 45 |
+
- **Native Apple Silicon MLX**: Optimized for Metal GPU acceleration on MacBooks.
|
| 46 |
+
|
| 47 |
+
---
|
| 48 |
+
|
| 49 |
+
## Systems Comparison
|
| 50 |
+
|
| 51 |
+
| Execution Path | Generated Tokens | Time (ms) | JSON Parsing Failure Risk | Cost / 1M Checks |
|
| 52 |
+
| :--- | :---: | :---: | :---: | :---: |
|
| 53 |
+
| **Ockev-1.5B (Local Pointer Head)** | **0** | **35 ms** | **0% (Pure Softmax)** | **$0.00** |
|
| 54 |
+
| **TypeSafe Jev 1.13.0 (Cloud API)** | 0 | 250 ms | 0% (Typed) | ~$42.00 |
|
| 55 |
+
| **Generative LLM (e.g. GPT-4o-mini)** | 85–140 | 1,450 ms | 1.8% (Malformed JSON) | ~$8.50 |
|
| 56 |
+
|
| 57 |
+
---
|
| 58 |
+
|
| 59 |
+
## Benchmark Results (TomatoEggBench-120)
|
| 60 |
+
|
| 61 |
+
| Benchmark Domain | Ockev-1.5B Accuracy | Official Jev 1.13.0 |
|
| 62 |
+
| :--- | :---: | :---: |
|
| 63 |
+
| Code Repositories | 86.2% | 90.0% |
|
| 64 |
+
| Excel / CSV | 95.0% | 95.0% |
|
| 65 |
+
| PDF Documents | 100.0% | 100.0% |
|
| 66 |
+
| Form RPA | 100.0% | 100.0% |
|
| 67 |
+
| Executive Slide Decks | 85.7% | 85.7% |
|
| 68 |
+
| **Overall (120 Tasks)** | **92.5%** | **93.3%** |
|
| 69 |
+
|
| 70 |
+
---
|
| 71 |
|
| 72 |
## Quickstart
|
| 73 |
|
|
|
|
| 75 |
pip install ockev
|
| 76 |
ockev check path/to/deliverable.md
|
| 77 |
```
|
| 78 |
+
|
| 79 |
+
Python usage:
|
| 80 |
+
|
| 81 |
+
```python
|
| 82 |
+
from ockev import OckevGate
|
| 83 |
+
|
| 84 |
+
gate = OckevGate()
|
| 85 |
+
res = gate.review_deliverable("Added JWT authentication without session cookies.")
|
| 86 |
+
print(res)
|
| 87 |
+
# {'verdict': 'revise', 'confidence': 0.96, 'latency_ms': 34.2}
|
| 88 |
+
```
|
| 89 |
+
|
| 90 |
+
For full codebase and benchmarks, visit the [GitHub Repository](https://github.com/fancyboi999/ockev).
|