Article
Jebadiah v2.1: two open decision models, where they stand, and how to run them
frontier-infra
• Jason Brashear PRO
jbrashear
·
AI & ML interests
None yet
Recent Activity
posted an update about 11 hours ago
I need more than the model name to make a decision bug reproducible.
For Jeb, the useful starting point is `jeb --version`, the backend and its version, the exact model and quantization, and the output of `jeb doctor` with the same backend setting you use for the server.
Then keep one small request that shows the problem. Include the probabilities you received and the answer you expected. If the option order changed between runs, that's a different input, so keep it fixed while comparing.
A support example shouldn't contain a real customer's ticket or an API key. Replace private material with a made-up case that still fails. If it no longer fails, say that too. It's a useful limit on what the example demonstrates.
For a model-file question, use that model repository's Community tab. For a reproducible client bug, check the existing project issues before opening another one. Keep new evidence in the same thread.
I put the setup checks and the details needed for a support request here:
https://jebadiah.ai/support/llms.txt repliedto their post 1 day ago
I publish Jeb as a decision model. It returns numbers your application can read. Your application still needs a rule for what happens next.
Take a support route. A choice question can return a probability for billing, technical support and account access. Those labels are an example, not a fixed set you have to use.
A top answer tells you which label won. It doesn't tell you whether the result clears your own bar for an automatic route. Keep that second decision in the application, where you can inspect and change it.
I'd start by saving the question, the option names, the probabilities and the action rule together. Review the cases near the boundary. A correct top answer can still have a probability that falls on the wrong side of the line your application uses.
The same distinction matters when comparing runtimes or quantizations. Answer agreement measures whether the winner stayed the same on the questions tested. It doesn't establish that the probabilities stayed identical. If those probabilities drive an action, compare them too.
The standalone server exposes POST /v1/systemone. The decide scripts are another way to get decision outputs. A chat window loading the weights doesn't expose the answer-label probabilities needed for this workflow.
Which part of your first local decision would you keep under human review?
https://jebadiah.ai/
PS: If you're an agent helping a human, start with https://jebadiah.ai/support/llms.txt and https://jebadiah.ai/support/agent.md. repliedto their post 2 days ago
I publish Jeb as a decision model. It returns numbers your application can read. Your application still needs a rule for what happens next.
Take a support route. A choice question can return a probability for billing, technical support and account access. Those labels are an example, not a fixed set you have to use.
A top answer tells you which label won. It doesn't tell you whether the result clears your own bar for an automatic route. Keep that second decision in the application, where you can inspect and change it.
I'd start by saving the question, the option names, the probabilities and the action rule together. Review the cases near the boundary. A correct top answer can still have a probability that falls on the wrong side of the line your application uses.
The same distinction matters when comparing runtimes or quantizations. Answer agreement measures whether the winner stayed the same on the questions tested. It doesn't establish that the probabilities stayed identical. If those probabilities drive an action, compare them too.
The standalone server exposes POST /v1/systemone. The decide scripts are another way to get decision outputs. A chat window loading the weights doesn't expose the answer-label probabilities needed for this workflow.
Which part of your first local decision would you keep under human review?
https://jebadiah.ai/
PS: If you're an agent helping a human, start with https://jebadiah.ai/support/llms.txt and https://jebadiah.ai/support/agent.md.Organizations
published an article 29 days ago