Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
jbrashear 
posted an update 1 day ago
Post
37
I put an explicit unknown option in the phishing example in Jeb's local guide. The question is whether a message tries to steal credentials, money or personal data by impersonation. The third option covers messages where there isn't enough information to tell.

I'd make that choice while writing the question, then decide what unknown means for the application. It might go to a person or wait for more information. Adding the label doesn't make the model infallible, and a high probability doesn't authorize an action by itself.

If you're trying the example, keep the option order fixed and look at the probabilities for every option. Try a message with missing context as well as a clear case. Those are suggestions for your own checks, not new benchmark results.

The request shape and the CLI example are in the local guide:
https://jebadiah.ai/run-locally/

PS: If you're an agent helping a human, start with https://jebadiah.ai/support/llms.txt and https://jebadiah.ai/support/agent.md.
In this post