| --- |
| license: apache-2.0 |
| datasets: |
| - bravesoftware/diverse-llm-prompts-34k |
| - lmsys/lmsys-chat-1m |
| base_model: |
| - answerdotai/ModernBERT-base |
| --- |
| # Androcles |
|
|
| Androcles is our inhouse **Prompt Classification Model**, used to classify user prompts into different categories to dynamically change the functionality based on the context of the request. It's a simple text classification model finetuned on a mix of synthetic & open source data across 21 labels |
|
|
| **No Brave Browser/Leo user data was used to train this model - even if we wanted to, we don't have it** |
|
|
| ## Labels |
| | Label | Short Description | |
| |---------------------|-----------------------------------------------------------------------------------| |
| | Brave | The user is asking about Brave products — Brave Browser, Brave Search, Brave Rewards | |
| | Browser | The user wants Leo to take actions in the browser — clicking, filling forms, navigating pages | |
| | Choice | The user is deciding between two or more specific options and wants help choosing | |
| | Coding | Anything related to coding — writing, reading, understanding, explaining, or running code in any language | |
| | Data Analysis | The user has data they want analysed, visualised, or interpreted — CSV, JSON, a table, or raw numbers | |
| | Diagrams | The user wants a diagram, flowchart, chart, or visual representation of a system or process | |
| | Fact Checking | The user is asking whether a claim, statement, or piece of information is true or false | |
| | Finance | The user is asking about money — currrency exchange/stock prices etc | |
| | Image Generation | Generating an image — the user is explicitly asking for an image to be created or produced | |
| | Math / Calculations | Anything involving numbers, arithmetic, algebra, statistics, or mathematical reasoning | |
| | Multilingualism | The user is writing in or asking for a response in a non-English language | |
| | News | The user is asking about current events, recent news, or what's happening in the world | |
| | Recommendation | The user is asking for a recommendation — a product, service, tool, or option suited to their needs | |
| | Sports | The user is asking about sports — scores, fixtures, standings, results, or sports news | |
| | Structured Writing | The user wants help writing something structured and polished — a cover letter, essay, email, report, blog post etc | |
| | Summarisation | The user wants a summary of something — an article, video, document, or piece of text | |
| | Thinking | The user wants Leo to think through something carefully and methodically — a complex problem, a multi-step question, or a nuanced topic | |
| | Time-critical | The request is urgent or time-sensitive — the user needs a fast answer, not a thorough one | |
| | Translation | Translating text from one language to another | |
| | Travel Planning | The user is planning a trip — flights, hotels, itineraries, destinations, things to do | |
| | Weather | The user is asking about the weather — current conditions, forecasts, or climate for a location | |
|
|
| *** |
| |
| ## Training |
| |
| Androcles 2 was trained using a mix of [`bravesoftware/diverse-llm-prompts-34k`](https://huggingface.co/datasets/bravesoftware/androcles-2-chat-50k](https://huggingface.co/datasets/bravesoftware/diverse-llm-prompts-34k)) + labelled data from [`LMSYS/lmsys-chat-1m`](https://huggingface.co/datasets/lmsys/lmsys-chat-1m) |
| |
| The scripts for training the model & generating the initial dataset can be found [here](https://github.com/brave/Androcles/tree/main) |
| |
| ## How We Use It |
| |
| We use Androcles in a couple of different ways: |
| |
| * As a first step in our [AI Gateway Server](https://github.com/brave/ai-gateway/tree/main) powering [Leo](https://brave.com/leo/), classifying prompts into categories with the aim of dynamically changing available tools/instructions depending on what the user asks for |
| |
| * As a 'primary category' labeller for our privacy-preserving Leo Analytics server |
| |
| We host it using Nvidia's Triton Inference Server to serve it with low-latency at scale, but we've also used KServe in the past |
| |
| ## Caveats |
| |
| This model was not trained to be perfect and will make mistakes in classification, the aim was to catch some low-hanging fruit semantically to improve the user experience |
| |
| |
| |
