| --- |
| license: mit |
| language: |
| - en |
| library_name: transformers |
| pipeline_tag: text-generation |
| tags: |
| - reasoning |
| - mathematics |
| - programming |
| - creative-writing |
| - chain-of-thought |
| - interpretability |
| - fairness |
| - security |
| - deployment |
| - sustainability |
| - monitoring |
| - plugin |
| --- |
| |
| # Brello Thinking |
|
|
| ## Model Description |
|
|
| **Brello Thinking** is an advanced language model created by **Epic Systems** as a part of **Brello AI Family**. Built on the robust Tencent Hunyuan base model, Brello Thinking specializes in deep reasoning, mathematical problem-solving, coding, and creative thinking with enhanced chain-of-thought capabilities. |
|
|
| ### Key Features |
|
|
| - **Advanced Reasoning**: Enhanced chain-of-thought with both fast and slow thinking modes |
| - **Mathematical Excellence**: Superior at math and symbolic computation |
| - **Programming Prowess**: Strong coding abilities across Python, JS, C++, SQL, and more |
| - **Long Context Understanding**: Handles up to 256K tokens, long docs, and codebases |
| - **Creative Problem Solving**: Generates new solutions and approaches |
| - **Multi-language Support**: Fluent in English and Chinese, robust cross-lingual transfer |
|
|
| --- |
|
|
| ## 1. Executive Summary |
|
|
| **Brello Thinking v1.1.0** (2025-08-07) is a 1.8B-parameter causal language model engineered for complex reasoning, mathematics, and creative tasks. It combines ultra-long context, dual “fast”/“deep” thinking modes, and a plugin SDK for live tool integration. It is designed for safe, sustainable, and fair production deployments. |
|
|
| #### Highlights in this Release |
|
|
| - **Mixed-precision quantization** (BF16 & INT8) |
| - **Plugin SDK** (JSON-RPC, HMAC auth, dynamic tool routing) |
| - **Monitoring** (Prometheus, Grafana, carbon tracking) |
| - **Sustainability Dashboard** (gCO₂eq/token metrics, CodeCarbon SDK) |
|
|
| --- |
|
|
| ## 2. Model Architecture |
|
|
| | Component | Specification | |
| |----------------------------|-----------------------------------------------------------------------------------------------------| |
| | **Base Model** | Tencent Hunyuan / EpicBrelloV1ForCausalLM | |
| | **Parameters** | 1.8B (BF16/INT8 quantization; LoRA adapters optional) | |
| | **Context Window** | 256,000 tokens (rotary cache, sliding window, eviction logic) | |
| | **Attention** | Grouped-Query + Multi-Head FlashAttention (16 heads, 4 KV heads) | |
| | **Feed-Forward** | Two-stage (SiLU → Linear → SiLU) with RMSNorm, hidden size 6144 | |
| | **Depth** | 32 transformer blocks + 4 “Safety Adapter” blocks | |
| | **Adapters** | LoRA for math, code, creative, and domain fine-tuning (10–18M params each) | |
| | **Inference Modes** | Autoregressive sampling (top-k, top-p), beam, contrastive decoding | |
| | **Sharding** | ZeRO-3 / tensor-parallel / model-parallel combinations | |
|
|
| --- |
|
|
| ## 3. Training & Tuning |
|
|
| ### 3.1 Pretraining Corpus |
|
|
| - **Web General**: 400B tokens (CommonCrawl, CC-100, curated news) |
| - **Science/Technical**: 50B tokens (arXiv, PubMed, patents) |
| - **Code**: 20B tokens (public GitHub, CodeSearchNet, MBPP) |
| - **Multilingual**: 30B tokens (Chinese, Spanish, German, Arabic) |
| - **Augmentations**: 15% span corruption, zh–en back-translation, dynamic masking |
|
|
| ### 3.2 Optimization |
|
|
| - **Optimizer**: AdamW (β₁=0.9, β₂=0.95, weight_decay=0.01) |
| - **LR Schedule**: Linear warmup (10K steps), cosine decay (500K steps) |
| - **Batch**: 2M tokens/step, grad accumulation ×8 |
| |
| ### 3.3 Instruction/RLHF Tuning |
| |
| - **Instruction Pairs**: 1.2M human-annotated QA/reasoning |
| - **Reward Model**: Dual human-preference ranking (5K raters, Elo) |
| - **Algorithm**: PPO w/ KL penalty (target KL=0.1), reward clipping |
| |
| --- |
| |
| ## 4. Specialized Modules |
| |
| | Adapter Name | Data Source | Params (M) | Use Case | |
| |-------------------|-----------------------------------|------------|----------------------------------| |
| | math-adapter | GSM8K, MATH, AIME datasets | 12 | Math proof, step-by-step logic | |
| | code-adapter | MBPP, MultiPL-E, GitHub repos | 18 | Coding, debugging, codegen | |
| | creative-adapter | Gutenberg, story corpora | 10 | Narrative, dialogue, ideation | |
| |
| --- |
| |
| ## 5. Plugin & Tooling SDK |
| |
| - **Interface**: JSON-RPC (Unix socket or REST), HMAC-SHA256 auth |
| - **Plugins**: |
| - DB connectors: PostgreSQL, MySQL, Snowflake |
| - HTTP client: retry/backoff |
| - Vector DB: FAISS, Pinecone |
| |
| #### Tool Call Example |
| |
| 1. Model emits: |
| ```json |
| {"tool_call": {"name": "weather_fetch", "args": {"location":"Mumbai"}}} |
| ``` |
| 2. Host executes plugin, returns: |
| ```json |
| {"tool_result": {"forecast":"Sunny, 32°C"}} |
| ``` |
| 3. Model resumes reasoning with tool result in context. |
| |
| --- |
|
|
| ## 6. Inference, Monitoring & Scaling |
|
|
| ### 6.1 Endpoint Performance |
|
|
| | Mode | Batch | Seq Len | Throughput (tok/s) | Latency (p50) | |
| |--------------|-------|----------|--------------------|---------------| |
| | Fast-Think | 8 | 4,096 | 250,000 | 15 ms | |
| | Deep-Think | 1 | 256,000 | 18,000 | 120 ms | |
| | INT8 Quant | 16 | 2,048 | 320,000 | 12 ms | |
|
|
| ### 6.2 Observability |
|
|
| - **Prometheus Metrics**: |
| - `brello_inference_latency_seconds` |
| - `brello_generated_tokens_total` |
| - `brello_cache_evictions_total` |
| - **Grafana**: |
| - Token latency histograms, CO₂ per generation |
|
|
| --- |
|
|
| ## 7. Sustainability & Carbon Tracking |
|
|
| - **Data Center PUE**: 1.2 |
| - **Carbon Emission**: ~0.0008 gCO₂eq/token (tracked with CodeCarbon) |
| - **Offset**: Epic Systems funds VER 2.0 credits |
|
|
| --- |
|
|
| ## 8. Robustness, Safety & Fairness |
|
|
| - **Adapters**: Real-time adversarial input filtering, personal data redaction, toxicity classifier (fine-tuned BERT-tox) |
| - **Bias Audits**: |
| - Toxicity variation <1.8% (12 demographic axes) |
| - Gender parity ±2% |
| - Dialect coverage 98% (EN & ZH) |
|
|
| --- |
|
|
| ## 9. Interpretability |
|
|
| - **Chain-of-Thought logs**: Token-level reasoning trace |
| - **Integrated Gradients**: Span attribution |
| - **Attention Rollouts**: Layer-wise visualization (custom plugin) |
|
|
| --- |
|
|
| ## 10. Hyperparameters |
|
|
| | Parameter | Value | |
| |-------------------|----------| |
| | num_layers | 32 | |
| | d_model | 2048 | |
| | d_hidden | 6144 | |
| | num_heads | 16 | |
| | kv_heads | 4 | |
| | rotary_pct | 0.25 | |
| | lr_warmup_steps | 10,000 | |
| | weight_decay | 0.01 | |
| | batch_size | 2M | |
| | dropout_rate | 0.1 | |
| |
| --- |
| |
| ## 11. Evaluation & Error Analysis |
| |
| - **Benchmarks**: GSM8K, MBPP, BBH, LongBench, MATH |
| - **Analysis**: Math/logic confusion matrix, hallucination drift cluster analysis |
| |
| --- |
| |
| ## 12. Roadmap |
| |
| | Version | Highlights | ETA | |
| |-----------|----------------------------------------------|----------| |
| | v1.1.0 | Plugins, carbon tracking, INT8 quantization | Released | |
| | v1.2.0 | Vision-language, adapter expansion | Nov 2025 | |
| | v1.3.0 | Audio, multilingual tuning | Feb 2026 | |
| | v2.0 | Federated RAG, continuous learning | Q4 2026 | |
| |
| --- |
| |
| ## 13. Licensing & Compliance |
| |
| - **License**: Proprietary, Epic Systems |
| - **Privacy**: GDPR, CCPA compliant |
| - **Certifications**: ISO 27001, SOC 2 Type II, HIPAA (BAA on request) |
| - **Restrictions**: No redistribution or large-scale rehosting |
| |
| --- |
| |
| ## 14. Usage Example |
| |
| ```python |
| import os |
| import torch |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| from peft import PeftModel # For LoRA adapters |
| from brello_sdk import BrelloPluginManager # Hypothetical SDK |
| from codecarbon import EmissionsTracker |
| from prometheus_client import CollectorRegistry, Counter, Histogram, push_to_gateway |
| |
| def setup_model( |
| model_id: str = "BrelloES/brello-thinking", |
| use_bf16: bool = True, |
| load_int8: bool = True, |
| ): |
| tokenizer = AutoTokenizer.from_pretrained(model_id) |
| model = AutoModelForCausalLM.from_pretrained( |
| model_id, |
| device_map="auto", |
| torch_dtype=torch.bfloat16 if use_bf16 else torch.float32, |
| load_in_8bit=load_int8, |
| ) |
| # Attach LoRA adapters |
| model = PeftModel.from_pretrained(model, "adapters/math-adapter") |
| model = PeftModel.from_pretrained(model, "adapters/code-adapter") |
| return tokenizer, model |
| |
| def setup_plugins(): |
| pm = BrelloPluginManager() |
| pm.register( |
| name="weather_fetch", |
| path="/opt/brello/plugins/weather_plugin.so", |
| auth_key=os.getenv("WEATHER_PLUGIN_KEY", "CHANGE_ME"), |
| ) |
| pm.register( |
| name="db_query", |
| path="/opt/brello/plugins/db_query_plugin.so", |
| auth_key=os.getenv("DB_PLUGIN_KEY", "CHANGE_ME"), |
| ) |
| return pm |
| |
| def setup_metrics(): |
| registry = CollectorRegistry() |
| Histogram( |
| "brello_inference_latency_seconds", |
| "Inference latency (seconds) per request", |
| registry=registry, |
| buckets=(0.01, 0.05, 0.1, 0.2, 0.5, 1.0), |
| ) |
| Counter( |
| "brello_generated_tokens_total", |
| "Total number of tokens generated by Brello", |
| registry=registry, |
| ) |
| return registry |
| |
| def generate_response(tokenizer, model, plugin_mgr, registry, messages, mode: str = "deep"): |
| inputs = tokenizer.apply_chat_template( |
| messages, |
| tokenize=True, |
| add_generation_prompt=True, |
| enable_thinking=True if mode == "deep" else False, |
| ) |
| tracker = EmissionsTracker(project_name="brello_inference", output_dir="carbon_logs") |
| tracker.start() |
| # (Metrics update simplified for clarity) |
| outputs = model.generate( |
| inputs.to(model.device), |
| max_new_tokens=512, |
| top_p=0.9, |
| temperature=0.6, |
| plugin_manager=plugin_mgr, |
| return_dict_in_generate=True, |
| output_scores=True, |
| ) |
| emissions_kg = tracker.stop() |
| text = tokenizer.decode(outputs.sequences[0], skip_special_tokens=True) |
| return text, emissions_kg |
| |
| def main(): |
| tokenizer, model = setup_model() |
| plugin_mgr = setup_plugins() |
| registry = setup_metrics() |
| messages = [ |
| {"role": "system", "content": "You are Brello Thinking in Deep-Think mode."}, |
| {"role": "user", "content": "Explain why prime factorization is unique."}, |
| ] |
| response, co2 = generate_response(tokenizer, model, plugin_mgr, registry, messages, mode="deep") |
| print("=== Deep-Think Output ===\n", response) |
| print(f"CO₂ Emitted: {co2:.6f} kg") |
| # Fast-Think comparison |
| messages[0]["content"] = "You are Brello Thinking in Fast-Think mode." |
| response_fast, co2_fast = generate_response(tokenizer, model, plugin_mgr, registry, messages, mode="fast") |
| print("\n=== Fast-Think Output ===\n", response_fast) |
| print(f"CO₂ Emitted: {co2_fast:.6f} kg") |
| |
| if __name__ == "__main__": |
| main() |
| ``` |
| |
| --- |
|
|
| ## Otvd |
|
|
| - **Creator**: Epic Systems |
| - **Engineer**: Rehan Temkar |
| - **Model**: Brello Thinking v1.0.0 |
|
|
| --- |
|
|
| *Brello Thinking - Advanced AI Reasoning by Epic Systems* |
|
|
| --- |
