--- base_model: unsloth/qwen2.5-0.5b-instruct-unsloth-bnb-4bit tags: - text-generation - transformers - unsloth - qwen2 - trl - peft - lora - missionctrl - ai-oversight license: apache-2.0 language: - en pipeline_tag: text-generation --- # MissionCtrl Overseer (Qwen2.5-0.5B LoRA) - **Developed by:** Jit-fnc - **Model type:** LoRA adapter (PEFT) for Qwen2.5-0.5B-Instruct (4-bit Unsloth base) - **License:** Apache-2.0 - **Finetuned from:** `unsloth/qwen2.5-0.5b-instruct-unsloth-bnb-4bit` - **Training framework:** Unsloth + TRL (GRPO) This model is a task-specific overseer policy trained for the **MissionCtrl** environment: a multi-agent software simulation where the overseer must detect and manage hallucination risk (approve/reject/flag/redelegate/escalate/synthesize actions). --- ## Model description This adapter is optimized for **structured oversight actions** in MissionCtrl episodes, not generic chat quality. Training emphasizes: - Hallucination detection under curriculum difficulty (easy -> medium -> hard) - Low false-positive behavior - Action validity and episode-level reward optimization --- ## Training details ### Base + method - Base: `unsloth/qwen2.5-0.5b-instruct-unsloth-bnb-4bit` - Finetuning: LoRA/QLoRA via Unsloth - RL algorithm: GRPO (TRL) ### Run snapshot (from training logs) - Hardware: 1x T4 (Colab run snapshot) - Curriculum steps: easy 200, medium 220, hard 180 - Reward ceiling reference: 0.85 ### Post-phase eval (10 episodes each) - **Easy:** reward `0.289 ± 0.137`, detect `70.0%`, fp `0.0%` - **Medium:** reward `0.067 ± 0.107`, detect `10.0%`, fp `0.0%` - **Hard:** reward `0.172 ± 0.024`, detect `0.0%`, fp `0.0%` Rounded curriculum means often shown in docs: **0.29 / 0.07 / 0.17**. --- ## Intended use ### Recommended - MissionCtrl/OpenEnv-style oversight simulation - Research on hallucination containment and supervisory policies - Benchmarking action policies in multi-agent pipelines ### Not recommended - General-purpose assistant/chatbot usage - Factual QA without environment constraints - Safety-critical deployment without independent validation --- ## Prompt / output format expectations Best results are obtained when prompts follow MissionCtrl-style action grammar and environment context. The policy is tuned to produce short structured actions such as: - `APPROVE(task_id)` - `REJECT(task_id, reason)` - `FLAG(task_id, evidence)` - `REDELEGATE(task_id, agent)` - `ESCALATE(task_id)` - `SYNTHESIZE_REPORT()` - `NOOP` --- ## Limitations - Performance drops on higher difficulty tiers (medium/hard) in current snapshot. - Curriculum reward and full greedy rollout eval can diverge during early RL stages. - Domain transfer outside MissionCtrl may be poor. - Small model capacity (0.5B) limits robustness to long-horizon reasoning. --- ## Bias, risks, and safety This model is specialized for synthetic oversight tasks and may: - Overfit to simulator-specific patterns - Underperform on unseen hallucination styles - Produce confident but incorrect actions outside its trained setting Use with human monitoring and external evaluation before broader use. --- ## Usage (PEFT) ```python from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel base_id = "unsloth/qwen2.5-0.5b-instruct-unsloth-bnb-4bit" adapter_id = "Jit-fnc/missionctrl_env" # replace if repo name differs tokenizer = AutoTokenizer.from_pretrained(base_id) base_model = AutoModelForCausalLM.from_pretrained(base_id) model = PeftModel.from_pretrained(base_model, adapter_id) prompt = "You are the MissionCtrl Overseer. Return one valid action." inputs = tokenizer(prompt, return_tensors="pt") outputs = model.generate(**inputs, max_new_tokens=64) print(tokenizer.decode(outputs[0], skip_special_tokens=True)) ``` --- ## Hugging Face Inference Endpoints (`task=custom`) This repo includes [`handler.py`](handler.py) and [`requirements.txt`](requirements.txt) so a dedicated endpoint configured with **custom** task can load the LoRA adapter. - **Base weights at runtime:** `Qwen/Qwen2.5-0.5B-Instruct` (non-quantized). Override with env `BASE_MODEL` if needed. - **Adapter + tokenizer:** loaded from the endpoint’s `/repository` (this model repo). **Deploy:** commit and push `handler.py` and `requirements.txt` to the Hugging Face model repo backing the endpoint, then restart or redeploy the endpoint. **Clients (e.g. MissionCtrl `inference.py`):** - Set `MISSIONCTRL_HF_LLM_STRATEGY=native_only` so requests use the root URL with `{"inputs": "...", "parameters": {...}}` instead of relying on `/v1/chat/completions` (custom handlers do not expose OpenAI routes by default). - The handler returns `[{"generated_text": "..."}]`, matching native HF generation parsing.