iamrahull commited on
Commit
3eb0a04
·
verified ·
1 Parent(s): 4a6adf5

Updated Readme.md

Browse files
Files changed (1) hide show
  1. README.md +35 -55
README.md CHANGED
@@ -1,11 +1,32 @@
1
- # Project AETHER: xLAM-AETHER-v1 Model Card & Technical Report
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
 
 
3
  ## 1. Executive Summary
4
  **xLAM-AETHER-v1** is a specialized, 3-Billion parameter Large Language Model (LLM) engineered to serve as the cognitive engine for autonomous agentic workflows.
5
 
6
  Developed as the core component of **Project AETHER**, this model is designed to act not just as a conversational chatbot, but as a **ReAct (Reasoning + Acting) agent**. It has been fine-tuned to understand tool schemas, reason through a user's goal, and output highly structured, syntactically perfect JSON payloads to trigger external tools (such as shell commands or API calls).
7
 
8
- Built upon the robust `Salesforce/xLAM-2-3b-fc-r` architecture (which is based on Qwen2.5), the model was distilled using QLoRA and quantized to a highly efficient 4-bit GGUF format. This allows it to run locally on consumer-grade laptops while retaining the complex reasoning capabilities of its parent architecture.
9
 
10
  ## 2. Core Capabilities
11
  - **Dual-Mode Operation:** The model seamlessly switches between general conversational chat (natural language) and Agentic execution (strict JSON tool calls) without suffering from catastrophic forgetting.
@@ -14,56 +35,39 @@ Built upon the robust `Salesforce/xLAM-2-3b-fc-r` architecture (which is based o
14
  - **Low Hardware Footprint:** Quantized to `Q4_K_M` (~1.93 GB), allowing it to run entirely in system RAM on standard laptops or free cloud tiers.
15
 
16
  ## 3. Technical Specifications
17
-
18
  | Attribute | Detail |
19
  | :--- | :--- |
20
  | **Base Model** | `Salesforce/xLAM-2-3b-fc-r` (Qwen2 architecture) |
21
  | **Total Parameters** | 3.09 Billion (Base) + 7.3 Million (LoRA Adapter) |
22
  | **Fine-Tuning Method** | QLoRA (Quantized Low-Rank Adaptation) |
23
  | **Training Precision** | 4-bit NormalFloat (NF4) with FP16 compute |
24
- | **Dataset** | `Salesforce/APIGen-MT-5k` (5,000 verified multi-turn function-calling trajectories) |
25
  | **Training Hardware** | Google Colab (NVIDIA Tesla T4) |
26
  | **Final Format** | GGUF (`q4_k_m` quantization) |
27
- | **Chat Template** | ChatML (`<\|im_start\|>` / `<\|im_end\|>`) |
28
  | **File Size** | 1.93 GB |
29
- | **License** | CC-BY-NC-4.0 (Non-Commercial) |
30
 
31
  ## 4. End-to-End Training Pipeline
32
-
33
  ### 4.1 Data Synthesis & Curation
34
- Rather than training from scratch, the model utilized the `Salesforce/APIGen-MT-5k` dataset. This dataset contains strictly verified, multi-turn API interactions. The data was formatted using the official ChatML template, ensuring the model learned the exact token boundaries for when to start reasoning and when to trigger a tool.
35
 
36
  ### 4.2 The Distillation Protocol (QLoRA)
37
- To inject reasoning capabilities into the 3B model without requiring enterprise-grade GPUs, the training utilized QLoRA:
38
  - **Base Quantization:** The base model was loaded in 4-bit NF4.
39
- - **Adapter Injection:** Trainable low-rank matrices (Matrix A and B) were injected into the attention layers (`q_proj`, `k_proj`, `v_proj`, `o_proj`).
40
- - **Hyperparameters:** Rank (\(r\)) was set to 16, Alpha (\(\alpha\)) to 32, with a learning rate of \(2e-4\).
41
- - **Efficiency:** Using the **Unsloth** framework, the training loop was optimized for the T4 GPU. The model was trained for 1 epoch (625 steps), achieving a final training loss of `0.0003`, indicating high convergence on the target ReAct formatting.
42
 
43
  ### 4.3 Merging & Quantization
44
- Post-training, the 28MB LoRA adapter was mathematically merged with the 6GB base model. To ensure the model could run on edge devices, the merged weights were converted to GGUF format and quantized to **Q4_K_M**. This 4-bit integer quantization protects the "salient" weights (those contributing most to the reasoning spikes) while compressing the model to under 2GB.
45
 
46
  ## 5. Benchmark & Behavior Tests
47
-
48
- Post-deployment, the model was subjected to a series of behavioral tests to ensure it did not lose its base intelligence while learning the tool-calling format.
49
-
50
- **Test A: General Knowledge (Chat Mode)**
51
- - *Prompt:* "What are the cultural beliefs of China and Egypt? Give a short summary."
52
- - *Output:* Coherent, well-structured natural language summary covering mummification, Confucianism, and filial piety. No hallucinated JSON. Retained its "chat" persona.
53
-
54
- **Test B: Reasoning (Math Mode)**
55
- - *Prompt:* "If a train travels at 60 mph for 2.5 hours, how far does it go? Explain the math."
56
- - *Output:* Step-by-step breakdown using LaTeX formatting, correctly applying the formula \(Distance = Speed \times Time\), and concluding with `150 miles`.
57
-
58
- **Test C: Agentic Tool Calling (ReAct Mode)**
59
- - *Prompt:* *"List all .tmp files in /tmp/aether-sandbox."* (Provided with a `shell` tool schema).
60
- - *Output:* `[{"name": "shell", "arguments": {"cmd": "find /tmp/aether-sandbox -type f -name '.tmp'"}}]`
61
- - *Latency:* 1.12 seconds (via API on a Colab T4 GPU).
62
 
63
  ## 6. Deployment & Inference
64
-
65
  ### Option 1: Local Inference (Ollama / LM Studio)
66
- Due to the GGUF format, the model can be run locally. A `Modelfile` is required to enforce the ChatML template.
67
  ```text
68
  FROM ./xlam-aether-q4_k_m.gguf
69
  TEMPLATE """<|im_start|>system
@@ -73,28 +77,4 @@ TEMPLATE """<|im_start|>system
73
  <|im_start|>assistant
74
  """
75
  PARAMETER stop "<|im_end|>"
76
- PARAMETER temperature 0.1
77
- ```
78
-
79
- ### Option 2: OpenAI-Compatible API (Colab)
80
- Using `llama-cpp-python`, the model can be served as a local OpenAI-compatible API. This is ideal for building FastAPI backends that interact with the model.
81
- ```bash
82
- python -m llama_cpp.server --model xlam-aether-q4_k_m.gguf --n_gpu_layers 99 --chat_format chatml --port 8000
83
- ```
84
-
85
- ## 7. Limitations & Future Work
86
-
87
- - **License Constraints:** Because it inherits from the xLAM base model, this model is licensed under **CC-BY-NC-4.0**. It is strictly for research and personal use. Commercial deployment would require re-training on a permissive base model (e.g., Llama-3.2-3B).
88
- - **Hardware Bottlenecks:** While fast on a GPU (~1.1s response), running this model purely on a CPU (like Hugging Face's free tier) results in massive latency (~250 seconds per prompt) due to the prefill phase processing the tool schema.
89
- - **Parallel Tool Calling:** The model currently excels at single-action tool calls. Future training iterations will focus on emitting arrays of multiple parallel tool calls.
90
-
91
- ## 8. Links & Resources
92
- - **Model Repository:** [huggingface.co/iamrahull/xlam-ather](https://huggingface.co/iamrahull/xlam-ather)
93
- - **Demo Space:** [huggingface.co/spaces/iamrahull/xlam-ather](https://huggingface.co/spaces/iamrahull/xlam-ather)
94
- - **Training Dataset:** [Salesforce/APIGen-MT-5k](https://huggingface.co/datasets/Salesforce/APIGen-MT-5k)
95
-
96
- ---
97
- *Report generated for Project AETHER. Designed and trained by Rahul.*
98
- ---
99
- license: cc-by-nc-4.0
100
- ---
 
1
+ ---
2
+ license: cc-by-nc-4.0
3
+ base_model: Salesforce/xLAM-2-3b-fc-r
4
+ library_name: gguf
5
+ pipeline_tag: text-generation
6
+ tags:
7
+ - xlam
8
+ - function-calling
9
+ - agent
10
+ - tool-calling
11
+ - react
12
+ - qlora
13
+ - gguf
14
+ - qwen2
15
+ - qwen2.5
16
+ - 3b
17
+ language:
18
+ - en
19
+ datasets:
20
+ - Salesforce/APIGen-MT-5k
21
+ ---
22
 
23
+ # Project AETHER: xLAM-AETHER-v1 Model Card & Technical Report
24
  ## 1. Executive Summary
25
  **xLAM-AETHER-v1** is a specialized, 3-Billion parameter Large Language Model (LLM) engineered to serve as the cognitive engine for autonomous agentic workflows.
26
 
27
  Developed as the core component of **Project AETHER**, this model is designed to act not just as a conversational chatbot, but as a **ReAct (Reasoning + Acting) agent**. It has been fine-tuned to understand tool schemas, reason through a user's goal, and output highly structured, syntactically perfect JSON payloads to trigger external tools (such as shell commands or API calls).
28
 
29
+ Built upon the robust `【entity-Salesforce¦canonical_name=Salesforce】/xLAM-2-3b-fc-r` architecture (which is based on Qwen2.5), the model was distilled using QLoRA and quantized to a highly efficient 4-bit GGUF format. This allows it to run locally on consumer-grade laptops while retaining the complex reasoning capabilities of its parent architecture.
30
 
31
  ## 2. Core Capabilities
32
  - **Dual-Mode Operation:** The model seamlessly switches between general conversational chat (natural language) and Agentic execution (strict JSON tool calls) without suffering from catastrophic forgetting.
 
35
  - **Low Hardware Footprint:** Quantized to `Q4_K_M` (~1.93 GB), allowing it to run entirely in system RAM on standard laptops or free cloud tiers.
36
 
37
  ## 3. Technical Specifications
 
38
  | Attribute | Detail |
39
  | :--- | :--- |
40
  | **Base Model** | `Salesforce/xLAM-2-3b-fc-r` (Qwen2 architecture) |
41
  | **Total Parameters** | 3.09 Billion (Base) + 7.3 Million (LoRA Adapter) |
42
  | **Fine-Tuning Method** | QLoRA (Quantized Low-Rank Adaptation) |
43
  | **Training Precision** | 4-bit NormalFloat (NF4) with FP16 compute |
44
+ | **Dataset** | `Salesforce/APIGen-MT-5k` |
45
  | **Training Hardware** | Google Colab (NVIDIA Tesla T4) |
46
  | **Final Format** | GGUF (`q4_k_m` quantization) |
47
+ | **Chat Template** | ChatML |
48
  | **File Size** | 1.93 GB |
49
+ | **License** | CC-BY-NC-4.0 |
50
 
51
  ## 4. End-to-End Training Pipeline
 
52
  ### 4.1 Data Synthesis & Curation
53
+ Rather than training from scratch, the model utilized the `Salesforce/APIGen-MT-5k` dataset.
54
 
55
  ### 4.2 The Distillation Protocol (QLoRA)
 
56
  - **Base Quantization:** The base model was loaded in 4-bit NF4.
57
+ - **Adapter Injection:** Trainable low-rank matrices injected into attention layers.
58
+ - **Hyperparameters:** Rank 16, Alpha 32, LR 2e-4.
59
+ - **Efficiency:** Using Unsloth framework, trained for 1 epoch (625 steps), final loss `0.0003`.
60
 
61
  ### 4.3 Merging & Quantization
62
+ Post-training, the 28MB LoRA adapter was merged and converted to GGUF `Q4_K_M`.
63
 
64
  ## 5. Benchmark & Behavior Tests
65
+ **Test A: General Knowledge** - Retained chat persona.
66
+ **Test B: Reasoning** - Correct math with LaTeX.
67
+ **Test C: Agentic Tool Calling** - Outputs `[{"name": "shell", "arguments": {"cmd": "..."}}]` with 1.12s latency.
 
 
 
 
 
 
 
 
 
 
 
 
68
 
69
  ## 6. Deployment & Inference
 
70
  ### Option 1: Local Inference (Ollama / LM Studio)
 
71
  ```text
72
  FROM ./xlam-aether-q4_k_m.gguf
73
  TEMPLATE """<|im_start|>system
 
77
  <|im_start|>assistant
78
  """
79
  PARAMETER stop "<|im_end|>"
80
+ PARAMETER temperature 0.1