sft_v2_lora (10b7252) SFT v2 cold start, sftv2
Browse files- checkpoints/sft_v2_lora/epoch1/README.md +207 -0
- checkpoints/sft_v2_lora/epoch1/adapter_config.json +54 -0
- checkpoints/sft_v2_lora/epoch1/adapter_model.safetensors +3 -0
- checkpoints/sft_v2_lora/epoch2/README.md +207 -0
- checkpoints/sft_v2_lora/epoch2/adapter_config.json +54 -0
- checkpoints/sft_v2_lora/epoch2/adapter_model.safetensors +3 -0
- checkpoints/sft_v2_lora/epoch3/README.md +207 -0
- checkpoints/sft_v2_lora/epoch3/adapter_config.json +54 -0
- checkpoints/sft_v2_lora/epoch3/adapter_model.safetensors +3 -0
- checkpoints/sft_v2_lora/final/README.md +207 -0
- checkpoints/sft_v2_lora/final/adapter_config.json +54 -0
- checkpoints/sft_v2_lora/final/adapter_model.safetensors +3 -0
- checkpoints/sft_v2_lora/manifest.json +61 -0
- checkpoints/sft_v2_lora/train_args.json +50 -0
- checkpoints/sft_v2_lora/train_log.jsonl +198 -0
checkpoints/sft_v2_lora/epoch1/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: /mnt/raid5/models/Qwen3.8-27B
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:/mnt/raid5/models/Qwen3.8-27B
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.21.2
|
checkpoints/sft_v2_lora/epoch1/adapter_config.json
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {},
|
| 4 |
+
"arrow_config": null,
|
| 5 |
+
"auto_mapping": null,
|
| 6 |
+
"base_model_name_or_path": "/mnt/raid5/models/Qwen3.8-27B",
|
| 7 |
+
"bias": "none",
|
| 8 |
+
"corda_config": null,
|
| 9 |
+
"ensure_weight_tying": false,
|
| 10 |
+
"eva_config": null,
|
| 11 |
+
"exclude_modules": null,
|
| 12 |
+
"fan_in_fan_out": false,
|
| 13 |
+
"inference_mode": true,
|
| 14 |
+
"init_lora_weights": true,
|
| 15 |
+
"kasa_config": null,
|
| 16 |
+
"layer_replication": null,
|
| 17 |
+
"layers_pattern": null,
|
| 18 |
+
"layers_to_transform": null,
|
| 19 |
+
"loftq_config": {},
|
| 20 |
+
"lora_alpha": 128,
|
| 21 |
+
"lora_bias": false,
|
| 22 |
+
"lora_dropout": 0.05,
|
| 23 |
+
"lora_ga_config": null,
|
| 24 |
+
"megatron_config": null,
|
| 25 |
+
"megatron_core": "megatron.core",
|
| 26 |
+
"modules_to_save": null,
|
| 27 |
+
"monteclora_config": null,
|
| 28 |
+
"peft_type": "LORA",
|
| 29 |
+
"peft_version": "0.21.2",
|
| 30 |
+
"qalora_group_size": 16,
|
| 31 |
+
"r": 64,
|
| 32 |
+
"rank_pattern": {},
|
| 33 |
+
"revision": null,
|
| 34 |
+
"target_modules": [
|
| 35 |
+
"down_proj",
|
| 36 |
+
"v_proj",
|
| 37 |
+
"up_proj",
|
| 38 |
+
"in_proj_z",
|
| 39 |
+
"o_proj",
|
| 40 |
+
"in_proj_qkv",
|
| 41 |
+
"out_proj",
|
| 42 |
+
"gate_proj",
|
| 43 |
+
"q_proj",
|
| 44 |
+
"k_proj"
|
| 45 |
+
],
|
| 46 |
+
"target_parameters": null,
|
| 47 |
+
"task_type": "CAUSAL_LM",
|
| 48 |
+
"trainable_token_indices": null,
|
| 49 |
+
"use_bdlora": null,
|
| 50 |
+
"use_dora": false,
|
| 51 |
+
"use_qalora": false,
|
| 52 |
+
"use_rslora": false,
|
| 53 |
+
"velora_config": null
|
| 54 |
+
}
|
checkpoints/sft_v2_lora/epoch1/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8a11ff164a79dfd19dde13f977ec46a97e04e680120ccc6ea3ac4652789fa16d
|
| 3 |
+
size 1740746640
|
checkpoints/sft_v2_lora/epoch2/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: /mnt/raid5/models/Qwen3.8-27B
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:/mnt/raid5/models/Qwen3.8-27B
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.21.2
|
checkpoints/sft_v2_lora/epoch2/adapter_config.json
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {},
|
| 4 |
+
"arrow_config": null,
|
| 5 |
+
"auto_mapping": null,
|
| 6 |
+
"base_model_name_or_path": "/mnt/raid5/models/Qwen3.8-27B",
|
| 7 |
+
"bias": "none",
|
| 8 |
+
"corda_config": null,
|
| 9 |
+
"ensure_weight_tying": false,
|
| 10 |
+
"eva_config": null,
|
| 11 |
+
"exclude_modules": null,
|
| 12 |
+
"fan_in_fan_out": false,
|
| 13 |
+
"inference_mode": true,
|
| 14 |
+
"init_lora_weights": true,
|
| 15 |
+
"kasa_config": null,
|
| 16 |
+
"layer_replication": null,
|
| 17 |
+
"layers_pattern": null,
|
| 18 |
+
"layers_to_transform": null,
|
| 19 |
+
"loftq_config": {},
|
| 20 |
+
"lora_alpha": 128,
|
| 21 |
+
"lora_bias": false,
|
| 22 |
+
"lora_dropout": 0.05,
|
| 23 |
+
"lora_ga_config": null,
|
| 24 |
+
"megatron_config": null,
|
| 25 |
+
"megatron_core": "megatron.core",
|
| 26 |
+
"modules_to_save": null,
|
| 27 |
+
"monteclora_config": null,
|
| 28 |
+
"peft_type": "LORA",
|
| 29 |
+
"peft_version": "0.21.2",
|
| 30 |
+
"qalora_group_size": 16,
|
| 31 |
+
"r": 64,
|
| 32 |
+
"rank_pattern": {},
|
| 33 |
+
"revision": null,
|
| 34 |
+
"target_modules": [
|
| 35 |
+
"k_proj",
|
| 36 |
+
"up_proj",
|
| 37 |
+
"down_proj",
|
| 38 |
+
"o_proj",
|
| 39 |
+
"gate_proj",
|
| 40 |
+
"v_proj",
|
| 41 |
+
"in_proj_qkv",
|
| 42 |
+
"in_proj_z",
|
| 43 |
+
"out_proj",
|
| 44 |
+
"q_proj"
|
| 45 |
+
],
|
| 46 |
+
"target_parameters": null,
|
| 47 |
+
"task_type": "CAUSAL_LM",
|
| 48 |
+
"trainable_token_indices": null,
|
| 49 |
+
"use_bdlora": null,
|
| 50 |
+
"use_dora": false,
|
| 51 |
+
"use_qalora": false,
|
| 52 |
+
"use_rslora": false,
|
| 53 |
+
"velora_config": null
|
| 54 |
+
}
|
checkpoints/sft_v2_lora/epoch2/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:51e9d732fb4576818eba7f987b2f3295b684941781a587b47a2e2d789a3bb193
|
| 3 |
+
size 1740746640
|
checkpoints/sft_v2_lora/epoch3/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: /mnt/raid5/models/Qwen3.8-27B
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:/mnt/raid5/models/Qwen3.8-27B
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.21.2
|
checkpoints/sft_v2_lora/epoch3/adapter_config.json
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {},
|
| 4 |
+
"arrow_config": null,
|
| 5 |
+
"auto_mapping": null,
|
| 6 |
+
"base_model_name_or_path": "/mnt/raid5/models/Qwen3.8-27B",
|
| 7 |
+
"bias": "none",
|
| 8 |
+
"corda_config": null,
|
| 9 |
+
"ensure_weight_tying": false,
|
| 10 |
+
"eva_config": null,
|
| 11 |
+
"exclude_modules": null,
|
| 12 |
+
"fan_in_fan_out": false,
|
| 13 |
+
"inference_mode": true,
|
| 14 |
+
"init_lora_weights": true,
|
| 15 |
+
"kasa_config": null,
|
| 16 |
+
"layer_replication": null,
|
| 17 |
+
"layers_pattern": null,
|
| 18 |
+
"layers_to_transform": null,
|
| 19 |
+
"loftq_config": {},
|
| 20 |
+
"lora_alpha": 128,
|
| 21 |
+
"lora_bias": false,
|
| 22 |
+
"lora_dropout": 0.05,
|
| 23 |
+
"lora_ga_config": null,
|
| 24 |
+
"megatron_config": null,
|
| 25 |
+
"megatron_core": "megatron.core",
|
| 26 |
+
"modules_to_save": null,
|
| 27 |
+
"monteclora_config": null,
|
| 28 |
+
"peft_type": "LORA",
|
| 29 |
+
"peft_version": "0.21.2",
|
| 30 |
+
"qalora_group_size": 16,
|
| 31 |
+
"r": 64,
|
| 32 |
+
"rank_pattern": {},
|
| 33 |
+
"revision": null,
|
| 34 |
+
"target_modules": [
|
| 35 |
+
"k_proj",
|
| 36 |
+
"up_proj",
|
| 37 |
+
"down_proj",
|
| 38 |
+
"o_proj",
|
| 39 |
+
"gate_proj",
|
| 40 |
+
"v_proj",
|
| 41 |
+
"in_proj_qkv",
|
| 42 |
+
"in_proj_z",
|
| 43 |
+
"out_proj",
|
| 44 |
+
"q_proj"
|
| 45 |
+
],
|
| 46 |
+
"target_parameters": null,
|
| 47 |
+
"task_type": "CAUSAL_LM",
|
| 48 |
+
"trainable_token_indices": null,
|
| 49 |
+
"use_bdlora": null,
|
| 50 |
+
"use_dora": false,
|
| 51 |
+
"use_qalora": false,
|
| 52 |
+
"use_rslora": false,
|
| 53 |
+
"velora_config": null
|
| 54 |
+
}
|
checkpoints/sft_v2_lora/epoch3/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:36f689d567f30f12943bb2edff416f916201886481a98224cc44fc60b4d18c87
|
| 3 |
+
size 1740746640
|
checkpoints/sft_v2_lora/final/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: /mnt/raid5/models/Qwen3.8-27B
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:/mnt/raid5/models/Qwen3.8-27B
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.21.2
|
checkpoints/sft_v2_lora/final/adapter_config.json
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {},
|
| 4 |
+
"arrow_config": null,
|
| 5 |
+
"auto_mapping": null,
|
| 6 |
+
"base_model_name_or_path": "/mnt/raid5/models/Qwen3.8-27B",
|
| 7 |
+
"bias": "none",
|
| 8 |
+
"corda_config": null,
|
| 9 |
+
"ensure_weight_tying": false,
|
| 10 |
+
"eva_config": null,
|
| 11 |
+
"exclude_modules": null,
|
| 12 |
+
"fan_in_fan_out": false,
|
| 13 |
+
"inference_mode": true,
|
| 14 |
+
"init_lora_weights": true,
|
| 15 |
+
"kasa_config": null,
|
| 16 |
+
"layer_replication": null,
|
| 17 |
+
"layers_pattern": null,
|
| 18 |
+
"layers_to_transform": null,
|
| 19 |
+
"loftq_config": {},
|
| 20 |
+
"lora_alpha": 128,
|
| 21 |
+
"lora_bias": false,
|
| 22 |
+
"lora_dropout": 0.05,
|
| 23 |
+
"lora_ga_config": null,
|
| 24 |
+
"megatron_config": null,
|
| 25 |
+
"megatron_core": "megatron.core",
|
| 26 |
+
"modules_to_save": null,
|
| 27 |
+
"monteclora_config": null,
|
| 28 |
+
"peft_type": "LORA",
|
| 29 |
+
"peft_version": "0.21.2",
|
| 30 |
+
"qalora_group_size": 16,
|
| 31 |
+
"r": 64,
|
| 32 |
+
"rank_pattern": {},
|
| 33 |
+
"revision": null,
|
| 34 |
+
"target_modules": [
|
| 35 |
+
"k_proj",
|
| 36 |
+
"up_proj",
|
| 37 |
+
"down_proj",
|
| 38 |
+
"o_proj",
|
| 39 |
+
"gate_proj",
|
| 40 |
+
"v_proj",
|
| 41 |
+
"in_proj_qkv",
|
| 42 |
+
"in_proj_z",
|
| 43 |
+
"out_proj",
|
| 44 |
+
"q_proj"
|
| 45 |
+
],
|
| 46 |
+
"target_parameters": null,
|
| 47 |
+
"task_type": "CAUSAL_LM",
|
| 48 |
+
"trainable_token_indices": null,
|
| 49 |
+
"use_bdlora": null,
|
| 50 |
+
"use_dora": false,
|
| 51 |
+
"use_qalora": false,
|
| 52 |
+
"use_rslora": false,
|
| 53 |
+
"velora_config": null
|
| 54 |
+
}
|
checkpoints/sft_v2_lora/final/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:36f689d567f30f12943bb2edff416f916201886481a98224cc44fc60b4d18c87
|
| 3 |
+
size 1740746640
|
checkpoints/sft_v2_lora/manifest.json
ADDED
|
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"name": "sft_v2_lora",
|
| 3 |
+
"uploaded_at": "2026-10-15T13:26:29+0800",
|
| 4 |
+
"git_commit": "10b7252",
|
| 5 |
+
"source_dir": "/mnt/raid5/workspace/Agent_Coding_Active/ckpts/sft_v2_lora",
|
| 6 |
+
"kind": "model",
|
| 7 |
+
"base_model": "/mnt/raid5/models/Qwen3.8-27B",
|
| 8 |
+
"train_args": {
|
| 9 |
+
"model": "/mnt/raid5/models/Qwen3.8-27B",
|
| 10 |
+
"data": "sft_pipeline/out/icae_sft_v2.tokens.jsonl",
|
| 11 |
+
"out": "/mnt/raid5/workspace/Agent_Coding_Active/ckpts/sft_v2_lora",
|
| 12 |
+
"device": "cuda",
|
| 13 |
+
"attn": "sdpa",
|
| 14 |
+
"max_len": 32768,
|
| 15 |
+
"limit": 0,
|
| 16 |
+
"kinds": "",
|
| 17 |
+
"balance": "ask/feature=4,ask/protocol=2,explore=1,proceed=1",
|
| 18 |
+
"exclude_runs": "dev01",
|
| 19 |
+
"save_each_epoch": true,
|
| 20 |
+
"resume_adapter": "/mnt/raid5/workspace/Agent_Coding_Active/ckpts/sft_v2_lora/epoch1",
|
| 21 |
+
"resume_step": 56,
|
| 22 |
+
"loss_chunk": 2048,
|
| 23 |
+
"epochs": 3.0,
|
| 24 |
+
"max_steps": 0,
|
| 25 |
+
"lr": 0.0001,
|
| 26 |
+
"warmup": 0.05,
|
| 27 |
+
"grad_accum": 4,
|
| 28 |
+
"lora_r": 64,
|
| 29 |
+
"lora_alpha": 128,
|
| 30 |
+
"lora_dropout": 0.05,
|
| 31 |
+
"grad_ckpt": true,
|
| 32 |
+
"save_every": 0,
|
| 33 |
+
"seed": 0,
|
| 34 |
+
"log_every": 1,
|
| 35 |
+
"dry_truncate": false,
|
| 36 |
+
"dry_vocab_clamp": 0,
|
| 37 |
+
"data_stats": {
|
| 38 |
+
"rows": 223,
|
| 39 |
+
"unique_rows": 169,
|
| 40 |
+
"kinds": {
|
| 41 |
+
"proceed": 61,
|
| 42 |
+
"explore": 76,
|
| 43 |
+
"ask": 86
|
| 44 |
+
},
|
| 45 |
+
"groups": {
|
| 46 |
+
"proceed": 61,
|
| 47 |
+
"explore": 76,
|
| 48 |
+
"ask/feature": 44,
|
| 49 |
+
"ask/protocol": 42
|
| 50 |
+
},
|
| 51 |
+
"tokens_total": 3784330,
|
| 52 |
+
"tokens_max": 32758,
|
| 53 |
+
"labelled_total": 369828,
|
| 54 |
+
"labelled_mean": 1658
|
| 55 |
+
},
|
| 56 |
+
"total_steps": 168
|
| 57 |
+
},
|
| 58 |
+
"n_files": 15,
|
| 59 |
+
"total_bytes": 6963035127,
|
| 60 |
+
"note": "SFT v2 cold start, sftv2"
|
| 61 |
+
}
|
checkpoints/sft_v2_lora/train_args.json
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model": "/mnt/raid5/models/Qwen3.8-27B",
|
| 3 |
+
"data": "sft_pipeline/out/icae_sft_v2.tokens.jsonl",
|
| 4 |
+
"out": "/mnt/raid5/workspace/Agent_Coding_Active/ckpts/sft_v2_lora",
|
| 5 |
+
"device": "cuda",
|
| 6 |
+
"attn": "sdpa",
|
| 7 |
+
"max_len": 32768,
|
| 8 |
+
"limit": 0,
|
| 9 |
+
"kinds": "",
|
| 10 |
+
"balance": "ask/feature=4,ask/protocol=2,explore=1,proceed=1",
|
| 11 |
+
"exclude_runs": "dev01",
|
| 12 |
+
"save_each_epoch": true,
|
| 13 |
+
"resume_adapter": "/mnt/raid5/workspace/Agent_Coding_Active/ckpts/sft_v2_lora/epoch1",
|
| 14 |
+
"resume_step": 56,
|
| 15 |
+
"loss_chunk": 2048,
|
| 16 |
+
"epochs": 3.0,
|
| 17 |
+
"max_steps": 0,
|
| 18 |
+
"lr": 0.0001,
|
| 19 |
+
"warmup": 0.05,
|
| 20 |
+
"grad_accum": 4,
|
| 21 |
+
"lora_r": 64,
|
| 22 |
+
"lora_alpha": 128,
|
| 23 |
+
"lora_dropout": 0.05,
|
| 24 |
+
"grad_ckpt": true,
|
| 25 |
+
"save_every": 0,
|
| 26 |
+
"seed": 0,
|
| 27 |
+
"log_every": 1,
|
| 28 |
+
"dry_truncate": false,
|
| 29 |
+
"dry_vocab_clamp": 0,
|
| 30 |
+
"data_stats": {
|
| 31 |
+
"rows": 223,
|
| 32 |
+
"unique_rows": 169,
|
| 33 |
+
"kinds": {
|
| 34 |
+
"proceed": 61,
|
| 35 |
+
"explore": 76,
|
| 36 |
+
"ask": 86
|
| 37 |
+
},
|
| 38 |
+
"groups": {
|
| 39 |
+
"proceed": 61,
|
| 40 |
+
"explore": 76,
|
| 41 |
+
"ask/feature": 44,
|
| 42 |
+
"ask/protocol": 42
|
| 43 |
+
},
|
| 44 |
+
"tokens_total": 3784330,
|
| 45 |
+
"tokens_max": 32758,
|
| 46 |
+
"labelled_total": 369828,
|
| 47 |
+
"labelled_mean": 1658
|
| 48 |
+
},
|
| 49 |
+
"total_steps": 168
|
| 50 |
+
}
|
checkpoints/sft_v2_lora/train_log.jsonl
ADDED
|
@@ -0,0 +1,198 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"step": 1, "loss": 0.3537, "labelled_tokens": 662, "lr": 2.5e-05, "elapsed_s": 130}
|
| 2 |
+
{"step": 2, "loss": 0.1313, "labelled_tokens": 4277, "lr": 3.7500000000000003e-05, "elapsed_s": 228}
|
| 3 |
+
{"step": 3, "loss": 0.2081, "labelled_tokens": 5818, "lr": 5e-05, "elapsed_s": 381}
|
| 4 |
+
{"step": 4, "loss": 0.3613, "labelled_tokens": 3689, "lr": 6.25e-05, "elapsed_s": 483}
|
| 5 |
+
{"step": 5, "loss": 0.1559, "labelled_tokens": 3170, "lr": 7.500000000000001e-05, "elapsed_s": 592}
|
| 6 |
+
{"step": 6, "loss": 0.2243, "labelled_tokens": 3380, "lr": 8.75e-05, "elapsed_s": 724}
|
| 7 |
+
{"step": 7, "loss": 0.1813, "labelled_tokens": 7301, "lr": 0.0001, "elapsed_s": 875}
|
| 8 |
+
{"step": 8, "loss": 0.2604, "labelled_tokens": 2002, "lr": 0.0001, "elapsed_s": 967}
|
| 9 |
+
{"step": 9, "loss": 0.4132, "labelled_tokens": 3834, "lr": 9.999036202410325e-05, "elapsed_s": 1071}
|
| 10 |
+
{"step": 10, "loss": 0.134, "labelled_tokens": 4367, "lr": 9.996145181203615e-05, "elapsed_s": 1193}
|
| 11 |
+
{"step": 11, "loss": 0.4138, "labelled_tokens": 3786, "lr": 9.991328050923581e-05, "elapsed_s": 1273}
|
| 12 |
+
{"step": 12, "loss": 0.2207, "labelled_tokens": 12809, "lr": 9.98458666866564e-05, "elapsed_s": 1409}
|
| 13 |
+
{"step": 13, "loss": 0.2697, "labelled_tokens": 11382, "lr": 9.975923633360985e-05, "elapsed_s": 1519}
|
| 14 |
+
{"step": 14, "loss": 0.1923, "labelled_tokens": 11077, "lr": 9.965342284774632e-05, "elapsed_s": 1663}
|
| 15 |
+
{"step": 15, "loss": 0.1117, "labelled_tokens": 5018, "lr": 9.952846702217886e-05, "elapsed_s": 1829}
|
| 16 |
+
{"step": 16, "loss": 0.2935, "labelled_tokens": 5841, "lr": 9.938441702975689e-05, "elapsed_s": 1968}
|
| 17 |
+
{"step": 17, "loss": 0.1754, "labelled_tokens": 10515, "lr": 9.922132840449459e-05, "elapsed_s": 2075}
|
| 18 |
+
{"step": 18, "loss": 0.1738, "labelled_tokens": 3469, "lr": 9.903926402016153e-05, "elapsed_s": 2173}
|
| 19 |
+
{"step": 19, "loss": 0.3035, "labelled_tokens": 4637, "lr": 9.883829406604363e-05, "elapsed_s": 2324}
|
| 20 |
+
{"step": 20, "loss": 0.3, "labelled_tokens": 17099, "lr": 9.861849601988383e-05, "elapsed_s": 2456}
|
| 21 |
+
{"step": 21, "loss": 0.1433, "labelled_tokens": 5045, "lr": 9.837995461801299e-05, "elapsed_s": 2589}
|
| 22 |
+
{"step": 22, "loss": 0.2908, "labelled_tokens": 1819, "lr": 9.812276182268236e-05, "elapsed_s": 2643}
|
| 23 |
+
{"step": 23, "loss": 0.1991, "labelled_tokens": 10107, "lr": 9.784701678661045e-05, "elapsed_s": 2799}
|
| 24 |
+
{"step": 24, "loss": 0.2258, "labelled_tokens": 4486, "lr": 9.755282581475769e-05, "elapsed_s": 2914}
|
| 25 |
+
{"step": 25, "loss": 0.4163, "labelled_tokens": 8947, "lr": 9.724030232334391e-05, "elapsed_s": 2980}
|
| 26 |
+
{"step": 26, "loss": 0.3024, "labelled_tokens": 12390, "lr": 9.690956679612421e-05, "elapsed_s": 3071}
|
| 27 |
+
{"step": 27, "loss": 0.2275, "labelled_tokens": 10197, "lr": 9.656074673794018e-05, "elapsed_s": 3173}
|
| 28 |
+
{"step": 28, "loss": 0.2581, "labelled_tokens": 4058, "lr": 9.619397662556435e-05, "elapsed_s": 3224}
|
| 29 |
+
{"step": 29, "loss": 0.1192, "labelled_tokens": 3382, "lr": 9.580939785585681e-05, "elapsed_s": 3274}
|
| 30 |
+
{"step": 30, "loss": 0.2257, "labelled_tokens": 4578, "lr": 9.540715869125407e-05, "elapsed_s": 3395}
|
| 31 |
+
{"step": 31, "loss": 0.1437, "labelled_tokens": 3699, "lr": 9.498741420261108e-05, "elapsed_s": 3528}
|
| 32 |
+
{"step": 32, "loss": 0.1646, "labelled_tokens": 12666, "lr": 9.45503262094184e-05, "elapsed_s": 3613}
|
| 33 |
+
{"step": 33, "loss": 0.2221, "labelled_tokens": 5447, "lr": 9.409606321741775e-05, "elapsed_s": 3721}
|
| 34 |
+
{"step": 34, "loss": 0.331, "labelled_tokens": 5707, "lr": 9.362480035363986e-05, "elapsed_s": 3838}
|
| 35 |
+
{"step": 35, "loss": 0.1762, "labelled_tokens": 9906, "lr": 9.31367192988896e-05, "elapsed_s": 3969}
|
| 36 |
+
{"step": 36, "loss": 0.1955, "labelled_tokens": 20194, "lr": 9.263200821770461e-05, "elapsed_s": 4127}
|
| 37 |
+
{"step": 37, "loss": 0.1505, "labelled_tokens": 3882, "lr": 9.211086168581433e-05, "elapsed_s": 4234}
|
| 38 |
+
{"step": 38, "loss": 0.1396, "labelled_tokens": 9287, "lr": 9.157348061512727e-05, "elapsed_s": 4358}
|
| 39 |
+
{"step": 39, "loss": 0.2546, "labelled_tokens": 1895, "lr": 9.102007217627568e-05, "elapsed_s": 4452}
|
| 40 |
+
{"step": 40, "loss": 0.154, "labelled_tokens": 3250, "lr": 9.045084971874738e-05, "elapsed_s": 4553}
|
| 41 |
+
{"step": 41, "loss": 0.1974, "labelled_tokens": 5524, "lr": 8.986603268863536e-05, "elapsed_s": 4647}
|
| 42 |
+
{"step": 42, "loss": 0.1287, "labelled_tokens": 2317, "lr": 8.926584654403724e-05, "elapsed_s": 4766}
|
| 43 |
+
{"step": 43, "loss": 0.2405, "labelled_tokens": 8645, "lr": 8.865052266813685e-05, "elapsed_s": 4924}
|
| 44 |
+
{"step": 44, "loss": 0.285, "labelled_tokens": 3442, "lr": 8.802029828000156e-05, "elapsed_s": 5045}
|
| 45 |
+
{"step": 45, "loss": 0.109, "labelled_tokens": 4948, "lr": 8.737541634312985e-05, "elapsed_s": 5173}
|
| 46 |
+
{"step": 46, "loss": 0.3477, "labelled_tokens": 5998, "lr": 8.671612547178428e-05, "elapsed_s": 5228}
|
| 47 |
+
{"step": 47, "loss": 0.2029, "labelled_tokens": 12830, "lr": 8.604267983514594e-05, "elapsed_s": 5358}
|
| 48 |
+
{"step": 48, "loss": 0.1458, "labelled_tokens": 10145, "lr": 8.535533905932738e-05, "elapsed_s": 5492}
|
| 49 |
+
{"step": 49, "loss": 0.2814, "labelled_tokens": 5748, "lr": 8.46543681272818e-05, "elapsed_s": 5615}
|
| 50 |
+
{"step": 50, "loss": 0.1019, "labelled_tokens": 1895, "lr": 8.39400372766471e-05, "elapsed_s": 5707}
|
| 51 |
+
{"step": 51, "loss": 0.1371, "labelled_tokens": 5914, "lr": 8.321262189556409e-05, "elapsed_s": 5759}
|
| 52 |
+
{"step": 52, "loss": 0.2487, "labelled_tokens": 5099, "lr": 8.247240241650918e-05, "elapsed_s": 5853}
|
| 53 |
+
{"step": 53, "loss": 0.3506, "labelled_tokens": 5943, "lr": 8.171966420818228e-05, "elapsed_s": 5998}
|
| 54 |
+
{"step": 54, "loss": 0.0871, "labelled_tokens": 6939, "lr": 8.095469746549172e-05, "elapsed_s": 6123}
|
| 55 |
+
{"step": 55, "loss": 0.1554, "labelled_tokens": 2497, "lr": 8.017779709767858e-05, "elapsed_s": 6237}
|
| 56 |
+
{"step": 56, "loss": 0.2093, "labelled_tokens": 17441, "lr": 7.938926261462366e-05, "elapsed_s": 6349}
|
| 57 |
+
{"step": 57, "loss": 0.1582, "labelled_tokens": 17116, "lr": 7.858939801138061e-05, "elapsed_s": 6554}
|
| 58 |
+
{"step": 58, "loss": 0.1455, "labelled_tokens": 15993, "lr": 7.777851165098012e-05, "elapsed_s": 6728}
|
| 59 |
+
{"step": 59, "loss": 0.0639, "labelled_tokens": 1807, "lr": 7.695691614555003e-05, "elapsed_s": 6779}
|
| 60 |
+
{"step": 60, "loss": 0.0613, "labelled_tokens": 6795, "lr": 7.612492823579745e-05, "elapsed_s": 6898}
|
| 61 |
+
{"step": 61, "loss": 0.0888, "labelled_tokens": 3969, "lr": 7.528286866889924e-05, "elapsed_s": 7019}
|
| 62 |
+
{"step": 62, "loss": 0.0939, "labelled_tokens": 5884, "lr": 7.443106207484776e-05, "elapsed_s": 7120}
|
| 63 |
+
{"step": 63, "loss": 0.3145, "labelled_tokens": 4515, "lr": 7.35698368412999e-05, "elapsed_s": 7233}
|
| 64 |
+
{"step": 64, "loss": 0.1151, "labelled_tokens": 9396, "lr": 7.269952498697734e-05, "elapsed_s": 7363}
|
| 65 |
+
{"step": 65, "loss": 0.0589, "labelled_tokens": 3266, "lr": 7.18204620336671e-05, "elapsed_s": 7458}
|
| 66 |
+
{"step": 66, "loss": 0.0775, "labelled_tokens": 9933, "lr": 7.09329868768714e-05, "elapsed_s": 7572}
|
| 67 |
+
{"step": 67, "loss": 0.1885, "labelled_tokens": 7753, "lr": 7.003744165515705e-05, "elapsed_s": 7710}
|
| 68 |
+
{"step": 68, "loss": 0.1138, "labelled_tokens": 11905, "lr": 6.91341716182545e-05, "elapsed_s": 7821}
|
| 69 |
+
{"step": 69, "loss": 0.0709, "labelled_tokens": 4079, "lr": 6.82235249939575e-05, "elapsed_s": 7964}
|
| 70 |
+
{"step": 70, "loss": 0.0324, "labelled_tokens": 3948, "lr": 6.730585285387465e-05, "elapsed_s": 8073}
|
| 71 |
+
{"step": 71, "loss": 0.3224, "labelled_tokens": 3478, "lr": 6.638150897808468e-05, "elapsed_s": 8130}
|
| 72 |
+
{"step": 72, "loss": 0.1066, "labelled_tokens": 3841, "lr": 6.545084971874738e-05, "elapsed_s": 8220}
|
| 73 |
+
{"step": 73, "loss": 0.15, "labelled_tokens": 2368, "lr": 6.451423386272312e-05, "elapsed_s": 8280}
|
| 74 |
+
{"step": 74, "loss": 0.2361, "labelled_tokens": 6098, "lr": 6.357202249325371e-05, "elapsed_s": 8362}
|
| 75 |
+
{"step": 75, "loss": 0.1019, "labelled_tokens": 1815, "lr": 6.26245788507579e-05, "elapsed_s": 8461}
|
| 76 |
+
{"step": 76, "loss": 0.0635, "labelled_tokens": 4825, "lr": 6.167226819279528e-05, "elapsed_s": 8598}
|
| 77 |
+
{"step": 77, "loss": 0.246, "labelled_tokens": 2626, "lr": 6.071545765325254e-05, "elapsed_s": 8691}
|
| 78 |
+
{"step": 78, "loss": 0.1806, "labelled_tokens": 5295, "lr": 5.9754516100806423e-05, "elapsed_s": 8815}
|
| 79 |
+
{"step": 79, "loss": 0.1466, "labelled_tokens": 2345, "lr": 5.8789813996717736e-05, "elapsed_s": 8968}
|
| 80 |
+
{"step": 80, "loss": 0.1591, "labelled_tokens": 6435, "lr": 5.782172325201155e-05, "elapsed_s": 9086}
|
| 81 |
+
{"step": 81, "loss": 0.1119, "labelled_tokens": 3303, "lr": 5.685061708409841e-05, "elapsed_s": 9219}
|
| 82 |
+
{"step": 82, "loss": 0.0736, "labelled_tokens": 2601, "lr": 5.587686987289189e-05, "elapsed_s": 9345}
|
| 83 |
+
{"step": 83, "loss": 0.1557, "labelled_tokens": 10207, "lr": 5.490085701647805e-05, "elapsed_s": 9501}
|
| 84 |
+
{"step": 84, "loss": 0.1322, "labelled_tokens": 3773, "lr": 5.392295478639225e-05, "elapsed_s": 9619}
|
| 85 |
+
{"step": 85, "loss": 0.1093, "labelled_tokens": 6183, "lr": 5.294354018255945e-05, "elapsed_s": 9743}
|
| 86 |
+
{"step": 86, "loss": 0.1643, "labelled_tokens": 11833, "lr": 5.196299078795344e-05, "elapsed_s": 9911}
|
| 87 |
+
{"step": 57, "loss": 0.2157, "labelled_tokens": 2096, "lr": 7.858939801138061e-05, "elapsed_s": 86}
|
| 88 |
+
{"step": 58, "loss": 0.0523, "labelled_tokens": 6184, "lr": 7.777851165098012e-05, "elapsed_s": 222}
|
| 89 |
+
{"step": 59, "loss": 0.1969, "labelled_tokens": 4929, "lr": 7.695691614555003e-05, "elapsed_s": 332}
|
| 90 |
+
{"step": 60, "loss": 0.0624, "labelled_tokens": 1171, "lr": 7.612492823579745e-05, "elapsed_s": 421}
|
| 91 |
+
{"step": 61, "loss": 0.3319, "labelled_tokens": 15737, "lr": 7.528286866889924e-05, "elapsed_s": 544}
|
| 92 |
+
{"step": 62, "loss": 0.0877, "labelled_tokens": 6357, "lr": 7.443106207484776e-05, "elapsed_s": 657}
|
| 93 |
+
{"step": 63, "loss": 0.3262, "labelled_tokens": 6295, "lr": 7.35698368412999e-05, "elapsed_s": 700}
|
| 94 |
+
{"step": 64, "loss": 0.3606, "labelled_tokens": 4984, "lr": 7.269952498697734e-05, "elapsed_s": 823}
|
| 95 |
+
{"step": 65, "loss": 0.1136, "labelled_tokens": 3859, "lr": 7.18204620336671e-05, "elapsed_s": 934}
|
| 96 |
+
{"step": 66, "loss": 0.1446, "labelled_tokens": 6344, "lr": 7.09329868768714e-05, "elapsed_s": 1047}
|
| 97 |
+
{"step": 67, "loss": 0.1303, "labelled_tokens": 9953, "lr": 7.003744165515705e-05, "elapsed_s": 1194}
|
| 98 |
+
{"step": 68, "loss": 0.1574, "labelled_tokens": 10332, "lr": 6.91341716182545e-05, "elapsed_s": 1325}
|
| 99 |
+
{"step": 69, "loss": 0.1224, "labelled_tokens": 5674, "lr": 6.82235249939575e-05, "elapsed_s": 1402}
|
| 100 |
+
{"step": 70, "loss": 0.3702, "labelled_tokens": 3964, "lr": 6.730585285387465e-05, "elapsed_s": 1438}
|
| 101 |
+
{"step": 71, "loss": 0.1432, "labelled_tokens": 3216, "lr": 6.638150897808468e-05, "elapsed_s": 1496}
|
| 102 |
+
{"step": 72, "loss": 0.2553, "labelled_tokens": 3943, "lr": 6.545084971874738e-05, "elapsed_s": 1561}
|
| 103 |
+
{"step": 73, "loss": 0.3204, "labelled_tokens": 7908, "lr": 6.451423386272312e-05, "elapsed_s": 1641}
|
| 104 |
+
{"step": 74, "loss": 0.1379, "labelled_tokens": 2347, "lr": 6.357202249325371e-05, "elapsed_s": 1698}
|
| 105 |
+
{"step": 75, "loss": 0.0505, "labelled_tokens": 1871, "lr": 6.26245788507579e-05, "elapsed_s": 1815}
|
| 106 |
+
{"step": 76, "loss": 0.1003, "labelled_tokens": 5431, "lr": 6.167226819279528e-05, "elapsed_s": 1958}
|
| 107 |
+
{"step": 77, "loss": 0.1285, "labelled_tokens": 11729, "lr": 6.071545765325254e-05, "elapsed_s": 2103}
|
| 108 |
+
{"step": 78, "loss": 0.2299, "labelled_tokens": 7974, "lr": 5.9754516100806423e-05, "elapsed_s": 2214}
|
| 109 |
+
{"step": 79, "loss": 0.0816, "labelled_tokens": 6763, "lr": 5.8789813996717736e-05, "elapsed_s": 2373}
|
| 110 |
+
{"step": 80, "loss": 0.1076, "labelled_tokens": 12081, "lr": 5.782172325201155e-05, "elapsed_s": 2502}
|
| 111 |
+
{"step": 81, "loss": 0.0836, "labelled_tokens": 6544, "lr": 5.685061708409841e-05, "elapsed_s": 2610}
|
| 112 |
+
{"step": 82, "loss": 0.3389, "labelled_tokens": 4641, "lr": 5.587686987289189e-05, "elapsed_s": 2702}
|
| 113 |
+
{"step": 83, "loss": 0.0357, "labelled_tokens": 1239, "lr": 5.490085701647805e-05, "elapsed_s": 2801}
|
| 114 |
+
{"step": 84, "loss": 0.1116, "labelled_tokens": 6650, "lr": 5.392295478639225e-05, "elapsed_s": 2930}
|
| 115 |
+
{"step": 85, "loss": 0.0747, "labelled_tokens": 4270, "lr": 5.294354018255945e-05, "elapsed_s": 3086}
|
| 116 |
+
{"step": 86, "loss": 0.1236, "labelled_tokens": 9680, "lr": 5.196299078795344e-05, "elapsed_s": 3189}
|
| 117 |
+
{"step": 87, "loss": 0.1215, "labelled_tokens": 4045, "lr": 5.0981684623031415e-05, "elapsed_s": 3277}
|
| 118 |
+
{"step": 88, "loss": 0.0894, "labelled_tokens": 4774, "lr": 5e-05, "elapsed_s": 3366}
|
| 119 |
+
{"step": 89, "loss": 0.1367, "labelled_tokens": 9998, "lr": 4.901831537696859e-05, "elapsed_s": 3488}
|
| 120 |
+
{"step": 90, "loss": 0.1073, "labelled_tokens": 8007, "lr": 4.8037009212046586e-05, "elapsed_s": 3639}
|
| 121 |
+
{"step": 91, "loss": 0.1563, "labelled_tokens": 15472, "lr": 4.7056459817440544e-05, "elapsed_s": 3757}
|
| 122 |
+
{"step": 92, "loss": 0.0348, "labelled_tokens": 2628, "lr": 4.607704521360776e-05, "elapsed_s": 3847}
|
| 123 |
+
{"step": 93, "loss": 0.0812, "labelled_tokens": 8247, "lr": 4.509914298352197e-05, "elapsed_s": 3936}
|
| 124 |
+
{"step": 94, "loss": 0.2059, "labelled_tokens": 1503, "lr": 4.412313012710813e-05, "elapsed_s": 4040}
|
| 125 |
+
{"step": 95, "loss": 0.021, "labelled_tokens": 4261, "lr": 4.3149382915901606e-05, "elapsed_s": 4174}
|
| 126 |
+
{"step": 96, "loss": 0.1293, "labelled_tokens": 5896, "lr": 4.2178276747988446e-05, "elapsed_s": 4337}
|
| 127 |
+
{"step": 97, "loss": 0.1761, "labelled_tokens": 2774, "lr": 4.1210186003282275e-05, "elapsed_s": 4453}
|
| 128 |
+
{"step": 98, "loss": 0.1225, "labelled_tokens": 11000, "lr": 4.0245483899193595e-05, "elapsed_s": 4580}
|
| 129 |
+
{"step": 99, "loss": 0.1137, "labelled_tokens": 3060, "lr": 3.928454234674747e-05, "elapsed_s": 4772}
|
| 130 |
+
{"step": 100, "loss": 0.1367, "labelled_tokens": 9892, "lr": 3.832773180720475e-05, "elapsed_s": 4966}
|
| 131 |
+
{"step": 101, "loss": 0.0762, "labelled_tokens": 1811, "lr": 3.73754211492421e-05, "elapsed_s": 5031}
|
| 132 |
+
{"step": 102, "loss": 0.0973, "labelled_tokens": 5340, "lr": 3.642797750674629e-05, "elapsed_s": 5214}
|
| 133 |
+
{"step": 103, "loss": 0.1288, "labelled_tokens": 8261, "lr": 3.5485766137276894e-05, "elapsed_s": 5330}
|
| 134 |
+
{"step": 104, "loss": 0.2213, "labelled_tokens": 5325, "lr": 3.4549150281252636e-05, "elapsed_s": 5443}
|
| 135 |
+
{"step": 105, "loss": 0.2085, "labelled_tokens": 18878, "lr": 3.361849102191533e-05, "elapsed_s": 5584}
|
| 136 |
+
{"step": 106, "loss": 0.1484, "labelled_tokens": 11633, "lr": 3.2694147146125345e-05, "elapsed_s": 5775}
|
| 137 |
+
{"step": 107, "loss": 0.041, "labelled_tokens": 1209, "lr": 3.177647500604252e-05, "elapsed_s": 5897}
|
| 138 |
+
{"step": 108, "loss": 0.11, "labelled_tokens": 2843, "lr": 3.086582838174551e-05, "elapsed_s": 6011}
|
| 139 |
+
{"step": 109, "loss": 0.1146, "labelled_tokens": 7354, "lr": 2.996255834484296e-05, "elapsed_s": 6166}
|
| 140 |
+
{"step": 110, "loss": 0.2059, "labelled_tokens": 18171, "lr": 2.9067013123128613e-05, "elapsed_s": 6370}
|
| 141 |
+
{"step": 111, "loss": 0.1165, "labelled_tokens": 9868, "lr": 2.8179537966332887e-05, "elapsed_s": 6498}
|
| 142 |
+
{"step": 112, "loss": 0.103, "labelled_tokens": 3608, "lr": 2.7300475013022663e-05, "elapsed_s": 6637}
|
| 143 |
+
{"step": 113, "loss": 0.08, "labelled_tokens": 2062, "lr": 2.6430163158700115e-05, "elapsed_s": 6737}
|
| 144 |
+
{"step": 114, "loss": 0.0946, "labelled_tokens": 9365, "lr": 2.556893792515227e-05, "elapsed_s": 6868}
|
| 145 |
+
{"step": 115, "loss": 0.0628, "labelled_tokens": 4898, "lr": 2.471713133110078e-05, "elapsed_s": 6984}
|
| 146 |
+
{"step": 116, "loss": 0.0223, "labelled_tokens": 3490, "lr": 2.3875071764202563e-05, "elapsed_s": 7092}
|
| 147 |
+
{"step": 117, "loss": 0.0191, "labelled_tokens": 8765, "lr": 2.3043083854449988e-05, "elapsed_s": 7240}
|
| 148 |
+
{"step": 118, "loss": 0.0413, "labelled_tokens": 3323, "lr": 2.2221488349019903e-05, "elapsed_s": 7352}
|
| 149 |
+
{"step": 119, "loss": 0.0676, "labelled_tokens": 3741, "lr": 2.1410601988619394e-05, "elapsed_s": 7479}
|
| 150 |
+
{"step": 120, "loss": 0.0145, "labelled_tokens": 7496, "lr": 2.061073738537635e-05, "elapsed_s": 7568}
|
| 151 |
+
{"step": 121, "loss": 0.0807, "labelled_tokens": 2035, "lr": 1.982220290232143e-05, "elapsed_s": 7654}
|
| 152 |
+
{"step": 122, "loss": 0.2671, "labelled_tokens": 9030, "lr": 1.9045302534508297e-05, "elapsed_s": 7792}
|
| 153 |
+
{"step": 123, "loss": 0.1251, "labelled_tokens": 12125, "lr": 1.8280335791817733e-05, "elapsed_s": 7897}
|
| 154 |
+
{"step": 124, "loss": 0.1005, "labelled_tokens": 14476, "lr": 1.7527597583490822e-05, "elapsed_s": 8001}
|
| 155 |
+
{"step": 125, "loss": 0.0937, "labelled_tokens": 3897, "lr": 1.678737810443593e-05, "elapsed_s": 8069}
|
| 156 |
+
{"step": 126, "loss": 0.0328, "labelled_tokens": 3897, "lr": 1.605996272335291e-05, "elapsed_s": 8168}
|
| 157 |
+
{"step": 127, "loss": 0.1218, "labelled_tokens": 9766, "lr": 1.5345631872718214e-05, "elapsed_s": 8342}
|
| 158 |
+
{"step": 128, "loss": 0.0921, "labelled_tokens": 6737, "lr": 1.4644660940672627e-05, "elapsed_s": 8478}
|
| 159 |
+
{"step": 129, "loss": 0.0856, "labelled_tokens": 9407, "lr": 1.3957320164854059e-05, "elapsed_s": 8619}
|
| 160 |
+
{"step": 130, "loss": 0.0452, "labelled_tokens": 6179, "lr": 1.3283874528215733e-05, "elapsed_s": 8757}
|
| 161 |
+
{"step": 131, "loss": 0.1358, "labelled_tokens": 10664, "lr": 1.2624583656870154e-05, "elapsed_s": 8909}
|
| 162 |
+
{"step": 132, "loss": 0.2056, "labelled_tokens": 7400, "lr": 1.1979701719998453e-05, "elapsed_s": 9031}
|
| 163 |
+
{"step": 133, "loss": 0.2211, "labelled_tokens": 6238, "lr": 1.134947733186315e-05, "elapsed_s": 9121}
|
| 164 |
+
{"step": 134, "loss": 0.0158, "labelled_tokens": 2538, "lr": 1.0734153455962765e-05, "elapsed_s": 9241}
|
| 165 |
+
{"step": 135, "loss": 0.0588, "labelled_tokens": 3150, "lr": 1.013396731136465e-05, "elapsed_s": 9339}
|
| 166 |
+
{"step": 136, "loss": 0.0315, "labelled_tokens": 6325, "lr": 9.549150281252633e-06, "elapsed_s": 9466}
|
| 167 |
+
{"step": 137, "loss": 0.2168, "labelled_tokens": 5629, "lr": 8.97992782372432e-06, "elapsed_s": 9574}
|
| 168 |
+
{"step": 138, "loss": 0.0404, "labelled_tokens": 5168, "lr": 8.426519384872733e-06, "elapsed_s": 9684}
|
| 169 |
+
{"step": 139, "loss": 0.0834, "labelled_tokens": 15183, "lr": 7.889138314185678e-06, "elapsed_s": 9793}
|
| 170 |
+
{"step": 140, "loss": 0.0946, "labelled_tokens": 7830, "lr": 7.367991782295391e-06, "elapsed_s": 9872}
|
| 171 |
+
{"step": 141, "loss": 0.0472, "labelled_tokens": 5554, "lr": 6.863280701110408e-06, "elapsed_s": 10006}
|
| 172 |
+
{"step": 142, "loss": 0.2212, "labelled_tokens": 3864, "lr": 6.375199646360142e-06, "elapsed_s": 10083}
|
| 173 |
+
{"step": 143, "loss": 0.0933, "labelled_tokens": 4872, "lr": 5.903936782582253e-06, "elapsed_s": 10143}
|
| 174 |
+
{"step": 144, "loss": 0.2881, "labelled_tokens": 11111, "lr": 5.449673790581611e-06, "elapsed_s": 10218}
|
| 175 |
+
{"step": 145, "loss": 0.1046, "labelled_tokens": 10767, "lr": 5.012585797388936e-06, "elapsed_s": 10360}
|
| 176 |
+
{"step": 146, "loss": 0.0478, "labelled_tokens": 6260, "lr": 4.592841308745932e-06, "elapsed_s": 10470}
|
| 177 |
+
{"step": 147, "loss": 0.0213, "labelled_tokens": 1251, "lr": 4.190602144143207e-06, "elapsed_s": 10538}
|
| 178 |
+
{"step": 148, "loss": 0.2567, "labelled_tokens": 5103, "lr": 3.8060233744356633e-06, "elapsed_s": 10659}
|
| 179 |
+
{"step": 149, "loss": 0.078, "labelled_tokens": 3442, "lr": 3.4392532620598216e-06, "elapsed_s": 10802}
|
| 180 |
+
{"step": 150, "loss": 0.1559, "labelled_tokens": 6346, "lr": 3.0904332038757977e-06, "elapsed_s": 10903}
|
| 181 |
+
{"step": 151, "loss": 0.0212, "labelled_tokens": 5443, "lr": 2.759697676656098e-06, "elapsed_s": 11006}
|
| 182 |
+
{"step": 152, "loss": 0.0052, "labelled_tokens": 4903, "lr": 2.4471741852423237e-06, "elapsed_s": 11149}
|
| 183 |
+
{"step": 153, "loss": 0.2046, "labelled_tokens": 21139, "lr": 2.152983213389559e-06, "elapsed_s": 11257}
|
| 184 |
+
{"step": 154, "loss": 0.1051, "labelled_tokens": 9897, "lr": 1.8772381773176417e-06, "elapsed_s": 11377}
|
| 185 |
+
{"step": 155, "loss": 0.1094, "labelled_tokens": 3696, "lr": 1.620045381987012e-06, "elapsed_s": 11515}
|
| 186 |
+
{"step": 156, "loss": 0.0183, "labelled_tokens": 2205, "lr": 1.3815039801161721e-06, "elapsed_s": 11574}
|
| 187 |
+
{"step": 157, "loss": 0.0382, "labelled_tokens": 2116, "lr": 1.1617059339563807e-06, "elapsed_s": 11640}
|
| 188 |
+
{"step": 158, "loss": 0.0437, "labelled_tokens": 1399, "lr": 9.607359798384785e-07, "elapsed_s": 11729}
|
| 189 |
+
{"step": 159, "loss": 0.0058, "labelled_tokens": 5004, "lr": 7.786715955054203e-07, "elapsed_s": 11850}
|
| 190 |
+
{"step": 160, "loss": 0.0924, "labelled_tokens": 10401, "lr": 6.15582970243117e-07, "elapsed_s": 11971}
|
| 191 |
+
{"step": 161, "loss": 0.0152, "labelled_tokens": 4128, "lr": 4.715329778211375e-07, "elapsed_s": 12026}
|
| 192 |
+
{"step": 162, "loss": 0.0893, "labelled_tokens": 9127, "lr": 3.465771522536854e-07, "elapsed_s": 12136}
|
| 193 |
+
{"step": 163, "loss": 0.1796, "labelled_tokens": 19709, "lr": 2.407636663901591e-07, "elapsed_s": 12277}
|
| 194 |
+
{"step": 164, "loss": 0.1558, "labelled_tokens": 9654, "lr": 1.5413331334360182e-07, "elapsed_s": 12367}
|
| 195 |
+
{"step": 165, "loss": 0.0506, "labelled_tokens": 4461, "lr": 8.671949076420882e-08, "elapsed_s": 12500}
|
| 196 |
+
{"step": 166, "loss": 0.0993, "labelled_tokens": 1111, "lr": 3.8548187963854956e-08, "elapsed_s": 12554}
|
| 197 |
+
{"step": 167, "loss": 0.0794, "labelled_tokens": 3872, "lr": 9.637975896759077e-09, "elapsed_s": 12620}
|
| 198 |
+
{"step": 168, "loss": 0.0543, "labelled_tokens": 4032, "lr": 0.0, "elapsed_s": 12704}
|