| --- |
| base_model: |
| - mllm-dev/gpt2_f_experiment_5 |
| - mllm-dev/gpt2_f_experiment_2 |
| - mllm-dev/gpt2_f_experiment_9 |
| - mllm-dev/gpt2_f_experiment_6 |
| - mllm-dev/gpt2_f_experiment_8 |
| - mllm-dev/gpt2_f_experiment_0 |
| - mllm-dev/gpt2_f_experiment_3 |
| - mllm-dev/gpt2_f_experiment_1 |
| - mllm-dev/gpt2_f_experiment_7 |
| - mllm-dev/gpt2_f_experiment_4 |
| library_name: transformers |
| tags: |
| - mergekit |
| - merge |
|
|
| --- |
| # sean_test_merge_out |
| |
| This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). |
| |
| ## Merge Details |
| ### Merge Method |
| |
| This model was merged using the [linear](https://arxiv.org/abs/2203.05482) merge method. |
| |
| ### Models Merged |
| |
| The following models were included in the merge: |
| * [mllm-dev/gpt2_f_experiment_5](https://huggingface.co/mllm-dev/gpt2_f_experiment_5) |
| * [mllm-dev/gpt2_f_experiment_2](https://huggingface.co/mllm-dev/gpt2_f_experiment_2) |
| * [mllm-dev/gpt2_f_experiment_9](https://huggingface.co/mllm-dev/gpt2_f_experiment_9) |
| * [mllm-dev/gpt2_f_experiment_6](https://huggingface.co/mllm-dev/gpt2_f_experiment_6) |
| * [mllm-dev/gpt2_f_experiment_8](https://huggingface.co/mllm-dev/gpt2_f_experiment_8) |
| * [mllm-dev/gpt2_f_experiment_0](https://huggingface.co/mllm-dev/gpt2_f_experiment_0) |
| * [mllm-dev/gpt2_f_experiment_3](https://huggingface.co/mllm-dev/gpt2_f_experiment_3) |
| * [mllm-dev/gpt2_f_experiment_1](https://huggingface.co/mllm-dev/gpt2_f_experiment_1) |
| * [mllm-dev/gpt2_f_experiment_7](https://huggingface.co/mllm-dev/gpt2_f_experiment_7) |
| * [mllm-dev/gpt2_f_experiment_4](https://huggingface.co/mllm-dev/gpt2_f_experiment_4) |
| |
| ### Configuration |
| |
| The following YAML configuration was used to produce this model: |
| |
| ```yaml |
| dtype: float16 |
| merge_method: linear |
| slices: |
| - sources: |
| - layer_range: [0, 12] |
| model: |
| model: |
| path: mllm-dev/gpt2_f_experiment_0 |
| parameters: |
| weight: 1.0 |
| - layer_range: [0, 12] |
| model: |
| model: |
| path: mllm-dev/gpt2_f_experiment_1 |
| parameters: |
| weight: 1.0 |
| - layer_range: [0, 12] |
| model: |
| model: |
| path: mllm-dev/gpt2_f_experiment_2 |
| parameters: |
| weight: 1.0 |
| - layer_range: [0, 12] |
| model: |
| model: |
| path: mllm-dev/gpt2_f_experiment_3 |
| parameters: |
| weight: 1.0 |
| - layer_range: [0, 12] |
| model: |
| model: |
| path: mllm-dev/gpt2_f_experiment_4 |
| parameters: |
| weight: 1.0 |
| - layer_range: [0, 12] |
| model: |
| model: |
| path: mllm-dev/gpt2_f_experiment_5 |
| parameters: |
| weight: 1.0 |
| - layer_range: [0, 12] |
| model: |
| model: |
| path: mllm-dev/gpt2_f_experiment_6 |
| parameters: |
| weight: 1.0 |
| - layer_range: [0, 12] |
| model: |
| model: |
| path: mllm-dev/gpt2_f_experiment_7 |
| parameters: |
| weight: 1.0 |
| - layer_range: [0, 12] |
| model: |
| model: |
| path: mllm-dev/gpt2_f_experiment_8 |
| parameters: |
| weight: 1.0 |
| - layer_range: [0, 12] |
| model: |
| model: |
| path: mllm-dev/gpt2_f_experiment_9 |
| parameters: |
| weight: 1.0 |
| ``` |
| |
