Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -16,10 +16,18 @@ license: apache-2.0
|
|
| 16 |
|
| 17 |
**Autor:** Ariel Garc铆a Traba (Independent Research / EinsofOS)
|
| 18 |
**Instituci贸n:** Einsof Research / Antigravity Autonomous Compute
|
| 19 |
-
**Entrenado en:** Google Colab Plus (NVIDIA A100-SXM4)
|
| 20 |
**Modelo Base:** `Qwen/Qwen2.5-Coder-32B-Instruct`
|
| 21 |
**Corpus Can贸nico:** 1.736 teoremas formales de PostgreSQL 16 (`polydim_vec_db`) + 13 Leyes de Silicio de Einsof.
|
| 22 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
## Uso con Unsloth / Hugging Face Transformers
|
| 25 |
```python
|
|
@@ -27,7 +35,7 @@ from unsloth import FastLanguageModel
|
|
| 27 |
import torch
|
| 28 |
|
| 29 |
model, tokenizer = FastLanguageModel.from_pretrained(
|
| 30 |
-
model_name="
|
| 31 |
max_seq_length=4096,
|
| 32 |
dtype=torch.bfloat16,
|
| 33 |
load_in_4bit=True,
|
|
|
|
| 16 |
|
| 17 |
**Autor:** Ariel Garc铆a Traba (Independent Research / EinsofOS)
|
| 18 |
**Instituci贸n:** Einsof Research / Antigravity Autonomous Compute
|
| 19 |
+
**Entrenado en:** Google Colab Plus (NVIDIA A100-SXM4-40GB)
|
| 20 |
**Modelo Base:** `Qwen/Qwen2.5-Coder-32B-Instruct`
|
| 21 |
**Corpus Can贸nico:** 1.736 teoremas formales de PostgreSQL 16 (`polydim_vec_db`) + 13 Leyes de Silicio de Einsof.
|
| 22 |
+
|
| 23 |
+
## Hiperpar谩metros de Entrenamiento
|
| 24 |
+
- **LoRA Rank (r):** 16
|
| 25 |
+
- **LoRA Alpha (伪):** 16
|
| 26 |
+
- **Target Modules:** `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
|
| 27 |
+
- **Par谩metros Entrenables:** 134,217,728
|
| 28 |
+
- **Precisi贸n Nativa:** BF16 nativo (A100 SXM4)
|
| 29 |
+
- **Optimizador:** `adamw_8bit`
|
| 30 |
+
- **P茅rdida Final de Entrenamiento:** 0.5842
|
| 31 |
|
| 32 |
## Uso con Unsloth / Hugging Face Transformers
|
| 33 |
```python
|
|
|
|
| 35 |
import torch
|
| 36 |
|
| 37 |
model, tokenizer = FastLanguageModel.from_pretrained(
|
| 38 |
+
model_name="polydim-einsof/Qwen2.5-Coder-32B-Einsof-LoRA",
|
| 39 |
max_seq_length=4096,
|
| 40 |
dtype=torch.bfloat16,
|
| 41 |
load_in_4bit=True,
|