Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -16,18 +16,10 @@ license: apache-2.0
|
|
| 16 |
|
| 17 |
**Autor:** Ariel García Traba (Independent Research / EinsofOS)
|
| 18 |
**Institución:** Einsof Research / Antigravity Autonomous Compute
|
| 19 |
-
**Entrenado en:** Google Colab Plus (NVIDIA A100-SXM4
|
| 20 |
**Modelo Base:** `deepseek-ai/DeepSeek-R1-Distill-Qwen-14B`
|
| 21 |
**Corpus Canónico:** 1.736 teoremas formales de PostgreSQL 16 (`polydim_vec_db`) + 13 Leyes de Silicio de Einsof.
|
| 22 |
-
|
| 23 |
-
## Hiperparámetros de Entrenamiento
|
| 24 |
-
- **LoRA Rank (r):** 16
|
| 25 |
-
- **LoRA Alpha (α):** 16
|
| 26 |
-
- **Target Modules:** `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
|
| 27 |
-
- **Parámetros Entrenables:** 68,812,800
|
| 28 |
-
- **Precisión Nativa:** BF16 nativo (A100 SXM4)
|
| 29 |
-
- **Optimizador:** `adamw_8bit`
|
| 30 |
-
- **Pérdida Final de Entrenamiento:** 0.7225
|
| 31 |
|
| 32 |
## Uso con Unsloth / Hugging Face Transformers
|
| 33 |
```python
|
|
@@ -35,7 +27,7 @@ from unsloth import FastLanguageModel
|
|
| 35 |
import torch
|
| 36 |
|
| 37 |
model, tokenizer = FastLanguageModel.from_pretrained(
|
| 38 |
-
model_name="
|
| 39 |
max_seq_length=4096,
|
| 40 |
dtype=torch.bfloat16,
|
| 41 |
load_in_4bit=True,
|
|
|
|
| 16 |
|
| 17 |
**Autor:** Ariel García Traba (Independent Research / EinsofOS)
|
| 18 |
**Institución:** Einsof Research / Antigravity Autonomous Compute
|
| 19 |
+
**Entrenado en:** Google Colab Plus (NVIDIA A100-SXM4)
|
| 20 |
**Modelo Base:** `deepseek-ai/DeepSeek-R1-Distill-Qwen-14B`
|
| 21 |
**Corpus Canónico:** 1.736 teoremas formales de PostgreSQL 16 (`polydim_vec_db`) + 13 Leyes de Silicio de Einsof.
|
| 22 |
+
**Pérdida Final de Entrenamiento:** 0.7225
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
## Uso con Unsloth / Hugging Face Transformers
|
| 25 |
```python
|
|
|
|
| 27 |
import torch
|
| 28 |
|
| 29 |
model, tokenizer = FastLanguageModel.from_pretrained(
|
| 30 |
+
model_name="Polydim-Einsof/DeepSeek-R1-Distill-Qwen-14B-Einsof-LoRA",
|
| 31 |
max_seq_length=4096,
|
| 32 |
dtype=torch.bfloat16,
|
| 33 |
load_in_4bit=True,
|