GPT OxideLLM_5M
Modelo de lenguaje tipo GPT entrenado desde cero con el texto de OxideLLM_5M de la Mancha.
Configuraci贸n del modelo
- Vocabulario: 221 caracteres
- Dimensi贸n del modelo (d_model): 256
- N煤mero de cabezas de atenci贸n: 4
- Dimensi贸n feedforward (dff): 512
- N煤mero de capas: 4
- Longitud m谩xima de secuencia: 128
Uso
from huggingface_hub import snapshot_download
import tensorflow as tf
import json
# Descargar modelo
model_path = snapshot_download(repo_id="ULFBERTO/OxideLLM_5M")
# Cargar vocabulario
with open(f"{model_path}/vocab.json", "r", encoding="utf-8") as f:
vocab_data = json.load(f)
char2idx = vocab_data["char2idx"]
idx2char = {int(k): v for k, v in vocab_data["idx2char"].items()}
# Cargar modelo
model = tf.keras.models.load_model(f"{model_path}/saved_model")
Entrenamiento
Entrenado con TensorFlow 2.x usando arquitectura Transformer (decoder-only).
- Downloads last month
- 18
# !pip install -U keras tensorflow huggingface_hub # Keras needs TensorFlow installed to read "hf://" paths, so the tensorflow backend is selected here; # "jax" and "torch" also work for computation once TensorFlow is installed. import os os.environ["KERAS_BACKEND"] = "tensorflow" import keras model = keras.saving.load_model("hf://ULFBERTO/OxideLLM_5M")