e1 / README.md
Finn-Org's picture
Upload e1 LoRA adapters
b0d50b7 verified
|
Raw History Blame Contribute Delete
1.26 kB
metadata
tags:
  - gemma-4
  - peft
  - lora
  - finn-organization
license: gemma

e1

Fine-tuned from google/gemma-4-E4B-it-qat-q4_0-unquantized.

Description

e1 is an AI model created by Finn Organization. It is fine-tuned on a diverse dataset covering mathematics, physics, computer science, biology, chemistry, philosophy, linguistics, and more.

Usage

from unsloth import FastLanguageModel
from peft import PeftModel

model, processor = FastLanguageModel.from_pretrained(
    "google/gemma-4-E4B-it-qat-q4_0-unquantized",
    max_seq_length=2048,
    load_in_4bit=True,
)
model = PeftModel.from_pretrained(model, "Finn-Technologies/e1")

messages = [
    {"role": "system", "content": "You are e1, an AI model created by Finn Organization."},
    {"role": "user", "content": "Your prompt here"},
]
text = processor.tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True)
outputs = model.generate(**processor(text, return_tensors="pt"), max_new_tokens=200)
print(processor.decode(outputs[0]))

Training

  • Base model: Gemma 4 E4B QAT (4-bit quantized)
  • Method: QLoRA (r=16, target_modules=[q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj])
  • Train loss: 1.09
  • Eval loss: 1.02