e1 / README.md
Finn-Org's picture
Upload e1 LoRA adapters
b0d50b7 verified
|
Raw History Blame Contribute Delete
1.26 kB
---
tags:
- gemma-4
- peft
- lora
- finn-organization
license: gemma
---
# e1
Fine-tuned from `google/gemma-4-E4B-it-qat-q4_0-unquantized`.
## Description
e1 is an AI model created by Finn Organization. It is fine-tuned on a diverse dataset covering mathematics, physics, computer science, biology, chemistry, philosophy, linguistics, and more.
## Usage
```python
from unsloth import FastLanguageModel
from peft import PeftModel
model, processor = FastLanguageModel.from_pretrained(
"google/gemma-4-E4B-it-qat-q4_0-unquantized",
max_seq_length=2048,
load_in_4bit=True,
)
model = PeftModel.from_pretrained(model, "Finn-Technologies/e1")
messages = [
{"role": "system", "content": "You are e1, an AI model created by Finn Organization."},
{"role": "user", "content": "Your prompt here"},
]
text = processor.tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True)
outputs = model.generate(**processor(text, return_tensors="pt"), max_new_tokens=200)
print(processor.decode(outputs[0]))
```
## Training
- **Base model**: Gemma 4 E4B QAT (4-bit quantized)
- **Method**: QLoRA (r=16, target_modules=[q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj])
- **Train loss**: 1.09
- **Eval loss**: 1.02