| --- |
| language: |
| - el |
| license: apache-2.0 |
| pipeline_tag: text-generation |
| tags: |
| - finetuned |
| inference: true |
| --- |
| |
| # kkOracle v0.1 |
|
|
| kkOracle v0.1 is a LORA fine-tuned version of [Meltemi 7B Instruct v1.5](https://huggingface.co/ilsp/Meltemi-7B-Instruct-v1.5) using a synthetic dataset based on text from the daily greek newspaper "Rizospastis" covering the timespan from 2008 to 2024. |
|
|
|
|
| # Running the model with mlx on a Mac |
|
|
| ``` |
| pip install mlx-lm |
| ``` |
|
|
| ``` |
| python -m mlx_lm.generate --model model_kkOracle --prompt "Καλημέρα!" --temp 0.3 |
| ``` |
|
|
|
|
| # Running the model on other systems |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| device = "cuda" # or "cpu" |
| |
| model = AutoModelForCausalLM.from_pretrained("model_kkOracle") |
| tokenizer = AutoTokenizer.from_pretrained("model_kkOracle") |
| |
| model.to(device) |
| |
| messages = [ |
| {"role": "user", "content": "Καλημέρα!"}, |
| ] |
| |
| prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False) |
| input_prompt = tokenizer(prompt, return_tensors='pt').to(device) |
| outputs = model.generate(input_prompt['input_ids'], max_new_tokens=256, do_sample=True, temperature=0.3, use_cache=True, pad_token_id=tokenizer.eos_token_id, attention_mask=input_prompt["attention_mask"]) |
| |
| print(tokenizer.batch_decode(outputs)[0]) |
| ``` |
|
|
|
|
| # Ethical Considerations |
|
|
| This model has been aligned with human preferences, but might generate misleading, harmful, and toxic content. |
|
|
|
|
|
|
