-
fix cuda
-
Fix colab bugs
-
Prepare fine-tune for colab
-
finished finetuning with 16 rank lora
-
added fine-tune testing on validation set with short queries only
-
Adding validation set testing of newly fine-tuned model
-
trained 8 rank LoRA finetuned model, saving train and validation datasets to disk
-
first cut at fine-tuning with masked training, working but very very slow for inference
-
Increasing LoRA rank to 32 and updating loss function
-
First decent attempt, seems to generate valid queries after pre-training. Needs evaluation still
-
First successful fine-tuning run, resulting model is no good but it does train successfully
-
Initial attempt at fine-tuning using LoRA with basic cross-entropy loss
