Instructions to use Nkr3ch17/mistral_instruct_generation with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Nkr3ch17/mistral_instruct_generation with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1") model = PeftModel.from_pretrained(base_model, "Nkr3ch17/mistral_instruct_generation") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from Nkr3ch17/mistral_instruct_generation: direct link, hf CLI and curl.
- Browser
- Download file 2.72 kB
-
https://huggingface.co/Nkr3ch17/mistral_instruct_generation/resolve/main/README.md
- Command line
-
hf download hf://Nkr3ch17/mistral_instruct_generation/README.md
-
curl -L -o README.md https://huggingface.co/Nkr3ch17/mistral_instruct_generation/resolve/main/README.md
2.72 kB
| library_name: peft | |
| tags: | |
| - trl | |
| - sft | |
| - generated_from_trainer | |
| base_model: mistralai/Mistral-7B-Instruct-v0.1 | |
| datasets: | |
| - generator | |
| model-index: | |
| - name: mistral_instruct_generation | |
| results: [] | |
| <!-- This model card has been generated automatically according to the information the Trainer had access to. You | |
| should probably proofread and complete it, then remove this comment. --> | |
| # mistral_instruct_generation | |
| This model is a fine-tuned version of [mistralai/Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1) on the generator dataset. | |
| It achieves the following results on the evaluation set: | |
| - Loss: 0.3233 | |
| ## Model description | |
| More information needed | |
| ## Intended uses & limitations | |
| More information needed | |
| ## Training and evaluation data | |
| More information needed | |
| ## Training procedure | |
| ### Training hyperparameters | |
| The following hyperparameters were used during training: | |
| - learning_rate: 0.0002 | |
| - train_batch_size: 4 | |
| - eval_batch_size: 8 | |
| - seed: 42 | |
| - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 | |
| - lr_scheduler_type: constant | |
| - lr_scheduler_warmup_steps: 0.03 | |
| - training_steps: 500 | |
| - mixed_precision_training: Native AMP | |
| ### Training results | |
| | Training Loss | Epoch | Step | Validation Loss | | |
| |:-------------:|:-----:|:----:|:---------------:| | |
| | No log | 0.05 | 20 | 0.5742 | | |
| | No log | 0.09 | 40 | 0.5366 | | |
| | No log | 0.14 | 60 | 0.5206 | | |
| | No log | 0.18 | 80 | 0.5128 | | |
| | 0.553 | 0.23 | 100 | 0.5081 | | |
| | 0.553 | 0.27 | 120 | 0.5027 | | |
| | 0.553 | 0.32 | 140 | 0.4991 | | |
| | 0.553 | 0.36 | 160 | 0.4961 | | |
| | 0.553 | 0.41 | 180 | 0.4938 | | |
| | 0.498 | 0.45 | 200 | 0.4892 | | |
| | 0.498 | 0.5 | 220 | 0.4848 | | |
| | 0.498 | 0.54 | 240 | 0.4804 | | |
| | 0.498 | 0.59 | 260 | 0.4764 | | |
| | 0.498 | 0.63 | 280 | 0.4698 | | |
| | 0.4806 | 0.68 | 300 | 0.4630 | | |
| | 0.4806 | 0.72 | 320 | 0.4543 | | |
| | 0.4806 | 0.77 | 340 | 0.4457 | | |
| | 0.4806 | 0.81 | 360 | 0.4324 | | |
| | 0.4806 | 0.86 | 380 | 0.4200 | | |
| | 0.4393 | 0.9 | 400 | 0.4105 | | |
| | 0.4393 | 0.95 | 420 | 0.3930 | | |
| | 0.4393 | 1.0 | 440 | 0.3763 | | |
| | 0.4393 | 1.04 | 460 | 0.3614 | | |
| | 0.4393 | 1.09 | 480 | 0.3438 | | |
| | 0.3692 | 1.13 | 500 | 0.3233 | | |
| ### Framework versions | |
| - PEFT 0.10.1.dev0 | |
| - Transformers 4.39.3 | |
| - Pytorch 2.2.1+cu121 | |
| - Datasets 2.18.0 | |
| - Tokenizers 0.15.2 |