File size: 3,379 Bytes
ed7ae7a 450486b ed7ae7a 450486b ed7ae7a 450486b ed7ae7a 450486b ed7ae7a 450486b ed7ae7a 450486b ed7ae7a 450486b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 | ---
tags:
- gguf
- llama.cpp
- unsloth
- mistral
- python
base_model:
- mistralai/Mistral-7B-Instruct-v0.3
---
# mistral-7b-python-gguf
Conversational Python fine-tune of Mistral 7B exported to GGUF format for local inference.
- Base model: Mistral 7B
- Fine-tuning framework: Unsloth
- Format: GGUF
- Author: AntoineChatry
---
# ⚠️ Disclaimer
This is an **early experimental fine-tune**.
It is **not production-ready**, not fully aligned, and not optimized for reliability or long-form reasoning.
This project was created primarily for learning and experimentation.
Please do not expect state-of-the-art coding performance.
---
# Model Overview
This model is a conversational fine-tune of Mistral 7B trained primarily on:
- ShareGPT-style conversations
- Python-focused discussions
- Coding Q&A format
The objective was to:
- Experiment with fine-tuning
- Build a conversational Python model
- Export to GGUF for llama.cpp compatibility
- Test local inference workflows
No RLHF or advanced alignment was applied beyond the base model.
---
# Known Limitations
## Repetition Issues
- Frequently repeats phrases like:
> "Here's the code:"
- Can loop or restate similar sentences
- Overuses patterns learned from dataset formatting
## Weak Long-Form Explanations
- Struggles with multi-paragraph structured reasoning
- May repeat itself when asked for detailed explanations
- Limited depth on conceptual explanations
## Instruction Following
- Not fully aligned
- May ignore strict formatting constraints
- Tends to prioritize generating code over detailed explanations
## Dataset Bias
- Strong ShareGPT conversational tone
- Python-heavy bias
- Some templated response structure
---
# What Works Reasonably Well
- Short Python snippets
- Basic debugging help
- Simple function generation
- Conversational coding prompts
Best performance is observed when:
- Prompts are clear and direct
- Expected output is short
- Tasks are code-focused
---
# Training Details
- Base: Mistral 7B
- Dataset format: ShareGPT-style conversational dataset (Python-oriented)
- Fine-tuned using Unsloth notebooks
- Converted to GGUF for llama.cpp compatibility
- Quantized version included (Q4_K_M)
No additional safety tuning or post-training optimization was applied.
---
# Example Usage
This model was finetuned and converted to GGUF format using Unsloth.
## llama.cpp
For text-only LLMs:
```bash
llama-cli -hf AntoineChatry/mistral-7b-python-gguf --jinja
```
For multimodal models:
```bash
llama-mtmd-cli -hf AntoineChatry/mistral-7b-python-gguf --jinja
```
---
## Available Model files:
- `mistral-7b-instruct-v0.3.Q4_K_M.gguf`
---
# Ollama
An Ollama Modelfile is included for easy deployment.
Example:
```bash
ollama create mistral-python -f Modelfile
ollama run mistral-python
```
---
# Why This Model Is Public
This model represents a learning milestone.
Sharing imperfect models helps:
- Document fine-tuning progress
- Enable experimentation
- Collect feedback
- Iterate toward better versions
This is not a finished product.
---
# Unsloth
This model was trained 2x faster using Unsloth.
https://github.com/unslothai/unsloth
<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>
---
# License
Please refer to the original Mistral 7B license from Mistral AI. |