Instructions to use Crystalcareai/CrystalMistral-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Crystalcareai/CrystalMistral-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Crystalcareai/CrystalMistral-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf Crystalcareai/CrystalMistral-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Crystalcareai/CrystalMistral-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf Crystalcareai/CrystalMistral-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Crystalcareai/CrystalMistral-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf Crystalcareai/CrystalMistral-GGUF:Q4_K_M
Use Docker
docker model run hf.co/Crystalcareai/CrystalMistral-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use Crystalcareai/CrystalMistral-GGUF with Ollama:
ollama run hf.co/Crystalcareai/CrystalMistral-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use Crystalcareai/CrystalMistral-GGUF with Docker Model Runner:
docker model run hf.co/Crystalcareai/CrystalMistral-GGUF:Q4_K_M
- Lemonade
How to use Crystalcareai/CrystalMistral-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Crystalcareai/CrystalMistral-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.CrystalMistral-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
CrystalMistral: A Potent Language Model Fine-Tuned on a Curated Blend of Data
Overview
CrystalMistral is a highly refined language model derived from the esteemed Mistral-7B architecture. To unlock its exceptional capabilities, CrystalMistral underwent meticulous fine-tuning on a meticulously curated dataset comprised of:
Evol-Instruct: A rich dataset emphasizing instruction following and task completion, fostering CrystalMistral's ability to accurately execute complex commands. Airoboros: This extensive dataset is geared towards open-ended dialogue and generation, sharpening CrystalMistral's conversational aptitude and creativity. OpenOrca: A dataset specializing in code generation and understanding, significantly reinforcing CrystalMistral's programming prowess. Additional GPT-4 Synthetic Data: Incorporation of curated GPT-4 synthetic data further amplifies CrystalMistral's reasoning abilities and factual knowledge. Strengths
CrystalMistral exhibits a remarkable command of the following domains:
Instruction Following: Effectively interprets and carries out detailed instructions, demonstrating proficiency in task completion. Dialogue and Text Generation: Engages in fluid and nuanced conversations, offering creative and compelling text generation capabilities. Coding: Exhibits advanced understanding of code, capable of generating functional code, translating between languages, and offering explanations.
Future Development
The CrystalMistral project endeavors to:
Expert Fine-Tuning: Explore additional fine-tuning with datasets specializing in specific areas (e.g., scientific literature, legal documents) to create targeted variants of CrystalMistral. Mixture of Experts (MoE): Transition to a 4x MoE architecture, enabling CrystalMistral to dynamically specialize in distinct tasks, significantly amplifying its efficiency and potential.
- Downloads last month
- 19
4-bit
5-bit