Instructions to use patdev/k3-a40-bootstrap with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use patdev/k3-a40-bootstrap with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf patdev/k3-a40-bootstrap:BF16 # Run inference directly in the terminal: llama cli -hf patdev/k3-a40-bootstrap:BF16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf patdev/k3-a40-bootstrap:BF16 # Run inference directly in the terminal: llama cli -hf patdev/k3-a40-bootstrap:BF16
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf patdev/k3-a40-bootstrap:BF16 # Run inference directly in the terminal: ./llama-cli -hf patdev/k3-a40-bootstrap:BF16
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf patdev/k3-a40-bootstrap:BF16 # Run inference directly in the terminal: ./build/bin/llama-cli -hf patdev/k3-a40-bootstrap:BF16
Use Docker
docker model run hf.co/patdev/k3-a40-bootstrap:BF16
- LM Studio
- Jan
- Ollama
How to use patdev/k3-a40-bootstrap with Ollama:
ollama run hf.co/patdev/k3-a40-bootstrap:BF16
- Unsloth Desktop
- Docker Model Runner
How to use patdev/k3-a40-bootstrap with Docker Model Runner:
docker model run hf.co/patdev/k3-a40-bootstrap:BF16
- Lemonade
How to use patdev/k3-a40-bootstrap with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull patdev/k3-a40-bootstrap:BF16
Run and chat with the model
lemonade run user.k3-a40-bootstrap-BF16
List all available models
lemonade list
- Atomic Chat
aides pour la montee de transformers
Browse files- aides/_verif_cfg.py +59 -0
aides/_verif_cfg.py
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Le modele est-il enfin reconnu ? Sort non nul si non, ce qui declenche
|
| 2 |
+
# le revert cote script appelant.
|
| 3 |
+
import glob
|
| 4 |
+
import sys
|
| 5 |
+
import warnings
|
| 6 |
+
|
| 7 |
+
warnings.filterwarnings("ignore")
|
| 8 |
+
|
| 9 |
+
MOTIF = ("/root/.cache/huggingface/hub/"
|
| 10 |
+
"models--RadixArk--Qwen3.8-Flash-Next-NVFP4/snapshots/*")
|
| 11 |
+
snaps = sorted(glob.glob(MOTIF))
|
| 12 |
+
if not snaps:
|
| 13 |
+
print(" snapshot introuvable")
|
| 14 |
+
sys.exit(7)
|
| 15 |
+
snap = snaps[-1]
|
| 16 |
+
|
| 17 |
+
try:
|
| 18 |
+
import transformers.models.qwen4_exp # noqa: F401
|
| 19 |
+
print(" module qwen4_exp : PRESENT")
|
| 20 |
+
except Exception as e:
|
| 21 |
+
print(" module qwen4_exp : ABSENT ->", repr(e)[:140])
|
| 22 |
+
sys.exit(7)
|
| 23 |
+
|
| 24 |
+
from transformers import AutoConfig, AutoProcessor
|
| 25 |
+
|
| 26 |
+
try:
|
| 27 |
+
cfg = AutoConfig.from_pretrained(snap, trust_remote_code=True)
|
| 28 |
+
except Exception as e:
|
| 29 |
+
print(" AutoConfig echoue ->", repr(e)[:200])
|
| 30 |
+
sys.exit(7)
|
| 31 |
+
|
| 32 |
+
texte = getattr(cfg, "text_config", cfg)
|
| 33 |
+
rope = getattr(texte, "rope_parameters", None)
|
| 34 |
+
section = None
|
| 35 |
+
if isinstance(rope, dict):
|
| 36 |
+
section = rope.get("mrope_section")
|
| 37 |
+
else:
|
| 38 |
+
section = getattr(rope, "mrope_section", None)
|
| 39 |
+
|
| 40 |
+
print(" config class :", type(cfg).__name__, "/", type(texte).__name__)
|
| 41 |
+
print(" rope_parameters :", rope)
|
| 42 |
+
print(" mrope_section :", section)
|
| 43 |
+
|
| 44 |
+
try:
|
| 45 |
+
proc = AutoProcessor.from_pretrained(snap, trust_remote_code=True)
|
| 46 |
+
ip = getattr(proc, "image_processor", None)
|
| 47 |
+
print(" processeur :", type(proc).__name__)
|
| 48 |
+
print(" image_processor :", type(ip).__name__ if ip is not None else "?")
|
| 49 |
+
except Exception as e:
|
| 50 |
+
print(" AutoProcessor echoue ->", repr(e)[:200])
|
| 51 |
+
sys.exit(7)
|
| 52 |
+
|
| 53 |
+
# Le mRoPE est la seule chose qui donne une geometrie aux patchs d'image.
|
| 54 |
+
# S'il manque encore, l'operation n'a servi a rien : autant revenir en arriere
|
| 55 |
+
# plutot que de redemarrer le moteur pour le meme resultat.
|
| 56 |
+
if not section:
|
| 57 |
+
print(" !!! mrope_section toujours absent")
|
| 58 |
+
sys.exit(7)
|
| 59 |
+
print(" -> reconnaissance OK")
|