run_full_pipeline.sh: pin transformers<5; drop xformers; +hf_hub
Browse filesProbe of the L4 image shows transformers 5.5.0 pre-shipped (its lazy
TrainingArguments import is broken). Force transformers>=4.46,<5
explicitly. Drop xformers (unsloth falls back gracefully) and add
huggingface_hub explicit (gradio's transitive doesn't pull it on this
image build).
Made-with: Cursor
scripts/run_full_pipeline.sh
CHANGED
|
@@ -24,11 +24,16 @@ pip install -q "future>=1.0.0"
|
|
| 24 |
pip install -q --extra-index-url https://download.pytorch.org/whl/cu128 \
|
| 25 |
"torch==2.10.0" "torchvision==0.25.0"
|
| 26 |
|
| 27 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
pip install -q --no-deps "unsloth @ git+https://github.com/unslothai/unsloth.git"
|
| 29 |
pip install -q --no-deps "unsloth_zoo"
|
| 30 |
pip install -q --no-deps trl peft accelerate bitsandbytes
|
| 31 |
-
pip install -q
|
| 32 |
pip install -q -r requirements.txt
|
| 33 |
|
| 34 |
echo "[2/6] Building / verifying datasets ..."
|
|
|
|
| 24 |
pip install -q --extra-index-url https://download.pytorch.org/whl/cu128 \
|
| 25 |
"torch==2.10.0" "torchvision==0.25.0"
|
| 26 |
|
| 27 |
+
# Force a known-good transformers (the L4 image ships transformers 5.x
|
| 28 |
+
# pre-release whose lazy TrainingArguments import is broken).
|
| 29 |
+
pip install -q "transformers>=4.46,<5"
|
| 30 |
+
|
| 31 |
+
# Install the unsloth stack without letting pip second-guess the torch
|
| 32 |
+
# pin (--no-deps) since the deps that matter are already locked above.
|
| 33 |
pip install -q --no-deps "unsloth @ git+https://github.com/unslothai/unsloth.git"
|
| 34 |
pip install -q --no-deps "unsloth_zoo"
|
| 35 |
pip install -q --no-deps trl peft accelerate bitsandbytes
|
| 36 |
+
pip install -q datasets tensorboard matplotlib sentencepiece protobuf huggingface_hub
|
| 37 |
pip install -q -r requirements.txt
|
| 38 |
|
| 39 |
echo "[2/6] Building / verifying datasets ..."
|