Pratham-math commited on
Commit
fffaeea
·
verified ·
1 Parent(s): 7ad981e

deps: bnb>=0.48.1 for vllm 0.18 4-bit gate

Browse files
Files changed (1) hide show
  1. scripts/job_train.sh +8 -4
scripts/job_train.sh CHANGED
@@ -27,10 +27,14 @@ pip install -q hydra-core omegaconf wandb 'huggingface_hub>=0.28' tyro
27
  # Core ML deps.
28
  # NOTE: trl 1.2 imports `is_trackio_available` from transformers, which is not
29
  # present in 4.49.0. Use a newer transformers line in jobs.
30
- # bitsandbytes 0.45.1 only ships up to cuda124 binary; vllm 0.18.0 pulls torch
31
- # with the cu128 wheel, so we MUST use a bnb release that ships libbitsandbytes_cuda128.so
32
- # (0.47.0 is the first stable line that does). Confirmed against PyPI Apr 2026.
33
- pip install -q transformers==4.56.2 accelerate==1.5.2 peft==0.14.0 'bitsandbytes>=0.47.0,<0.48'
 
 
 
 
34
  pip install -q datasets==4.7.0
35
  # Keep TRL pinned for OpenEnv path but bypass resolver deadlock with datasets pin.
36
  pip install -q --no-deps trl==1.2.0
 
27
  # Core ML deps.
28
  # NOTE: trl 1.2 imports `is_trackio_available` from transformers, which is not
29
  # present in 4.49.0. Use a newer transformers line in jobs.
30
+ # bitsandbytes pin notes:
31
+ # - 0.45.1: ships cu124 only breaks under vllm's cu128 torch.
32
+ # - 0.47.x: cu128 + cu124 dual-wheel, but vllm's BitsAndBytesLinearMethod
33
+ # gates on `bnb >= 0.48.1` (vllm/model_executor/layers/quantization/
34
+ # bitsandbytes.py:_check_bitsandbytes_version) and raises ImportError
35
+ # during `LLM(...)` init when used with a 4-bit model.
36
+ # - >=0.48.1: satisfies both vllm's gate and our cu128 wheel needs.
37
+ pip install -q transformers==4.56.2 accelerate==1.5.2 peft==0.14.0 'bitsandbytes>=0.48.1'
38
  pip install -q datasets==4.7.0
39
  # Keep TRL pinned for OpenEnv path but bypass resolver deadlock with datasets pin.
40
  pip install -q --no-deps trl==1.2.0