YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

Week 08 Compression 40 โ€” Submission 01

This submission recreates the best 40% experiment:

  • source checkpoint: merged behavioral_concise_lora_r8 Qwen3.5-4B checkpoint
  • quantizer: GPTQModel
  • weights: W4A16
  • group size: 64
  • desc_act: false
  • HMMT is never used for calibration

The earlier experiment with this recipe reached 55/100 on HMMT100 with 100% parse. The model weights themselves are not stored in git; build the packed checkpoint and upload it to the Week08 Hugging Face model repository.

1. Recreate/persist the behavioral BF16 parent

The expected parent checkpoint is:

/marimo/CS6013/artifacts/behavioral_concise_lora_r8/bf16_model

Use the repository's configs/distill/qwen35_4b_behavioral_concise_lora_r8.yaml and verified concise SFT traces to rebuild it if necessary. Freeze/save that parent checkpoint before quantization.

2. Compress

Install dependencies:

cd /marimo/CS6013/24b1064/Week08/Compression_40/Submission01
uv sync

Run GPTQ W4/G64 using a held-out math calibration JSONL (prompt + completion; no HMMT):

uv run python compress.py \
  --model_name Qwen/Qwen3.5-4B \
  --checkpoint_path /marimo/CS6013/artifacts/behavioral_concise_lora_r8/bf16_model \
  --calibration_path /marimo/CS6013/data/traces/behavioral_concise_lora_r8_rac_150k.jsonl \
  --output_path /marimo/CS6013/artifacts/week08_behavioral_gptq_w4_g64/compressed

The compressor writes compression_recipe.json and refuses success if the measured text-model fraction is not < 0.40 relative to the course 8.0585 GiB text baseline.

3. Decompress

uv run python decompress.py \
  --model_name Qwen/Qwen3.5-4B \
  --checkpoint_path /marimo/CS6013/artifacts/week08_behavioral_gptq_w4_g64/compressed \
  --output_path /marimo/CS6013/artifacts/week08_behavioral_gptq_w4_g64/decompressed

The output is a normal dense Hugging Face checkpoint for vLLM evaluation.

4. Submission model repo

The uploaded artifact is in the public Hugging Face repository:

sDarth/24b1064-Week08-Compression-40-Submission01

Upload the complete packed compressed/ directory only after the hard size check passes. Also preserve compression_recipe.json, the training config, calibration metadata, and the behavioral parent checkpoint/adapter in durable storage.

The Week08 evaluator audit measured size_frac=0.3705326200. The existing reconstructed-checkpoint compatibility smoke passed 3/3 problems with 100% parse rate. Fresh Python 3.12 submission dependency installation and imports also passed. These checks do not represent a new full HMMT evaluation.

Downloads last month
13
Safetensors
Model size
4B params
Tensor type
F16
ยท
I32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support