Zishan-Shao/scratchspace

Basis rebuttal: frozen machine-handoff inputs

This repository carries the inputs for the three rebuttal experiments on branch rebuttal/20261104-defense-experiments of Zishan-Shao/Basis. The package is ready when UPLOAD_COMPLETE.json exists and its status is complete.

• 26 frozen model sources: 301.786 GiB, including the two dense parents.
• A 0.653 GiB archive with 331 files, including 68 unique adapters, token data, dense signed vectors, original run specs and required metadata.
• Original paths beneath checkpoints/frozen_sources/ are preserved.
• transfer_manifest.json records each file's size/SHA256 and each model's frozen canonical identity.
• Optional historical Experiment 1 raw feature-extraction inputs (10.3 GiB) are excluded. Existing CPU results are in the code branch; all new GPU measurements use this package.

Download on the destination cluster

Start from the code checkout and install its environment as described in rebuttal/NEW_MACHINE.md. Run the following from the Basis repository root:

python -m pip install 'huggingface-hub==0.35.3'
hf download Zishan-Shao/scratchspace \
  --include 'checkpoints/frozen_sources/**' 'rebuttal/migration/**' \
  --local-dir .
(cd rebuttal/migration && sha256sum -c rebuttal_inputs.tar.sha256)
tar -xf rebuttal/migration/rebuttal_inputs.tar -C .
python rebuttal/migrate.py check --verify-models
python rebuttal/verify.py
python rebuttal/analyze_measurements.py --partial

The download can be resumed by repeating the same hf download command. Allow at least 304 GiB of free disk space plus room for experiment outputs. Do not edit frozen run specs or rerun prepare.py / add_cost_jobs.py after moving.

After CPU validation, launch the GPU queue explicitly:

nohup python -u rebuttal/pipeline.py > rebuttal/pipeline.log 2>&1 < /dev/null &

These are input artifacts, not completed GPU results. The handoff snapshot has 0/134 completed GPU measurements. Full FP32 measurement requires at least 64 GiB free GPU memory; the scheduler runs its required preflights and uses at most two GPUs.

Upload from the source cluster

conda activate flashsvd
hf auth login
python rebuttal/publish_hf.py --upload

The publisher checks frozen checkpoint identities, stages symlinks without copying model data, resumes using local upload metadata, and verifies remote sizes and hashes before writing UPLOAD_COMPLETE.json. Local preparation without authentication: python rebuttal/publish_hf.py --prepare.