instruct-mask-tools / docs /UPLOAD_GUIDELINES.md

Sanitize: remove residual token-prefix fragment (hf_Lb) from masking example

454413e verified about 2 months ago

6.79 kB

RECOVERED 2026-05-23T23:57:12 from Claude session transcripts (last Write; later Edit-tool revisions not applied). Original file was missing from disk.

Upload Guidelines

Status: STUB (to be expanded after EVAL_GUIDELINES.md is locked through 31B Q4_K_M + Q6_K + dual-eval wrapper). Holding scaffolding so cross-references in companion docs resolve to a real path.

Scope: how to upload artifacts (adapters, merged safetensors, GGUF quants, datasets, model cards) to Hugging Face reliably and in the right order.

Companion docs:

EVAL_GUIDELINES.md — how to evaluate models after uploading.
INSTRUCT_MASKING_TECHNIQUES.md — masking/merging techniques that produce the artifacts.
MERGE_MADNESS.md — community-model merge recipe cookbook.
This doc — how to upload the outputs of the above.

0. Hard rules (always apply)

0.1 HF_HUB_DISABLE_XET=1 is mandatory for every upload. Without it, Xet de-dup throttled our 31B uploads to ~1 MB/s (2026-05-22/23). With it, sustained ~6 MB/s on the same files. Set the env var inline on every Python invocation:

HF_HUB_DISABLE_XET=1 python upload_script.py

0.2 chat_template.jinja must be present in the source dir before quant. Otherwise the GGUF ships templateless → llama-server guesses ChatML → leak + IFEval drop. (Top-level rule, memory feedback_copy_jinja_before_quant.md. Lived through this on 2026-05-23.)

0.3 GGUF provider for community quants: bartowski only. NEVER unsloth. (Memory feedback_gguf_quants.md.) AesSedai / ddh0 are also OK for MoE specifically.

0.4 File-size verification before declaring "done". Especially with xet stalls or transient HF errors: after upload, fetch the file's HF-side size via HfApi.model_info(repo).siblings and compare against the local size. Don't trust the absence of a Python exception alone. (Memory project_b200_runpod_orchestration.md.)

0.5 Adapter-first upload order for multi-artifact releases. Adapter (~~hundreds of MB) → merged safetensors (~~hundreds of GB raw / ~30-120 GB bf16) → GGUF quants. So downstream users can grab the cheap thing while the heavy artifacts are still uploading.

1. Token storage and which account uploads where

1.1 Local token file. The default Hugging Face CLI token lives at /home/aibox/.cache/huggingface/token on this box (single line, starts with hf_…). huggingface_hub.HfApi() and the huggingface-cli both pick this up automatically — scripts should not read it explicitly or print it.

1.2 Verify which account a token belongs to without printing the token itself:

python -c "from huggingface_hub import whoami; print(whoami())"

This prints {'name': '<username>', ...} plus the orgs list the token can act under.

1.3 ToastyPigeon vs rpDungeon. The default token is currently the ToastyPigeon account token. Whether it has org-membership / write access to rpDungeon/* repos depends on org settings on the HF side:

If it does: same token writes to both. Verify with whoami()'s orgs field.
If it does not: we need a separate token for rpDungeon. Store as a second file (suggested path: /home/aibox/.cache/huggingface/token_rpdungeon) and select via the token= kwarg in HfApi(token=open(path).read().strip()) rather than mutating the global file.

Action item (PENDING): confirm whether the current default token has write access to rpDungeon/*. The gembrain_equinox upload at 21:07 UTC succeeded against rpDungeon/Gemma4-31b-Gembrain-Equinox, so at least at that moment it did. If a future upload to rpDungeon returns 403, swap the token via the second-file pattern.

1.4 Never paste tokens into committed files, scripts, Discord, or this doc. Read from /home/aibox/.cache/huggingface/token or the named-second-file path. Anything printed in CI/logs must be masked (hf_… style).

1.5 Rotation. If a token is ever exposed (Discord paste, accidental commit, leaked log), rotate immediately at https://huggingface.co/settings/tokens. Update the local file. No emergency rebuild of repos is needed — only the token changes.

2. Upload patterns (TO BE FILLED — placeholders for now)

2.1 Single file (GGUF / safetensors shard / adapter zip)

import os
os.environ.setdefault("HF_HUB_DISABLE_XET", "1")
from huggingface_hub import HfApi
HfApi().upload_file(
    path_or_fileobj=local_path,
    path_in_repo=remote_name,
    repo_id=repo,
    repo_type="model",
)

2.2 Whole directory (adapter folder, merged dir)

Use HfApi.upload_folder with allow_patterns / ignore_patterns.
Always ignore checkpoint dirs / optimizer states unless explicitly republishing them.

2.3 Template-only re-mint of an existing GGUF on HF

gguf_new_metadata.py --chat-template-file <jinja> on the local GGUF (~5 min).
Swap-and-upload over the existing HF copy (~50 min at 6 MB/s for 18.7 GB Q4_K_M).
Reference script: instruct_mask/refresh_gguf_chat_templates.sh + upload_tplmod_ggufs.sh.

2.4 Large datasets

(to fill — covers shard-size choice, repo_type='dataset', parquet vs jsonl)

3. Detached-job harness for uploads

Every upload that's expected to take longer than ~5 min should run via run-detached (systemd-user) so it survives Claude restarts, hooks into the OOM watcher, and has a memory cap. Pure-network uploads can use RUN_DETACHED_ALLOW_SMALL_CAP=1 run-detached <name> 8G ... (network-only, no model load).

Reference: discord-project/instruct_mask/upload_tplmod_ggufs.sh + its run-detached launch line.

4. Validation checklist (after every upload)

HfApi.model_info(repo) returns successfully and lists the file with the expected size
If GGUF: file is downloadable + llama-server boots against it cleanly (smoke ping)
If adapter: PeftModel.from_pretrained loads it cleanly against the declared base
Repo README updated if filename / content changed
Discord ping with the HF URL (so the next eval / merge step has the source)

5. Open questions (to scope when expanding this doc)

Best multipart chunk size for 25-50 GB files via HfApi.upload_file
Resume semantics: if upload script dies at 80%, can we resume vs starting over?
How to handle quota / repo-size limits — staging on a different repo first?
Concurrent upload throttling — does running 2 uploads in parallel double or halve throughput at our 6 MB/s ceiling?
Whether to set commit_message per file (good audit trail) vs leaving default

6. Changelog

2026-05-23 — Stub created. Cross-referenced from MASK / MERGE / EVAL docs. Awaiting full content pass after EVAL_GUIDELINES.md lands.