ounce100m-code / train /shard_dataset.py

Commit History

  1. perm_sha: identify the visit order by hashing the permutation, not just its seed (E-035 residual) -- cursor gains shuffle_perm_sha, asserted on resume; session repins the two files

    Verified signature
    398d16e
  2. mix_fingerprint: fold the manifest's per-shard sha256 into the corpus identity (names+sizes could collide across rebuilds); phase4_session repins shard_dataset

    Verified signature
    c588f4b

  1. train: shard_dataset -- static-review fixes: absolute-step cursor, hub_listing pagination, cursor required on resume, repo_type everywhere, rank-0 repo create

    Verified signature
    cfc38f1
  2. shard_dataset: drop the removed drop_last param's leftover body check (NameError at first call)

    Verified signature
    d5f46da
  3. train: add shard_dataset (Phase 3/4 glue)

    Verified signature
    644b0cd