Subject: Zero-filled checkpoint / Mount error breaking torch.load again

#3
by filetaker - opened

Hello! It looks like the read-only model-repo mount issue from 15 days ago is happening again, but this time it is throwing a pickle error instead of the invalid load key:
_pickle.UnpicklingError: persistent IDs in protocol 0 must be ASCII strings
The checkpoint.pt file seems corrupted or zero-filled mid-way during the GPU container run again. Could you please check the mount data or force the fallback local copy? Thank you!

Sign up or log in to comment