Safetensors
phylogpn
custom_code

Call post_init() in PhyloGPNModel for transformers v5 compatibility

#1
by jburtoft - opened

Loading PhyloGPN with transformers v5 (tested on 5.22.2) fails inside from_pretrained:

File ".../transformers/modeling_utils.py", in _move_missing_keys_from_meta_to_device
    for key in missing_keys - self.all_tied_weights_keys.keys():
AttributeError: 'PhyloGPNModel' object has no attribute 'all_tied_weights_keys'. Did you mean: '_tied_weights_keys'?

In transformers v5, PreTrainedModel.post_init() sets attributes the loader depends on (e.g. all_tied_weights_keys). PhyloGPNModel.__init__ never calls it. v4 didn't require the call; v5 does.

This PR adds a single self.post_init() call at the end of PhyloGPNModel.__init__.

Testing (CPU, torch 2.15.0):

  • transformers 5.22.2: the model-card snippet now runs. All 485 tensors load, with no missing or unexpected keys.
  • transformers 4.57.6: still works.
  • Outputs (embeddings and F81 log-rates, on the model-card example and a random 2048-bp sequence) are bit-identical between 4.57.6 without the patch and 5.22.2 with it (max abs diff 0.0). post_init() does not re-initialize loaded weights.
calbors changed pull request status to merged

Sign up or log in to comment