how to consume the output merge model?
#2
by vhug - opened
I see that you guys have modified the mergekit and added the custom inference code. After trying to modify the mergekit-moe, I finally have a merged phi2 model, but honestly no clue on how to consume it, I am going through the configuration_phi.py and modeling_phi.py from your repo, probably I cannot use them as such because of the way I modified the mergekit-moe or is that irrelevant?
cause now running the inference script while using your configuration_phi.py and modeling.py gives me an error with accelerate lib as follows. Could you please shed some light? @vince62s