ifire's picture
Initial mirror of VectorSpaceLab/EditScore@4609c5d
da50cbf verified
Raw History Blame Contribute Delete
110 Bytes
import torch.nn.functional as F
def swiglu(x, y):
return F.silu(x.float(), inplace=False).to(x.dtype) * y