UltraBERT: Beyond the Limit of BERT Pre-Training. Pre-trained 150M/380M encoders and a text embedding model, English, 32K context.