| --- |
| language: |
| - en |
| library_name: transformers |
| pipeline_tag: feature-extraction |
| --- |
| A RoBERTa model trained using Data2Vec based on the paper [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://arxiv.org/abs/2202.03555).<br> |
| This model is provided here for [this repo](https://github.com/AryanShekarlaban/data2vec-pytorch) but was NOT trained using that codebase but instead, copied from `facebook/data2vec-text-base` for convenience and reproducibility. |
|
|
| ### BibTeX entry and citation info |
|
|
| ```bibtex |
| @misc{https://doi.org/10.48550/arxiv.2202.03555, |
| doi = {10.48550/ARXIV.2202.03555}, |
| url = {https://arxiv.org/abs/2202.03555}, |
| author = {Baevski, Alexei and Hsu, Wei-Ning and Xu, Qiantong and Babu, Arun and Gu, Jiatao and Auli, Michael}, |
| keywords = {Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences}, |
| title = {data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language}, |
| publisher = {arXiv}, |
| year = {2022}, |
| copyright = {arXiv.org perpetual, non-exclusive license} |
| } |
| ``` |
