The Volctrans Machine Translation System for WMT20

L Wu, X Pan, Z Lin, Y Zhu, M Wang, L Li - arXiv preprint arXiv:2010.14806, 2020

… We use all parallel data available: Eu- roparl v10, ParaCrawl v5.1, Common Crawl corpus, News Commentary v15, Wiki Titles v2, Tilde Rapid corpus and WikiMatrix corpus … Each part contains 10M common crawl sentences and 3M Newscrawl sentences …


TheNorth@ HaSpeeDe 2: BERT-based Language Model Fine-tuning for Italian Hate Speech Detection

… ERT. AlBERTo was pretrained on TWITA, that is a collection of Italian tweets (Polignano et al., 2019b). UmBERTo was pretrained on Commoncrawl ITA exploiting OSCAR Italian large corpus (Parisi et al., 2020). Finally, PoliBERT …


This is a post-peer-review, pre-copyedit version of an article in press in Motivation and Emotion. The final authenticated version is available online at: http://dx. doi. org …

… datasets. Based on these experiments we decided to use Facebook's FastText subword embeddings of 300 dimensions trained on Common Crawl (600 billion tokens).5 This is the set of pre-trained vectors that we …


This is a post-peer-review, pre-copyedit version of an article in press in Motivation and Emotion. The final authenticated version will be available online at: http://dx. doi …


Tight Integrated End-to-End Training for Cascaded Speech Translation

P Bahar, T Bieschke, R Schlüter, H Ney - arXiv preprint arXiv:2011.12167, 2020

… For MT training on En→De, we utilize the parallel data allowed for the IWSLT 2020. After filtering the noisy corpora, namely ParaCrawl, CommonCrawl, Rapid and OpenSubtitles2018, we end up with almost 27M bilingual text sentences …


Tilde at WMT 2020: News Task Systems

R Krišlauks, M Pinnis - arXiv preprint arXiv:2010.15423, 2020

… translation. In order to make use of the Polish CommonCrawl corpus, we scored sentences using the in-domain language models and selected top-scoring sentences as additional monolingual data for back-translation. Many …


Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!

S Sia, A Dalmia, SJ Mielke - arXiv preprint arXiv:2004.14914, 2020


To BERT or Not to BERT: Comparing Task-specific and Task-agnostic Semi-Supervised Approaches for Sequence Tagging

K Bhattacharjee, M Ballesteros, R Anubhai, S Muresan… - arXiv preprint arXiv …, 2020

KBMBR Anubhai, S Muresan, JMFLY Al, OA AI

… Cloze (Baevski et al., 2019) and BERT-MRC+DSC (Li et al., 2019) are SOTA baselines for CONLL-2003 and CONLL-2012, respectively, for this task. Baevski et al. (2019) also use subsampled Common Crawl and News Crawl …


Tohoku-AIP-NTT at WMT 2020 News Translation Task

S Kiyono, T Ito, R Konno, M Morishita, J Suzuki - … of the Fifth Conference on Machine …, 2020

… 2.2 Monolingual Corpus The origins of the monolingual corpus in our system are the Europarl, NewsCommentary, and en- tire NewsCrawl (2008-2019) corpora for English and German, and the Europarl …


Page 1. Topics in Sequence-to-Sequence Learning for Natural Language Processing Roee Aharoni Ph.D. Thesis Submitted to the Senate of Bar-Ilan University Ramat Gan, Israel May 2020 Page 2. This work was …


Touché: First Shared Task on Argument Retrieval

A Bondarenko, M Hagen, M Potthast, H Wachsmuth…

… Systems. pp. 44–52 (2012) 2. Bevendorff, J., Stein, B., Hagen, M., Potthast, M.: Elastic ChatNoir: Search Engine for the ClueWeb and the Common Crawl. In: Proceedings of the 40th European Conference on IR Research (ECIR). pp …


Toward building recommender systems for the circular economy: Exploring the perils of the European Waste Catalogue

G van Capelleveen, C Amrit, H Zijm, DM Yazan, A Abdi - Journal of Environmental …

JavaScript is disabled on your browser. Please enable JavaScript to use all the features on this page. Skip to main content Skip to article …


Towards Context-Aware Opinion Summarization for Monitoring Social Impact of News

A Ramón-Hernández, A Simón-Cuevas, MMG Lorenzo… - Information, 2020

… Specifically, those vectors are generated by using the word2vec pre-trained model included in the es_core_news_md model of the spaCy library, which includes 300-dimensional vectors trained using FastText CBOW on Wikipedia …


Towards countering hate speech against journalists on social media

P Charitidis, S Doropoulos, S Vologiannidis… - Online Social Networks and …, 2020


Towards Effective Utilization of Pretrained Language Models—Knowledge Distillation from BERT

… For non-contextual embeddings, there are multiple pre-trained word vectors, such as word2vec trained on Google News, GloVe [60] trained on Wikipedia/Gigaword/Common Crawl, and fastText[67] trained on Wikipedia/Common Crawl …


Towards Efficient and Reproducible Natural Language Processing

… multiple epochs is standard). For example, the July 2019 Common Crawl contains 242 TB of uncompressed data,8 so even storing the data is expensive … 7https://opensource.google.com/projects/open-images-dataset …


Page 1. April 27, 2020 DRAFT Thesis Proposal Towards Generalized Neural Semantic Parsing Pengcheng Yin April 27, 2020 Language Technologies Institute School of Computer Science Carnegie Mellon University Pittsburgh, PA 15123 Thesis Committee …


Z Wang, Q Li, J Song, H Wang, L Sun - Proceedings of The Web Conference 2020, 2020

Page 1. Towards IP-based Geolocation via Fine-grained and Stable Webcam Landmarks Zhihao Wang Institute of Information Engineering Chinese Academy of Sciences School of Cyber Security, University of Chinese Academy …


Towards Orthographic and Grammatical Clinical Text Correction: a First Approach

… Their application to GEC is based on the idea that correct sequences are bound to have a higher probability score than incorrect ones. They are very dependent on the data that is used to build them, and so large corpora …


Towards Useful Word Embeddings

V Novotný, M Štefánik, D Lupták, P Sojka

… The size of our dataset is only 4% of the Common Crawl dataset used by Mikolov et al … We will also train our word vector models using larger corpora such as Common Crawl to enable meaningful comparison to sota results. Acknowledgments …


Towards Visual Dialog for Radiology

O Kovaleva, C Shivade, S Kashyap, K Kanjaria, J Wu… - Proceedings of the 19th …, 2020

… the models, (b) domain-independent GloVe Common Crawl embeddings (Pennington et al., 2014), and (c) domain-specific fastText embeddings trained by (Romanov and Shivade, 2018). The latter are initialized with GloVe …


Y Liu, J Lin, J Cleland-Huang - arXiv preprint arXiv:2006.16940, 2020

Page 1. Traceability Support for Multi-Lingual Software Projects Yalin Liu, Jinfeng Lin, Jane Cleland-Huang University of Notre Dame Notre Dame, IN yliu26@nd.edu, jlin6@nd.edu,JaneHuang@nd.edu ABSTRACT Software …


Tracing the emergence of gendered language in childhood

B Prystawski, E Grant, A Nematzadeh, SWS Lee…

… We used three commonly-used sets of pre-trained word em- beddings: the word2vec embeddings trained on the Google News corpus (Mikolov et al., 2013a), the GloVe embeddings trained on the Common Crawl corpus, and …


Train Hard, Finetune Easy: Multilingual Denoising for RDF-to-Text Generation

Z Kasner, O Dušek - Proceedings of the 3rd WebNLG Workshop on Natural …, 2020

… al., 2019). Adopting BART's objective and architecture, mBART (Liu et al., 2020) is pre-trained on the large-scale CC25 corpus extracted from Common Crawl, which contains data in 25 languages (Wenzek et al., 2020). The …


Transformer based Deep Intelligent Contextual Embedding for Twitter sentiment analysis

U Naseem, I Razzak, K Musial, M Imran - Future Generation Computer Systems, 2020


Transforming machine translation: a deep learning system reaches news translation quality comparable to human professionals

M Popel, M Tomkova, J Tomek, Ł Kaiser, J Uszkoreit… - Nature Communications, 2020

The quality of human translation was long thought to be unattainable for computer translation systems. In this study, we present a deep-learning system, CUBBITT, which challenges this view. In a context-aware blind evaluation …


Translation Artifacts in Cross-lingual Transfer Learning

M Artetxe, G Labaka, E Agirre - arXiv preprint arXiv:2004.04721, 2020

… We first collect the premises from a filtered version of CommonCrawl (Buck et al., 2014), taking a subset of 5 websites that represent a diverse set of genres: a newspaper, an economy forum, a celebrity magazine, a literature blog, and a consumer magazine …


Translation System and Method

N Bertoldi, D Caroselli, MA Farajian, M Federico… - US Patent App. 16/118,273, 2020

US20200073947A1 - Translation System and Method - Google Patents. Translation System and Method. Download PDF Info. Publication number US20200073947A1. US20200073947A1 US16/118,273 US201816118273A US2020073947A1 …


TransQuest: Translation Quality Estimation with Cross-lingual Transformers

T Ranasinghe, C Orasan, R Mitkov - arXiv preprint arXiv:2011.01536, 2020

… to acquire. Instead, XLM-R trains RoBERTa(Liu et al., 2019) on a huge, multilingual dataset at an enormous scale: unlabelled text in 104 languages is extracted from CommonCrawl datasets, totalling 2.5TB of text. It is trained …


Triclustering in Big Data Setting

D Egurnov, DI Ignatov, D Tochilkin - arXiv preprint arXiv:2010.12933, 2020

Page 1. Triclustering in Big Data Setting Dmitry Egurnov, Dmitry I. Ignatov, and Dmitry Tochilkin Abstract In this paper, we describe versions of triclustering algorithms adapted for efficient calculations in distributed environments …


Triple E-Effective Ensembling of Embeddings and Language Models for NER of Historical German.

… We use the FastText embeddings trained on Wikipedia (FastText Wiki) and Common Crawl (FastText CC) in a ”classic” word embeddings manner, that means we do not use subwords … BPE MultiBPEmb Wikipedia < 7000 …


TULIP: A Five-Star Table and List-from Machine-Readable to Machine-Understandable Systems

J Nandakwang, P Chongstitvatana - Linked Open Data-Applications, Trends and …, 2020

Currently, Linked Data is increasing at a rapid rate as the growth of the Web. Aside from new information that has been created exclusively as Semantic Web-ready, part of them comes from the transformation of existing structural …


TweetBERT: A Pretrained Language Representation Model for Twitter Text Analysis

MMA Qudar, V Mago - arXiv preprint arXiv:2010.11091, 2020

… It has been pre-trained on an extremely large, five different types of corpora: BookCorpus, English Wikipedia, CC-News (collected from CommonCrawl News) dataset, OpenWebText, a WebText corpus [23], and Stories, a dataset containing story-like content [23] …


Two-Level Transformer and Auxiliary Coherence Modeling for Improved Text Segmentation

G Glavaš, S Somasundaran - arXiv preprint arXiv:2001.00891, 2020

… In all our experiments we use 300dimensional monolingual FASTTEXT word embeddings pretrained on the Common Crawl corpora of respective languages: EN, CS, FI, and TR.9 We induce a cross-lingual word embedding …


UHH-LT & LT2 at SemEval-2020 Task 12: Fine-Tuning of Pre-Trained Transformer Networks for Offensive Language Detection

G Wiedemann, SM Yimam, C Biemann - arXiv preprint arXiv:2004.11493, 2020

… languages at once (Conneau et al., 2019). The model itself is equivalent to RoBERTa, but the training data consists of texts from more than 100 languages filtered from the CommonCrawl1 dataset. ALBERT – A Lite BERT for Self …


Uncertainty-Aware Machine Support for Paper Reviewing on the Interspeech 2019 Submission Corpus

L Stappen, G Rizos, M Hasan, T Hain, BW Schuller

… compressed to 300 dimensions. FastText and GloVe are based on the Common Crawl (1.9 M unique words, 840 B tokens) and Word2Vec on the GoogleNews (3 M unique words, 100 B total) dataset. We additionally experimented …


Underlying Cause of Death Identification from Death Certificates using Reverse Coding to Text and a NLP Based Deep Learning Approach

V Della Mea, MH Popescu, K Roitero - Informatics in Medicine Unlocked, 2020

… XLM-R (a variation of XLM), trained on one hundred languages using more than two terabytes of filtered CommonCrawl data, outperformed multilingual BERT (mBERT) on a variety of cross-lingual benchmarks [5]. 3.3.4. XLNet …


Understanding phishers' strategies of mimicking uniform resource locators to leverage phishing attacks: A machine learning approach

JS Tharani, NAG Arachchilage - arXiv preprint arXiv:2007.00489, 2020

… webpage data downloaded from PhishTank2, OpenPhish3 and Legitimate ones are downloaded from Alexa4 and Common Crawl5 According … (4) 4 https://www.alexa.com/topsites/category/Computers/Internet/OntheW eb/W …


Understanding Word Embeddings and Language Models

JM Gomez-Perez, R Denaux, A Garcia-Silva - A Practical Guide to Hybrid Natural …, 2020

… 1) pre-trained contextualized word embeddings (ELMo), (2) pre-trained context-independent word embeddings learnt from Common Crawl (fastText), Twitter … Another version of this classifier using in addition fastText embeddings …


UniBO@ KIPoS: Fine-tuning the Italian “BERTology” for PoS-tagging Spoken Data

… project. Also for GilBERTo it is available only the uncased model. • UmBERTo4: the more recent model de- veloped explicitly for Italian, as far as we know, is UmBERTo ('Musixmatch/umbertocommoncrawl-cased-v1' – umC). As …


UninaStudents@ SardiStance: Stance Detection in Italian Tweets-Task A

M Moraca, G Sabella, S Morra - Proceedings of the 7th Evaluation Campaign of …, 2020

… As Master students, we approached these NLP topics for the first time. Therefore, we are aware 5https://huggingface.co/Musixmatch/umbertocommoncrawl-cased-v1 that our results are not at the state of the art in the field. However …


Unit Test Case Generation with Transformers

M Tufano, D Drain, A Svyatkovskiy, SK Deng… - arXiv preprint arXiv …, 2020

… It has been pre-trained on the Common Crawl dataset [32] constituting nearly a trillion words, an expanded version of the WebText [33] dataset, two internet-based books corpora (Books1 and Books2), and English-language Wikipedia …


Uniting Plain Language, Cognitive Fluency, and Believability

… considering at least four linguistic features (Romanyshyn, 2018). Similar to Randall (2019), Romanyshyn (2018) considers the frequency of a word using Common Crawl, a large corpus of web content. Taking this approach one step further, she lemmatizes the 1 …


UNITOR@ Sardistance2020: Combining Transformer-based Architectures and Transfer Learning for Robust Stance Detection

S Giorgioni, M Politi, S Salman, D Croce, R Basili - … of the 7th Evaluation Campaign of …, 2020

… with <negativo>3. 2https://huggingface.co/Musixmatch/ umberto-commoncrawlcased-v1 3We discarded the few available messages with mixed po- larity, to simplify the final classification task. Irony Detection. We speculate …


Unsupervised Cross-Lingual Part-of-Speech Tagging for Truly Low-Resource Scenarios

R Eskander, S Muresan, M Collins - Proceedings of the 2020 Conference on …, 2020

… essential when the domain of the training data is different from the one of the pre-trained em- beddings, which is the case in our learning setup, where we use the Bible data for training, while the XLM-R model is trained on text …


Unsupervised Cross-lingual Representation Learning for Speech Recognition

A Conneau, A Baevski, R Collobert, A Mohamed… - arXiv preprint arXiv …, 2020

… For comparison with [29] only, we train 4-gram n-gram language models on CommonCrawl data [25, 50] for Assamese (140MiB of text data), Swahili (2GiB), Tamil (4.8GiB) and Lao (763MiB); for this experiment only we report word error rate (WER). 3.2 Training details …


Unsupervised Domain Clusters in Pretrained Language Models

R Aharoni, Y Goldberg - arXiv preprint arXiv:2004.02105, 2020

… exact requirements from such data with respect to all the aforementioned aspects. On top of that, domain labels are usually unavailable – eg in large-scale web-crawled data like Common Crawl1 which was recently used to …


Unsupervised Evaluation of Human Translation Quality

… publicly available monolingual word embeddings. Specifically, we first use the monolingual word embeddings, which are trained on Wikipedia and Common Crawl using fastText (Grave et al., 2018). Because our dataset contains …


Unsupervised Label Refinement Improves Dataless Text Classification

Z Chu, K Stratos, K Gimpel - arXiv preprint arXiv:2012.04194, 2020

… We use the 300 dimensional GloVe vectors7 trained on Common Crawl.8 We experiment with two distance functions when using GloVe: cosine and L2 … 7http://nlp.stanford.edu/ data/glove.840B.300d.zip 8https://commoncrawl.org/ ROBERTA Dual Encoder …


Unsupervised Question Decomposition for Question Answering

… Specifically, by leveraging >10M questions from Common Crawl, we learn to map from the distribution of multi-hop questions to the distribution of single-hop subquestions … We retrieve candidates from a corpus of 10M simple …


UPB at GermEval-2020 Task 3: Assessing Summaries for German Texts using BERTScore and Sentence-BERT

… bert-base-german-europeana-uc 2 Uncased Europeana newspapers bert-base-germanuc2 Uncased Wikipedia, Subtitles, News, Commoncrawl literary-german-bert3 Uncased German Fiction Literature bert-adapted-german-press4 Uncased Newspapers …


F Liu, R Lebret, D Orel, P Sordet, K Aberer - arXiv preprint arXiv:2004.11449, 2020

Page 1. 1 Upgrading the Newsroom: An Automated Image Selection System for News Articles FANGYU LIU∗, Language Technology Lab (LTL), University of Cambridge, United Kingdom RÉMI LEBRET, Distributed Information …


UR NLP@ HaSpeeDe 2 at EVALITA 2020: Towards Robust Hate Speech Detection with Contextual Embeddings

… XLM-R is based on XLM and RoBERTa. It is trained on data covering 100 languages in a very large (2TB) CommonCrawl. Transformer document embeddings are obtained from (the large version of) XLM-R. In addition Page 3 …


Urban Dictionary Embeddings for Slang NLP Applications

S Wilson, W Magdy, B McGillivray, K Garimella… - Proceedings of The 12th …, 2020

… with the goal of producing generally applicable word embeddings, many popular pre-trained word embeddings have been fit to large and diverse corpora of text from the web such as the Common Crawl.3 In … 2 http://smash …


URL-based Phishing Attack Detection by Convolutional Neural Networks

J Nowak, M Korytkowski, P Najgebauer, M Wozniak…

… The database downloaded during the article writing contained 10,604 records. To obtain legitimate websites, the second part of the training dataset was downloaded from the Common Crawl Foundation (http://commoncrawl.org/) …


Using Natural Language Preprocessing Architecture (NLPA) for Big Data Text Sources

M Novo-Lourés, R Pavón, R Laza, D Ruano-Ordas… - Scientific Programming, 2020

Journals; Publish with us; Publishing partnerships; About us; Blog. Scientific Programming. +Journal Menu. PDF. Journal overview. For authorsFor reviewersFor editorsTable of Contents Special Issues.


J Navrozidis, H Jansson - LU-CS-EX, 2020

Page 1. MASTER'S THESIS 2020 Using Natural Language Processing to Identify Similar Patent Documents Hannes Jansson, Jakob Navrozidis ISSN 1650-2884 LU-CS-EX: 2020-05 DEPARTMENT OF COMPUTER …


Using Probabilistic Soft Logic to Improve Information Extraction in the Legal Domain

B Kirsch, S Giesselbach, T Schmude, M Völkening…

… spaCy Classifier: This architecture is based on a CNN with mean pooling and a final feed-forward layer. The network is fed with pretrained word embeddings trained on the German Wikipedia and the German common crawl (Ortiz Suárez et al., 2019).9 …


Page 1. Using Publisher Partisanship for Partisan News Detection A Comparison of Performance between Annotation Levels Chia-Lun Yeh Page 2. Using Publisher Partisanship for Partisan News Detection A …


Using Sentences as Semantic Representations in Large Scale Zero-Shot Learning

YL Cacheux, HL Borgne, M Crucianu - arXiv preprint arXiv:2010.02959, 2020

… For the same reason, we used FastText and Glove models pre-trained on Common Crawl We used a 300-dimension version for all three … Fasttext: https://fasttext.cc/docs/en/englishvectors.html (version trained on Common Crawl with 600B tokens, no subword information) …


Using Word Embeddings to Learn a Better Food Ontology. Front

J Youn, T Naravane, I Tagkopoulos - Artif. Intell, 2020

… W ikinews W ikipedia 2017 + UMBC webbase + s tatmt.org 0.313 2.98 Crawl Common Crawl 0 .317 3.00 Word2vec (Mikolov …


ValNorm: A New Word Embedding Intrinsic Evaluation Method Reveals Valence Biases are Consistent Across Languages and Over Decades

A Toney, A Caliskan - arXiv preprint arXiv:2006.03950, 2020

… We choose six widely used pre-trained word embedding sets, listed in Table 2, to compare ValNorm's performance on different algorithms (GloVe, fastText, word2vec) and training corpora (Common Crawl, Wikipedia, OpenSubtitles …


Vandalism Detection in Crowdsourced Knowledge Bases

… Manual OpenStreetMap, Uniprot, WordNet, MusicBrainz, IMDb Wikipedia, WikiHow, YouTube Wikia/FANDOM, StackExchange, Quora, Yahoo Answers DBpedia, YAGO, NELL dblp, BabelNet Internet Archive, Common Crawl NASA …


Variational Transformers for Diverse Response Generation

Z Lin, GI Winata, P Xu, Z Liu, P Fung - arXiv preprint arXiv:2003.12738, 2020

… embeddings. The first is EMBFT (Liu et al., 2016) that calculates the average of word embeddings in a sentence using FastText (Mikolov et al., 2018) which is trained with Common Crawl and Wikipedia data. We use FastText …


VECO: Variable Encoder-decoder Pre-training for Cross-lingual Understanding and Generation

F Luo, W Wang, J Liu, Y Liu, B Bi, S Huang, F Huang… - arXiv preprint arXiv …, 2020

… We adopt the same 250K vocabulary that is also used by XLM-R (Conneau et al., 2019) and mBART (Liu et al., 2020b). Pre-Training Datasets For monolingual training datasets, we reconstruct Common-Crawl Corpus used in XLM-R (Conneau et al., 2019) …


Video Question Answering on Screencast Tutorials

… visual cues, and graph embeddings. All the models have the word embeddings initialized with the 300-dimensional pretrained fastText [Bojanowski et al., 2017] vectors on Common Crawl dataset. The convolutional layer in …


Visual and Textual Deep Feature Fusion for Document Image Classification

S Bakkali, Z Ming, M Coustaty, M Rusinol - Proceedings of the IEEE/CVF Conference …, 2020

… FastText algorithm we used was pretrained on 2 million word vectors trained on Common Crawl (600B tokens), and uses 1,999,996 word vectors. Bert: Bert [11] is a contextualized bidirectional word embedding based on the transformer architecture …


Visual Relations Augmented Cross-modal Retrieval

Y Guo, J Chen, H Zhang, YG Jiang - … of the 2020 International Conference on …, 2020

… vector with the corresponding visual feature. The label embedding vector is obtained with a learnable embedding layer initialized by GloVe [25] that pre-trained on the Common-Crawl dataset. Given a set of object categories …


Visualizing and Interpreting RNN Models in URL-based Phishing Detection

T Feng, C Yue - Proceedings of the 25th ACM Symposium on Access …, 2020

… The legitimate URLs came from the Common Crawl (www.commoncrawl.org) open web searching database, while the phishing URLs came from the popular PhishTank (www.phishtank.com) phishing website repository. In …


Wat zei je? Detecting Out-of-Distribution Translations with Variational Transformers

TZ Xiao, AN Gomez, Y Gal - arXiv preprint arXiv:2006.08344, 2020

… The following datasets were used in our experiments: • WMT EN ↔ DE: The training set for translation tasks between English (EN) and German (DE) composed of news-commentary-v13 with 284k sentences pairs, wmt13 …


Weakly-Supervised Multi-Level Attentional Reconstruction Network for Grounding Textual Queries in Videos

Y Song, J Wang, L Ma, Z Yu, J Yu - arXiv preprint arXiv:2003.07048, 2020

… For each second we uniformly sample 16 frames as input to C3D, and obtain a 4096-dimentional visual feature from fc6 layer. Each word from the query is represented by GloVe [22] word embedding vector pre-trained on Common Crawl …


Web Crawl Processing on Big Data Scale

JM Patel - Getting Structured Data from the Internet, 2020

… We got this domain ranks file and column names from the common crawl blog post (https://commoncrawl.org/2020/02/host-and-domain-level-webgraphs-novdecjan-2019-2020/); they publish new domain ranks about four …


Web Table Extraction, Retrieval, and Augmentation: A Survey

S Zhang, K Balog - ACM Transactions on Intelligent Systems and …, 2020

… Table corpora Type #tables Source WDC 2012 Web Table Corpus Web tables 147M Web crawl (Common Crawl) WDC 2015 Web Table Corpus Web tables 233M Web crawl (Common Crawl) Dresden Web Tables Corpus …


Webis at TREC 2019: Decision Track

A Bondarenko, M Fröbe, V Kasturia, M Völske, B Stein…

… In Proceedings of SIGIR 2017. 1419–1420. [2] Janek Bevendor , Benno Stein, Ma hias Hagen, and Martin Po hast. 2018. Elastic ChatNoir: Search Engine for the ClueWeb and the Common Crawl. In Proceedings of ECIR 2018. 820–824 …


Webis at TREC 2020: Health Misinformation Track

A Bondarenko, M Fröbe, S Günther, M Hagen… - 2020

J Bevendor, A Bondarenko, M Fröbe, S Günther…

… During retrieval, we used ChatNoirs existing weighting scheme for the two Common Crawl snapshots, which combines BM25 scores of multiple elds … we relax the precondition of documents' 1We have indexed a 2015 and …


Webly Supervised Semantic Embeddings for Large Scale Zero-Shot Learning

YL Cacheux, A Popescu, HL Borgne - arXiv preprint arXiv:2008.02880, 2020

… prototypes. These embeddings are extracted from generic large scale text collections such as Wikipedia [21,34] or Common Crawl [6,33] … use. For GloVe on ImageNet, the model pretrained on Common Crawl has the best performance …


WeChat Neural Machine Translation Systems for WMT20

F Meng, J Yan, Y Liu, Y Gao, X Zeng, Q Zeng, P Li… - arXiv preprint arXiv …, 2020

… Commentary, Common Crawl and Gigaword corpus. The English monolingual data includes News crawl, News discussions, Europarl v10, News Commentary, Common Crawl, Wiki dumps and the Gigaword corpus. After …


WEFE: The Word Embeddings Fairness Evaluation Framework

… The following are the pre-trained em- bedding models that we consider: 1) conceptnet, 2) fasttextwikipedia, 3) glove-twitter, 4) glove-wikipedia, 5) lexveccommoncrawl, 6) word2vec-googlenews, and 7) word2vecgender-hard …


WEmbSim: A Simple yet Effective Metric for Image Captioning

N Sharif, L White, M Bennamoun, W Liu, SAA Shah - arXiv preprint arXiv:2012.13137, 2020

… Page 5. TABLE I T - . Name Source Dims Corpus Corpus Size Vocabulary Size GloVE 840B [12] 300 Common Crawl 8.4 · 1011 2 · 106 Word2vec [6] 300 Google News (100B) 1.0 · 1011 3 · 106 FastText [13] 300 Wikipedia 4.0 · 109 3 · 106 …


What determines the order of adjectives in English? Comparing efficiency-based theories using dependency treebanks

R Futrell, W Dyer, G Scontras - Proceedings of the 58th Annual Meeting of the …, 2020

… sklearn.cluster.KMeans applied to a pretrained set of 1.9 million 300-dimension GloVe vectors2 generated from the Common Crawl corpus … Table 1a shows the accuracies of our predictors in predicting held-out …


What Sparks Joy: The AffectVec Emotion Database

S Raji, G de Melo - Proceedings of The Web Conference 2020, 2020

… We consider the cosine similarity of word– emotion pairs in word2vec trained on the Google News corpus [18], GloVe [26] trained on Twitter (200-dim.) and CommonCrawl (840B, 300-dim.), as well as the counterfitted vectors by Mrksic et al. [24]. Results …


What the [MASK]? Making Sense of Language-Specific BERT Models

D Nozza, F Bianchi, D Hovy - arXiv preprint arXiv:2003.02912, 2020

… OSCAR (Open Super-large Crawled Almanach coRpus) (Or- tiz Suárez et al., 2019) is a huge multilingual corpus obtained by filtering the Common Crawl corpus, which is a parallel multilingual corpus comprised of crawled documents from the internet …


When and Why is Unsupervised Neural Machine Translation Useless?

Y Kim, M Graça, H Ney - arXiv preprint arXiv:2004.10581, 2020

YKM Graça, H Ney - 22nd Annual Conference of the European Association …

… However, for low-resource language pairs, it is difficult to match the data domain of both sides on a large scale. For example, our monolingual data for Kazakh is mostly from Wikipedia and Common Crawl, while the English data is solely from News Crawl …


When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models

B Muller, A Anastasopoulos, B Sagot, D Seddah - arXiv preprint arXiv:2010.12858, 2020

… al., 2019). OSCAR is a corpus extracted from a Common Crawl Web snapshot.3 It provides a significant 2Also see the discussion in Section §3.2 on the script distributions in mBERT. 3http://commoncrawl.org/ Language (iso …


When do Word Embeddings Accurately Reflect Surveys on our Beliefs About People?

K Joseph, JH Morgan - arXiv preprint arXiv:2004.12043, 2020

Page 1. When do Word Embeddings Accurately Reflect Surveys on our Beliefs about People? Kenneth Joseph Computer Science and Engineering University at Buffalo Buffalo, NY, 14226 kjoseph@buffalo.edu Jonathan H. Morgan …


When Does Unsupervised Machine Translation Work?

K Marchisio, K Duh, P Koehn - arXiv preprint arXiv:2004.05516, 2020

… “News crawl” (News) and “Common Crawl” (CC) settings determine whether the system can flexibly handle diverse datasets. Specifics of the datasets used are described in subsequent subsections … UN = United Nations …


Which* BERT? A Survey Organizing Contextualized Encoders

P Xia, S Wu, B Van Durme - arXiv preprint arXiv:2010.00854, 2020

… Raffel et al. (2019) curate a 745GB subset of Common Crawl (CC),10 which starkly contrasts with the 13GB used in BERT … 9https://sites.google.com/ view/ sustainlp2020/shared-task 10https://commoncrawl.org/ scrapes publicly …


Who is asking? humans and machines experience

… licenses/by/4.0/). Similarly, the motivation behind the recent study by Thompson and Jian [14] based on two Common Crawl samples of the web was to quantify the use of HTTP DOIs versus URLs of landing pages. They found …


Why are events important and how to compute them in geospatial research?

… GPT-3 is a gigantic neural network with 175 billion input parameters and 96 layers of transformer decoders, each of which has 1.8 billion parameters, and is pre-trained with 45TB (499 billion tokens) compressed data from five …


Why Not Simply Translate? A First Swedish Evaluation Benchmark for Semantic Similarity

T Isbister, M Sahlgren - arXiv preprint arXiv:2009.03116, 2020

… et al., 2018). We use the CBOW model that has been trained on Common Crawl and Wikipedia.6 As with Word2Vec, the vectors for sentences are obtained by averaging the embedding vector for each word. BERT: Deep Transformer …


Why Overfitting Isn't Always Bad: Retrofitting Cross-Lingual Word Embeddings to Dictionaries

M Zhang, Y Fujinuma, MJ Paul, J Boyd-Graber

… We align English embeddings with six target languages: German (DE), Spanish (ES), French (FR), Italian (IT), Japanese (JA), and Chinese (ZH). We use 300-dimensional fastText vectors trained on Wikipedia and Common Crawl (Grave et al., 2018) …


Wide range screening of algorithmic bias in word embedding models using large sentiment lexicons reveals underreported bias types

… This work systematically analyzed 3 popular word embeddings methods: Word2vec (Skip-gram) [4], Glove [9] and FastText [10], externally pretrained on a wide array of corpora such as Google News, Wikipedia, Twitter or Common Crawl …


WikiAsp: A Dataset for Multi-domain Aspect-based Summarization

H Hayashi, P Budania, P Wang, C Ackerson… - arXiv preprint arXiv …, 2020

… sections of Wikipedia from referenced web pages. Following the WikiSum data generation script,3 we first crawled cited references covered by CommonCrawl for each Wikipedia article. We then recover all the sections4 of …


Will it Unblend?

Y Pinter, CL Jacobs, J Eisenstein - arXiv preprint arXiv:2009.09123, 2020

Page 1. Will it Unblend? Yuval Pinter School of Interactive Computing Georgia Institute of Technology Atlanta, GA, USA uvp@gatech.edu Cassandra L. Jacobs Department of Psychology University of Wisconsin Madison, WI, USA cjacobs2@wisc.edu …


Word associations and the distance properties of context-aware word embeddings

MA Rodriguez, P Merlo - Proceedings of the 24th Conference on Computational …, 2020

… However, in this work, we used the pre-trained FASTTEXT embeddings provided by the official site of FASTTEXT, that we expressly do not modify.3 The embeddings are trained on 600-billion tokens from …


Word Embedding Evaluation for Sinhala

D Lakmal, S Ranathunga, S Peramuna, I Herath - Proceedings of The 12th Language …, 2020

… Common Crawl can be considered as a precious starting point for building a cleaned large corpus for … Common Crawl monthly dataset only contains 0.007% of content in Sinhala4, however, this amount is still … 4https …


WORD EMBEDDINGS IN ROMANIAN FOR THE RETAIL BANKING DOMAIN

… In addition, Facebook released a year later another version of FastText pre-trained word embeddings, trained on Common Crawl and Wikipedia [4]. Another pre-trained word embeddings in Romanian can be found at …


Word Embeddings Inherently Recover the Conceptual Organization of the Human Mind

V Swift - arXiv preprint arXiv:2002.10284, 2020

… Sub-word information was incorporated on the basis of n-grams (length = 5), with a window size of 5 and 10 negatives, and a step size of .05. The English model was trained on a Common Crawl corpus comprised of English text from 2.96 billion webpages …


Word meaning in minds and machines

BM Lake, GL Murphy - arXiv preprint arXiv:2008.01766, 2020

… is illustrated in Figure 1A. CBOW has been trained on tremendous corpora; for instance, in this article, we analyze a large-scale CBOW model trained on the Common Crawl corpus of 630 billion words. CBOW learns a word …


Word Representations for Named Entity Recognition

… Transformers: Bertin (Gigaword+Wikipedia), XLM-RoBERTa (Common Crawl) and mBERT (Wikipedia + books) • Project annotations (various strategies) Page 54 … BETO (various sources) – XLM-RoBERTa (Common Crawl 2.5TB) – mBERT (Wikipedia + books) …


Word Representations for Neural Network Based Myanmar Text-to-Speech System

… In [21], the size of word vectors is small, and it contains about 55K entries for Myanmar language and can be downloaded from the link [23]. In [22], the word vectors are trained on Common Crawl and Wikipedia using fastText …


Word Rotator's Distance

S Yokoi, R Takahashi, R Akama, J Suzuki, K Inui - Proceedings of the 2020 …, 2020

Page 1. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 2944–2960, November 16–20, 2020. c 2020 Association for Computational Linguistics 2944 Word Rotator's Distance …


S Yokoi, R Takahashi, R Akama, J Suzuki, K Inui - arXiv preprint arXiv:2004.15003, 2020

Page 1. Word Rotator's Distance: Decomposing Vectors Gives Better Representations Sho Yokoi1 Ryo Takahashi1,2 Reina Akama1,2 Jun Suzuki1,2 Kentaro Inui1,2 1 Tohoku University 2 RIKEN {yokoi, ryo.t, reina.a, jun.suzuki, inui}@ecei.tohoku.ac.jp Abstract …


Word Sense Disambiguation for 158 Languages using Word Embeddings Only

V Logacheva, D Teslenko, A Shelmanov, S Remus… - arXiv preprint arXiv …, 2020

… The contributions of our work are the following: 1The full list languages is available at fasttext.cc and includes English and 157 other languages for which embeddings were trained on a combination of Wikipedia and CommonCrawl texts …


Word2Sent: A new learning sentiment‐embedding model with low dimension for sentence level sentiment classification

M Kasri, M Birjali, A Beni‐Hssane - Concurrency and Computation: Practice and …

Abstract Word embedding models become an increasingly important method that embeds words into a high dimensional space. These models have been widely utilized to extract semantic and syntactic feat...


Words Matter: Gender, Jobs and Applicant Behavior in India

S Chaturvedi, K Mahajan, Z Siddique - 2020

… Pennington et al., 2014). The 300 dimensional pretrained word vectors have been obtained by training the algorithm on web data from common crawl, and comprise 2.2 million unique words. Cosine similarity between any …


Words, constructions and corpora: Network representations of constructional semantics for Mandarin space particles

ACH Chen - Corpus Linguistics and Linguistic Theory, 2020

Jump to Content Jump to Main Navigation Publications. Subjects. Architecture and Design Arts Asian and Pacific Studies Business and Economics Chemistry Classical and Ancient Near Eastern Studies Computer Sciences Cultural …


Wrestling with Complexity in Computational Social Science: Theory, Estimation and Representation

S de Marchi - The SAGE Handbook of Research Methods in Political …, 2020


WT5?! Training Text-to-Text Models to Explain their Predictions

S Narang, C Raffel, K Lee, A Roberts, N Fiedel… - arXiv preprint arXiv …, 2020

… 1997; Ruder, 2017). In Raffel et al. (2019), this framework was used to pre-train Transformer (Vaswani et al., 2017) models on a large collection of unlabeled text drawn from the Common Crawl web scrape. We use the result …


X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models

Z Jiang, A Anastasopoulos, J Araki, H Ding, G Neubig - Proceedings of the 2020 …, 2020

Page 1. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 5943–5959, November 16–20, 2020. c 2020 Association for Computational Linguistics 5943 X-FACTR: Multilingual …


XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation

Y Liang, N Duan, Y Gong, N Wu, F Guo, W Qi, M Gong… - arXiv preprint arXiv …, 2020

… 2.1.2 Large Corpus (LC) Multilingual Corpus Following Wenzek et al. (2019), we construct a clean version of Common Crawl (CC)3 as the multilingual corpus … 2https://github.com/ attardi/wikiextractor. 3https://commoncrawl.org/. available in English …


Xiaomi's Submissions for IWSLT 2020 Open Domain Translation Task

Y Sun, M Guo, X Li, J Cui, B Wang - Proceedings of the 17th International Conference …, 2020

… And for unconstrained submission, we choose the largescale amounts of Commoncrawl Chinese10 and Japanese11 dataset as additional monolingual data for training LMs and executing BT to enhance our NMT systems …


YNU OXZ@ HaSpeeDe 2 and AMI: XLM-RoBERTa with Ordered Neurons LSTM for classification task at EVALITA 2020

X Ou, H Li - Proceedings of Sixth Evaluation Campaign of Natural …, 2020

… scale multi-language pre-training model. It can be un- derstood as a combination of XLM and RoBER- Ta. It is trained on 2.5 TB of newly created clean CommonCrawl data in 100 languages. Because the training of the model …


Zero Shot Domain Generalization

U Maniyar, AA Deshmukh, U Dogan… - arXiv preprint arXiv …, 2020

… Thus using semantic space helps us in the visual classification task. We use word embeddings of classes - in particular, simple GloVe embeddings [28] trained on Common Crawl corpus - as the semantic space in this work …


Zero-shot semantic segmentation using relation network

Page 1. University of Jyväskylä Faculty of Information Technology Yindong Zhang Zero-shot Semantic Segmentation using Relation Network Master's thesis of information technology May 28, 2020 Page 2. i Author: Yindong …