NLP-CISUC/humetrix-resources

Humetrix resources

Resources downloaded automatically by the Humetrix library.

Contents

Folder	Content	Used by
glove/{lang}/	GloVe 300d: embeddings.safetensors (key embeddings) + vocab.txt (one word per line, aligned with rows)	Quantum Entropy metrics
skipgram/{lang}/	Distant skip-gram (SGNS, 300d) weights. Vocabulary: ngrams/{lang}/1gram.csv	Kao et al. metrics
ngrams/{lang}/	Unigram and trigram frequency tables	Kao et al. metrics

Portuguese GloVe embeddings are not hosted here; Humetrix downloads them from nilc-nlp/glove-300d (Hartmann et al., 2017).

Sources and attribution

• GloVe en: Pennington et al. (2014), Stanford NLP. Public domain (PDDL).
• GloVe es: trained by us on the Spanish Billion Words Corpus (Cardellino, 2019), CC BY-SA 4.0.
• GloVe fr: trained by us with data and code from French-Word-Embeddings, MIT License.
• GloVe zh: trained by us on Chinese Wikipedia (2023-11-01 dump), CC BY-SA 4.0.
• Skip-gram: trained by us on BookCorpus (en), BrWaC (pt), CroissantLLM (fr), SBWC (es) and Chinese Wikipedia (zh).
• N-grams en/es/fr/zh: derived from Google Books Ngram (CC BY 3.0) via google-books-ngram-frequency.
• N-grams pt: derived from BrWaC via Portuguese-Ngram-Frequencies.

Citation

To be announced