Commit History
-
Update README.md
Verified signature36bd0e1 -
Update README.md
Verified signatureef6962c -
Update README.md
Verified signaturebb765a1 -
Update README.md
Verified signatured1d6f56 -
Update README.md
Verified signatureLexCube is a BERT-based Masked Language Model fine-tuned on Italian legal texts to predict masked tokens ([MASK]) and capture domain-specific semantic and syntactic structures. It uses Masked Language Modeling (MLM) with Whole Word Masking (WWM), masking entire words rather than subword tokens to encourage deeper contextual learning of multi-token legal terms and complex sentence structures. The model was trained on 15,646 Italian legal documents from Infocube, including legislative acts, court rulings, resolutions, and regulatory communications. The texts are formal, technical, and highly structured, with numbered provisions and frequent references to laws and decrees. Tokenization produces an average of ~2,193 tokens per document, with some exceeding 11,000 tokens. **LexCube is suitable for: Predicting masked tokens in legal texts Generating embeddings for legal NLP tasks Transfer learning for downstream applications like classification, NER, or legal QA Due to confidentiality …c19fe8d -
Update README.md
Verified signature## Fine-tuning Method The model was fine-tuned on Italian legal texts using **Masked Language Modeling (MLM) with Whole Word Masking (WWM)**. - **Whole Word Masking** ensures that when a word is masked, all its subword tokens are masked together. - **Masking probability**: 15% - Implemented via `DataCollatorForWholeWordMask` from Hugging Face Transformers. This approach improves the model’s understanding of complex legal terminology and encourages learning deeper contextual representations. ## Dataset - **Source**: Provided by *Infocube*, a company specializing in legal data solutions. - **Size**: 15,646 documents. - **Language**: Italian. - **Domain**: Legal and administrative texts. - **Characteristics**: - Formal and technical legal language. - Frequent references to laws, decrees, and legislative articles. - Structured format with numbered provisions and cross-citations. - Avg. length: ~909 words (≈2,193 tokens per document). Some documents exceed 11k t…50444a8 -
Update README.md
Verified signature69c7bbf -
initial commit
Verified signature9dbe1c9
