Hugging Face's logo

Spaces:
Ethosoft
/
NedoTokenizer-Demo
Running

App Files Community
NedoTokenizer-Demo / wasm-src /src /lib.rs

Commit History

  1. release: deploy 20GB max48 vocabulary

    Ethosoft TRUBA Deploy
    5f4471c

  1. release: deploy root-hard suffix-soft tokenizer

    Ethosoft TRUBA Deploy
    f6987ef

  2. fix: restore morphology tokens and readable unicode

    Ethosoft TRUBA Deploy
    f1d80ed

  3. release: deploy lexical byte-BPE tokenizer demo

    Ethosoft TRUBA Deploy
    7ae0777

  4. fix: render real surface vocabulary tokens

    Ethosoft TRUBA Deploy
    9adb645

  1. Deploy accepted NedoTokenizer core as static WASM demo

    Ethosoft TRUBA Deploy
    03058ad