Hugging Face's logo

flexitok
/
bpe_arb_Arab_8000

Standard Arabic
tokenizer
bpe
flexitok
fineweb2
Model card Files
xet
 Community
bpe_arb_Arab_8000

Commit History

  1. Upload safe_v2/flexitok--bpe_arb_Arab_8000_overlap.json with huggingface_hub

    gsaltintas
    Verified signature
    60edd95

  2. Upload safe_v2/participating_tokenizers.json with huggingface_hub

    gsaltintas
    Verified signature
    22213df

  3. Upload safe_v2/flexitok--bpe_arb_Arab_8000.yaml with huggingface_hub

    gsaltintas
    Verified signature
    6df191d

  4. Upload safe_v2/flexitok--bpe_arb_Arab_8000_info.json with huggingface_hub

    gsaltintas
    Verified signature
    ea49e25

  5. Upload safe_v2/flexitok--bpe_arb_Arab_8000_vocab.json with huggingface_hub

    gsaltintas
    Verified signature
    a94f2f5

  6. Upload safe_v2/flexitok--bpe_arb_Arab_8000_super_mapping.json with huggingface_hub

    gsaltintas
    Verified signature
    927b6ea

  1. Upload safe/flexitok--bpe_arb_Arab_8000_overlap.json with huggingface_hub

    gsaltintas
    Verified signature
    96fdf3c

  2. Upload safe/participating_tokenizers.json with huggingface_hub

    gsaltintas
    Verified signature
    051100a

  3. Upload safe/flexitok--bpe_arb_Arab_8000.yaml with huggingface_hub

    gsaltintas
    Verified signature
    2f6e2c8

  4. Upload safe/flexitok--bpe_arb_Arab_8000_info.json with huggingface_hub

    gsaltintas
    Verified signature
    a6ede60

  5. Upload safe/flexitok--bpe_arb_Arab_8000_vocab.json with huggingface_hub

    gsaltintas
    Verified signature
    5e78f80

  6. Upload safe/flexitok--bpe_arb_Arab_8000_super_mapping.json with huggingface_hub

    gsaltintas
    Verified signature
    8e14c98

  1. Upload fineweb2_hq/flexitok--bpe_arb_Arab_8000_overlap.json with huggingface_hub

    gsaltintas
    Verified signature
    8384471

  2. Upload fineweb2_hq/participating_tokenizers.json with huggingface_hub

    gsaltintas
    Verified signature
    6805476

  3. Upload fineweb2_hq/flexitok--bpe_arb_Arab_8000.yaml with huggingface_hub

    gsaltintas
    Verified signature
    ed8cbfb

  4. Upload fineweb2_hq/flexitok--bpe_arb_Arab_8000_info.json with huggingface_hub

    gsaltintas
    Verified signature
    89ca1c5

  5. Upload fineweb2_hq/flexitok--bpe_arb_Arab_8000_vocab.json with huggingface_hub

    gsaltintas
    Verified signature
    e2ebbec

  6. Upload fineweb2_hq/flexitok--bpe_arb_Arab_8000_super_mapping.json with huggingface_hub

    gsaltintas
    Verified signature
    265946f

  1. Upload folder using huggingface_hub

    gsaltintas
    Verified signature
    7508411

  2. Upload folder using huggingface_hub

    gsaltintas
    Verified signature
    87664a8

  3. initial commit

    gsaltintas
    Verified signature
    aecfd7e