1BTF

1-Byte Text Format

1BTF is a byte-oriented text representation layer.

Core

  • One 1BTF unit = exactly 1 byte.
  • Raw UTF-8 byte matching.
  • No grapheme segmentation.
  • No tokenizer.
  • No final token.
  • English letters are canonical.
  • English words are not stored as 1BTF units.
  • Duplicate English representations are not stored in other domains.
  • Global IDs belong to the master registry.
  • Runtime representation uses 1-byte local IDs.
  • Original UTF-8 unit information is stored in the registry.

Package

  • 1btf_master.bin โ€” single master registry
  • tables/ โ€” individual 1BTF sub-tables
  • manifest.json โ€” package metadata and checksums

Runtime

Raw UTF-8 โ†’ exact 1BTF unit โ†’ 1-byte representation

1BTF is not a tokenizer and is not a final token format.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support