| --- |
| license: apache-2.0 |
| datasets: |
| - CEAMFA/rewrite |
| --- |
| |
| <style> |
| img { |
| display: block; |
| position: static; |
| width: min(76%, 480px); |
| height: auto; |
| max-width: 100%; |
| margin: 3rem auto 2.5rem; |
| object-fit: contain; |
|
|
| border: 0px solid rgba(255, 255, 255, 0.16); |
| border-radius: 1rem; |
| outline: none; |
|
|
| user-select: none; |
| -webkit-user-select: none; |
| -moz-user-select: none; |
| -webkit-user-drag: none; |
|
|
| filter: none !important; |
| transform: scale(1) !important; |
| animation: none !important; |
| box-shadow: none !important; |
|
|
| position: relative; |
| z-index: 1; |
| transform-origin: center; |
| } |
| </style> |
|
|
| <img src="https://huggingface.co/appvoid/bytebox.1/resolve/main/bytebox.png"/> |
|
|
| # details |
|
|
| bytebox is a model designed to change public perspective on byte-level language models. It's a deep model pretrained on roughly 1B tokens that demonstrates instruction-following skills with only 60m parameters. Training and inference codes are included. |
|
|
| To the author's knowledge, this is the first-ever decoder-only instruction pretrained byte-level transformer openly available to date. |
|
|
|
|
| ```bash |
| python infer_bytefast60m.py --architecture-script bytefalcon_fast60m.py \ |
| --model /bytefast-60m/ \ |
| --tokenizer /byte-tokenizer/ \ |
| --instruction "Fix the sentence" \ |
| --text "this sentence need fixing you know i love it" \ |
| --max-new-tokens 128 \ |
| --temperature 0 |
| ``` |
|
|
| *** |
| |
| **Prompt** |
|
|
| What is Python? |
|
|
| **Output** |
|
|
| Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability, and its syntax allows programmers to express concepts in fewer lines of code than would be possible in languages such as C++ or Java. The language provides confidence towards the use of programming languages and their debugging problems."<eos> |
