Text Generation
Transformers
Safetensors
English
gabgpt
gift-of-gab
small-language-model
trained-from-scratch
byte-level
game-boy-advance
browser
conversational
custom_code
Instructions to use gszauer/ChatGBA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use gszauer/ChatGBA with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="gszauer/ChatGBA", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("gszauer/ChatGBA", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use gszauer/ChatGBA with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "gszauer/ChatGBA" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "gszauer/ChatGBA", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/gszauer/ChatGBA
- SGLang
How to use gszauer/ChatGBA with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "gszauer/ChatGBA" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "gszauer/ChatGBA", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "gszauer/ChatGBA" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "gszauer/ChatGBA", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use gszauer/ChatGBA with Docker Model Runner:
docker model run hf.co/gszauer/ChatGBA
Download source/chat.h from gszauer/ChatGBA: direct link, hf CLI and curl.
- Browser
- Download file 1.16 kB
-
https://huggingface.co/gszauer/ChatGBA/resolve/main/source/chat.h
- Command line
-
hf download hf://gszauer/ChatGBA/source/chat.h
-
curl -L -o chat.h https://huggingface.co/gszauer/ChatGBA/resolve/main/source/chat.h
1.16 kB
| // Chat on top of engine.c: the conversation as turn pairs, | |
| // <|user|>...<|end|><|assistant|>...<|end|> | |
| // fed to the engine as it happens, once. The engine's context slides, so a | |
| // long conversation simply forgets its oldest bytes, the way the model was | |
| // fine-tuned (on windows that start anywhere in a conversation). | |
| typedef struct { | |
| int32_t temperature; // Q16; 0 picks the most likely byte | |
| int top_k; | |
| int max_reply; // bytes | |
| uint32_t rng; // xorshift32 state, nonzero | |
| void (*on_byte)(int byte); // each reply byte, once it cannot be part of <|end|> | |
| void (*on_prompt)(int done, int total); // progress through the bytes processed before replying | |
| int (*stop)(void); // asked before each reply byte after the first; nonzero ends the reply | |
| } ChatSettings; | |
| void chat_reset(void); | |
| // Reply to message. Returns the reply's length (written to reply, without | |
| // <|end|>). A message longer than the context is fine; its start slides out. | |
| int chat_reply(const uint8_t* message, int length, ChatSettings* settings, uint8_t* reply, int capacity); | |