Text Generation
Transformers
Safetensors
English
gabgpt
gift-of-gab
small-language-model
trained-from-scratch
byte-level
game-boy-advance
browser
conversational
custom_code
Instructions to use gszauer/ChatGBA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use gszauer/ChatGBA with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="gszauer/ChatGBA", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("gszauer/ChatGBA", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use gszauer/ChatGBA with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "gszauer/ChatGBA" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "gszauer/ChatGBA", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/gszauer/ChatGBA
- SGLang
How to use gszauer/ChatGBA with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "gszauer/ChatGBA" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "gszauer/ChatGBA", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "gszauer/ChatGBA" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "gszauer/ChatGBA", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use gszauer/ChatGBA with Docker Model Runner:
docker model run hf.co/gszauer/ChatGBA
Download source/engine.h from gszauer/ChatGBA: direct link, hf CLI and curl.
- Browser
- Download file 1.77 kB
-
https://huggingface.co/gszauer/ChatGBA/resolve/main/source/engine.h
- Command line
-
hf download hf://gszauer/ChatGBA/source/engine.h
-
curl -L -o engine.h https://huggingface.co/gszauer/ChatGBA/resolve/main/source/engine.h
1.77 kB
| // ChatGBA inference in integer arithmetic, for the Game Boy Advance (and the | |
| // Mac, where the same code measures what quantization costs). | |
| // | |
| // Numbers are fixed-point: an int32 "Q16" value v stands for v / 65536. | |
| // Weights are int8 with one scale per output; activations entering a matrix | |
| // are rounded to at most 13 bits with a shared power-of-two shift, so a | |
| // product of the two fits easily in an int32 sum. | |
| // Reads the blob that export.py wrote (it stays where it is; on the GBA, in | |
| // cartridge ROM). Returns 0, or a negative number if the blob does not match | |
| // model_config.h. | |
| int engine_init(const uint8_t* blob); | |
| // Forget the context: the next byte is position 0. | |
| void engine_reset(void); | |
| // Bytes processed so far: the position of the next one. | |
| int engine_position(void); | |
| // Process one byte at the next position and write the scores for the byte that | |
| // follows it into logits (Q16, MODEL_VOCAB of them). There is no last position: | |
| // the context slides, so each byte sees at most the MODEL_CONTEXT bytes up to | |
| // and including itself (fewer in a block with a shorter window). | |
| void engine_step(int byte, int32_t* logits); | |
| // Pick the next byte from logits. temperature is Q16; 0 picks the most likely | |
| // byte. Sampling keeps the top_k most likely bytes. rng is an xorshift32 state. | |
| int engine_sample(const int32_t* logits, int32_t temperature, int top_k, uint32_t* rng); | |
| // With PROFILE defined, engine_step adds the time it spends in each part to | |
| // profile[] (in whatever units profile_clock counts). | |
| enum { PROFILE_MATRICES, PROFILE_ATTENTION, PROFILE_NORMS, PROFILE_CANON, PROFILE_MLP, PROFILE_PARTS }; | |
| extern uint32_t profile[PROFILE_PARTS]; | |
| extern uint32_t (*profile_clock)(void); | |