Text Generation
Transformers
Safetensors
English
gabgpt
gift-of-gab
small-language-model
trained-from-scratch
byte-level
game-boy-advance
browser
conversational
custom_code
Instructions to use gszauer/ChatGBA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use gszauer/ChatGBA with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="gszauer/ChatGBA", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("gszauer/ChatGBA", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use gszauer/ChatGBA with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "gszauer/ChatGBA" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "gszauer/ChatGBA", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/gszauer/ChatGBA
- SGLang
How to use gszauer/ChatGBA with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "gszauer/ChatGBA" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "gszauer/ChatGBA", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "gszauer/ChatGBA" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "gszauer/ChatGBA", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use gszauer/ChatGBA with Docker Model Runner:
docker model run hf.co/gszauer/ChatGBA
File size: 1,772 Bytes
546acd4 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | // ChatGBA inference in integer arithmetic, for the Game Boy Advance (and the
// Mac, where the same code measures what quantization costs).
//
// Numbers are fixed-point: an int32 "Q16" value v stands for v / 65536.
// Weights are int8 with one scale per output; activations entering a matrix
// are rounded to at most 13 bits with a shared power-of-two shift, so a
// product of the two fits easily in an int32 sum.
#pragma once
#include <stdint.h>
#include "model_config.h"
// Reads the blob that export.py wrote (it stays where it is; on the GBA, in
// cartridge ROM). Returns 0, or a negative number if the blob does not match
// model_config.h.
int engine_init(const uint8_t* blob);
// Forget the context: the next byte is position 0.
void engine_reset(void);
// Bytes processed so far: the position of the next one.
int engine_position(void);
// Process one byte at the next position and write the scores for the byte that
// follows it into logits (Q16, MODEL_VOCAB of them). There is no last position:
// the context slides, so each byte sees at most the MODEL_CONTEXT bytes up to
// and including itself (fewer in a block with a shorter window).
void engine_step(int byte, int32_t* logits);
// Pick the next byte from logits. temperature is Q16; 0 picks the most likely
// byte. Sampling keeps the top_k most likely bytes. rng is an xorshift32 state.
int engine_sample(const int32_t* logits, int32_t temperature, int top_k, uint32_t* rng);
// With PROFILE defined, engine_step adds the time it spends in each part to
// profile[] (in whatever units profile_clock counts).
enum { PROFILE_MATRICES, PROFILE_ATTENTION, PROFILE_NORMS, PROFILE_CANON, PROFILE_MLP, PROFILE_PARTS };
extern uint32_t profile[PROFILE_PARTS];
extern uint32_t (*profile_clock)(void);
|