File size: 1,089 Bytes
287ca83 a06f072 287ca83 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | # glance-runtime
The runtime for Glance decision models: the prompt layout, two-way attention, answer-code readout
and calibration that turn the weights in this repository into a decision model.
```bash
pip install -U huggingface_hub
hf download bpHigh/glance-0.1 --include "runtime/*" --local-dir glance-0.1
pip install "./glance-0.1/runtime[gpu]" # on a GPU
```
On a CPU-only machine, install the CPU build of PyTorch first (the default Linux build pulls
several GB of CUDA libraries):
```bash
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install ./glance-0.1/runtime
```
```python
from glance_runtime import Glance
glance = Glance.load("bpHigh/glance-0.1", device="cuda") # or "cpu"
glance(state="...", questions={...}) # -> answers
glance.decide({"state": ..., "questions": {...}}) # the full /v1/systemone response
```
```bash
python -m glance_runtime serve --model bpHigh/glance-0.1 --device cuda --port 8741
curl -s localhost:8741/v1/systemone -d @request.json
```
See the model card for the request format.
|