Oasis

Oasis is Cactus Compute's Mirage with a new CPU engine and a retrained text reader. Mirage turns a photo into a JSON record of the scene, the objects in it with boxes and masks, the text it can read, how the objects relate and a short reasoning trace; given a form, it fills in choices, counts, text and boxes. Oasis stores the model in 18.8 MB against Mirage's 21.6 MB, runs it faster on x86, reads more of the text, and always returns valid JSON.

Oasis and Mirage

Both models described the same photos with the same options, Mirage on its own runtime. comparison.json holds every figure with the hashes of both runtimes and weight files.

	Mirage	Oasis
Weight file	21.6 MB	18.8 MB
Describe a photo, x86	912 ms	251 ms
Describe a photo, 64-bit Arm	813 ms	1012 ms
Read the text in a photo, x86	848 ms	68 ms
Read the text in a photo, 64-bit Arm	683 ms	276 ms
Fill a four-field form, x86	650 ms	109 ms
Fill a four-field form, 64-bit Arm	334 ms	488 ms
Records that are not valid JSON, of 300 COCO photos	58	0
Empty records at conf 0.5, of 300 COCO photos	2	0
ICDAR2013 words read	80.9%	81.7%
IIIT5K words read	77.6%	78.9%
Accented letters read	0.7%	29.7%
COCO objects found, F1	0.273	0.302
COCO mask IoU	0.670	0.681
COCO outline IoU	0.673	0.677

Timings are medians over 24 photos; on Arm the two engines took turns, photo by photo, on an otherwise idle board. Words are read end to end: each word image is placed in a scene, found by the engine's own text detector and read. Accented letters are counted on 500 synthetic lines in the seven description languages, set in fonts the reader never trained on. Objects are scored against COCO val2017 boxes at IoU 0.5, and masks and filled outlines against COCO's instance outlines.

What changed

The engine is new. It multiplies Mirage's 2-bit weights in place with integer kernels for x86 and 64-bit Arm, keeps its worker threads awake while the decoder writes, and skips the decoder when only text is requested; on Arm, whose caches are small, it holds the image memory the decoder reads at every token in 16-bit floats. On Arm its COCO F1 is 0.299 and it reads 81.8% of the ICDAR2013 words. Every record it writes parses. On about one photo in five, Mirage's runtime adds a phantom box in the top-right corner, as an object or as a line of text, with a nan score that breaks the JSON; the same phantom box becomes the answer to box questions in forms. When the decoder's reasoning trace falls into a loop, Mirage's runtime spends the whole token budget on it and returns an empty record; Oasis closes the trace and the description is written. Outlines follow the probability field between mask cells, keywords steer the description toward the words an application cares about, and a language the model was not trained to describe in is refused with an error.

The text reader is retrained. Mirage's reader never learned 67 of its 306 characters, and on accented text cut as tightly as its detector cuts it, it writes almost no accents; Oasis's reader learned from TextOCR word crops and from synthetic lines in the seven description languages, cut as tightly as the detector cuts text, through the same 2-bit quantiser the engine runs, with Mirage's reader as its teacher on real words. The rest of the model is Mirage's.

The weight file is smaller. Mirage stores the encoder's HConvGLU bands, a quarter of its file, as 16-bit floats; Oasis stores them as 12-bit integers with one scale per output channel, keeps the group scales of the 2-bit matrices outside the text reader in 8 bits, and deflates every tensor that compresses, which the engine inflates as it loads.

Use

git clone https://huggingface.co/phanerozoic/oasis && cd oasis
python oasis.py build
python oasis.py photo.jpg '{"language": "de"}'

from oasis import Oasis

model = Oasis("oasis.oasis")
record = model.describe("photo.jpg", sections=["objects", "text"], mask_format="polygons")
form = model.extract("photo.jpg", {
    "sign": {"enum": ["stop", "yield", "speed limit"]},
    "cars": {"type": "integer", "minimum": 0, "maximum": 10},
    "where": {"x-source": "bbox", "x-of": "sign"},
})

Options and form fields follow Mirage's: language (en, de, fr, es, it, nl, pl), keywords, conf, text_conf, max_objects, max_text_lines, max_tokens, sections, fields, masks, mask_format (largest, polygons, merged or bitmap), mask_threshold, nms and reasoning. Boxes and outlines use a 0–1000 scale on each image axis. The engine is one C++17 file that builds with GCC, Clang and MSVC; python oasis.py build compiles it with the compiler it finds, or with CXX, and prints the command. It uses every processor, less one on machines with four or fewer; OMP_NUM_THREADS sets the count. oasis.py reads image files with Pillow.

Files

Path	Contents
oasis.oasis	Engine weights
oasis.safetensors	Float32 weights for further training
oasis.cpp	The engine and its C API
oasis.py	Python binding, command line, build, and the benchmarks against Mirage
reference.py	PyTorch forward pass, quantiser, reader training and export
comparison.json	The measurements above
config.json	Architecture, quantisation and file hashes

python oasis.py --help and python reference.py list their commands. reference.py reads Mirage's mirage.safetensors and mirage.cact from mirage/ beside it (hf download Cactus-Compute/mirage mirage.safetensors mirage.cact --local-dir mirage).

Oasis is released under Apache-2.0, as Mirage is, and contains Mirage's weights and configuration by Cactus Compute: oasis.safetensors is Mirage's float32 checkpoint with the text reader retrained and the training-only multi-token-prediction and name heads removed, and oasis.oasis keeps the codes of every quantised matrix of Mirage's mirage.cact outside the reader. The reader was retrained on word crops from TextOCR (Singh et al., 2021, CC BY 4.0) and on synthetic lines. The image resize in oasis.cpp follows Pillow's bilinear filter and carries Pillow's notice.