Nanobe

Nanobe is a local multimodal embedding model for searching mixed folders. It encodes text, images, document pages, audio, and video into a shared vector space.

Audio and video are encoded directly, without generating transcripts or captions during indexing. Pages and recording segments get their own vectors, so a relevant page or moment can retrieve the whole file.

This release includes quantized model weights and a Python reference implementation using MLX on Apple Silicon.

Read the release post.

Benchmark results

Nanobe Bench tests retrieval across mixed file types, including concepts, style, and relevant content deep inside documents and recordings.

Results on conventional retrieval benchmarks:

Usage

Use an Apple Silicon Mac with Python 3.12 and uv. Download the release and install the reference implementation:

uvx --from huggingface-hub hf download lokimorty/nanobe --local-dir nanobe-release
cd nanobe-release/source
uv sync --frozen
export NANOBE_MODEL_DIR="$(cd ../model && pwd)"

Python inference

Encode a text query and compare it with a local file. Run from the source directory and replace example.jpg with your file:

from pathlib import Path

import numpy as np

from nanobe.inference import encode_file
from nanobe.runtime import load, load_lanes, load_query

package = Path("../model")
model = load(package)
calibration = load_lanes(package)
query = load_query(package).encode_query("a flamenco performance")
kind, segments = encode_file(model, Path("example.jpg"))
vectors = np.stack([calibration.item(segment.vector, kind) for segment in segments])
scores = vectors @ query / (np.linalg.norm(vectors, axis=1) * np.linalg.norm(query))
print(float(scores.max()))

The same file encoder accepts supported images, documents, audio, and video. See Python inference for details and a search example that ranks multiple files.

Folder search

The reference implementation also includes a local folder index:

uv run --frozen nanobe index ~/Creative
uv run --frozen nanobe search ~/Creative "a presentation about sustainable packaging"
uv run --frozen nanobe search ~/Creative --file ~/Desktop/reference.jpg

See the source README for supported formats and setup details.

Model lineage

Nanobe builds on Jina AI's jina-embeddings-v5-omni-small-retrieval and Meta's Perception Encoder AV. See model lineage for the recorded revisions.

License

I release Nanobe under MIT because I believe reusable computational capabilities should be available for anyone to use, modify, and build on, including commercially.

See LICENSE for the full MIT License.