| --- |
| license: apache-2.0 |
| library_name: streamembed |
| base_model: |
| - Qwen/Qwen3-VL-Embedding-2B |
| - Qwen/Qwen3-VL-Embedding-8B |
| - google/siglip2-so400m-patch16-naflex |
| - google/siglip2-base-patch16-naflex |
| - vultr/VultronRetrieverFlash-Qwen3.5-0.8B |
| tags: |
| - screenshots |
| - retrieval |
| - lora |
| - qwen3-vl |
| - siglip2 |
| --- |
| |
| # StreamEmbed models |
|
|
| LoRA adapters for [StreamEmbed](https://pypi.org/project/streamembed/), which searches your own screen history by describing what you saw. Only the adapters are stored here; each base model comes from its own repository, pinned to the commit the adapters were trained against. |
|
|
| | Model | Base model (commit) | Size | StreamEmbed support | |
| |---|---|---|---| |
| | `qwen3-vl-2b` (default) | [Qwen/Qwen3-VL-Embedding-2B](https://huggingface.co/Qwen/Qwen3-VL-Embedding-2B) (`9f2f7e7`) | 2B | yes | |
| | `qwen3-vl-8b` | [Qwen/Qwen3-VL-Embedding-8B](https://huggingface.co/Qwen/Qwen3-VL-Embedding-8B) (`2c45655`) | 8B | yes | |
| | `siglip2-so400m` | [google/siglip2-so400m-patch16-naflex](https://huggingface.co/google/siglip2-so400m-patch16-naflex) (`cc24074`) | 1.1B | weights only | |
| | `vultron-0.8b` | [vultr/VultronRetrieverFlash-Qwen3.5-0.8B](https://huggingface.co/vultr/VultronRetrieverFlash-Qwen3.5-0.8B) (`6f6b675`) | 0.8B | weights only | |
| | `siglip2-base` | [google/siglip2-base-patch16-naflex](https://huggingface.co/google/siglip2-base-patch16-naflex) (`b53b807`) | 0.4B | weights only | |
|
|
| StreamEmbed's stream encoder runs Qwen3-VL models. The SigLIP2 and Vultron adapters are published for use with your own code; StreamEmbed lists them but does not run them yet. The SigLIP2 adapters were trained with NaFlex inputs of at most 1024 patches (`max_num_patches` in `models.json`). |
|
|
| Each model folder has two LoRAs (rank 32, alpha 32) on the attention layers of the image and text (or language model) encoders: |
|
|
| - `query_lora.safetensors`: the retrieval LoRA, trained to find screenshots from a description. It encodes questions, and full screenshots when nothing is skipped. |
| - `frame_adapter.safetensors`: encodes screenshots when repeated blocks are skipped and unchanged ones are reused across frames, so that these vectors stay close to full-screenshot vectors. |
|
|
| `models.json` lists every model with its base repository, commit and files. |
|
|
| ## Use |
|
|
| ```python |
| import streamembed as se |
| |
| se.configure(model="qwen3-vl-2b") |
| se.build_index_from_directory("screenshots") |
| results = se.query_index("the flight search page with fares to Denver") |
| se.visualize_results(results) |
| ``` |
|
|
| To apply an adapter yourself, load it into a PEFT model of its base: |
|
|
| ```python |
| from peft import LoraConfig, get_peft_model, set_peft_model_state_dict |
| from safetensors.torch import load_file |
| |
| state = load_file("qwen3-vl-8b/query_lora.safetensors") |
| targets = sorted({k.split("base_model.model.")[1].rsplit(".lora_", 1)[0] for k in state}) |
| model = get_peft_model(base_model, LoraConfig(r=32, lora_alpha=32, target_modules=targets)) |
| set_peft_model_state_dict(model, state) |
| ``` |
|
|
| The files are PEFT LoRA state dicts (keys `base_model.model.<module>.lora_A.weight` and `lora_B.weight`) in float32. |
|
|
