# EmbeddingGemma-2 demo examples

Fourteen small media files with a matching description each, for the multimodal search demo in
[torq-examples](https://github.com/synaptics-torq/torq-examples/tree/main/EmbeddingGemma-2):
six subjects with a photo and a sound each (cat, dog, bird, piano, train, police car), plus two
short videos. Every file is scored against every description; each one ranks its own description
first, by a clear margin over the descriptions of the other subjects (checked with the 8-bit
build on the SL2619 NPU).

`examples.json` lists each file, its kind (image, audio or video) and its description. With the
model downloaded by `python setup_demos.py EmbeddingGemma-2`, run them with:

```sh
cd EmbeddingGemma-2
python src/infer.py --examples           # every example in one session
```

With the bf16 or 8-bit build, the NPU cannot map the vision and audio encoders together; run
`--examples visual` and `--examples audio` as two sessions instead.

## Files and credits

All files are CC0 or public domain. Sounds were cut to 5 seconds and converted to 16 kHz mono WAV;
photos were resized to at most 1024 pixels. The sounds come from [FSD50K](https://zenodo.org/records/4060432)
(Fonseca et al., 2022), which collects them from Freesound.

| File | Description | Original | Author | License |
|---|---|---|---|---|
| `cat_tabby.jpg` | a fluffy tabby and white cat with bright blue eyes | [Tabby cat with blue eyes-3336579.jpg](https://commons.wikimedia.org/wiki/File:Tabby_cat_with_blue_eyes-3336579.jpg) | AdinaVoicu | CC0 |
| `cat_meowing.wav` | a cat meowing loudly | [cat meowing](https://freesound.org/s/414042/) | nekoninja | CC0 1.0 (FSD50K) |
| `dog_labrador.jpg` | a happy yellow Labrador retriever with its tongue out | [Labrador Retriever (1210559).jpg](https://commons.wikimedia.org/wiki/File:Labrador_Retriever_(1210559).jpg) | chiemsee2016 (Pixabay) | CC0 |
| `dog_barking.wav` | a dog barking and growling | [barking dog 2](https://freesound.org/s/415023/) | roman_cgr | CC0 1.0 (FSD50K) |
| `bird_sparrow.jpg` | a small brown sparrow perched on a tree branch | [Bird on branch (Unsplash).jpg](https://commons.wikimedia.org/wiki/File:Bird_on_branch_(Unsplash).jpg) | Pritiranjan Maharana pritiranjan101 | CC0 |
| `birds_chirping.wav` | birds chirping and calling in the trees | [Bird Calls.mp3](https://freesound.org/s/415297/) | Yin_Yang_Jake007 | CC0 1.0 (FSD50K) |
| `piano_keyboard.jpg` | the wooden keyboard of an antique piano | [Grand Piano MET DP300910.jpg](https://commons.wikimedia.org/wiki/File:Grand_Piano_MET_DP300910.jpg) | The Metropolitan Museum of Art | CC0 |
| `piano_arpeggio.wav` | a soft, relaxing piano melody | [Arpegio A-menor 90 BPM.wav](https://freesound.org/s/176925/) | Spol | CC0 1.0 (FSD50K) |
| `train_locomotive.jpg` | a yellow and green locomotive pulling passenger cars in the mountains | [Passenger Train with WPYR Diesel locomotive no 100 at Carcross station, Yucon, Canada, 2011.jpg](https://commons.wikimedia.org/wiki/File:Passenger_Train_with_WPYR_Diesel_locomotive_no_100_at_Carcross_station,_Yucon,_Canada,_2011.jpg) | Tuskönils | CC0 |
| `train_passing.wav` | the rumble of a train passing by on the tracks | [TRAIN PASSING SLOWLY](https://freesound.org/s/393780/) | _CAZAPROD_ | CC0 1.0 (FSD50K) |
| `police_car.jpg` | a white police car with blue and yellow stripes and a light bar | [Police car Vinnytsia 2016 G1.jpg](https://commons.wikimedia.org/wiki/File:Police_car_Vinnytsia_2016_G1.jpg) | George Chernilevsky | Public domain |
| `police_siren.wav` | a police car siren wailing | [Police Car Siren in Traffic](https://freesound.org/s/268809/) | hyderpotter | CC0 1.0 (FSD50K) |
| `waterfall.ogv` | a tall waterfall pouring from a rocky cliff into a turquoise pool | [Video of the waterfall of Seythenex.ogv](https://commons.wikimedia.org/wiki/File:Video_of_the_waterfall_of_Seythenex.ogv) | Tangopaso | Public domain |
| `sea_turtle.ogv` | a sea turtle swimming underwater in a blue aquarium | [Sealife-Munich-Turtle.ogv](https://commons.wikimedia.org/wiki/File:Sealife-Munich-Turtle.ogv) | MartinThoma | CC0 |