Spaces:
Running
Running
Present the laboratory and its projects, Maverick as one of them
Browse files
README.md
CHANGED
|
@@ -8,32 +8,33 @@ pinned: false
|
|
| 8 |
---
|
| 9 |
|
| 10 |
The Extended Reality Laboratory (XRLab) at Manisa Celal Bayar University works on virtual, augmented and mixed
|
| 11 |
-
reality, 3D virtual environments and educational technology, and on the AI models
|
| 12 |
-
|
| 13 |
|
| 14 |
-
|
| 15 |
|
| 16 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
A 4B language model that turns spoken or typed English into actions in Unity XR scenes. It picks up, places and moves
|
| 19 |
objects, switches devices on and off, moves the user between rooms, and asks which object you mean when a command is
|
| 20 |
-
unclear. It runs offline through llama.cpp
|
| 21 |
-
|
| 22 |
-
On 499 instructions written by people for the ALFRED benchmark, it chose the right action on the right object 83.8% of
|
| 23 |
-
the time, ahead of models with up to 120B parameters. The [article](https://huggingface.co/blog/ErenAta00/maverick-4b-unity-xr-agent) explains how it was trained and tested.
|
| 24 |
|
| 25 |
-
|
|
|
|
| 26 |
|
| 27 |
-
|
|
|
|
|
|
|
|
|
|
| 28 |
|
| 29 |
-
|
| 30 |
-
|---|---|---|
|
| 31 |
-
| [Maverick-4B-Unity-XR-Agent 1.1](https://huggingface.co/ErenAta00/Maverick-4B-Unity-XR-Agent-1.1) | Model | Adds undo and corrections, shared attention, actions across rooms and short plans |
|
| 32 |
-
| [Maverick-4B-Unity-XR-Agent](https://huggingface.co/ErenAta00/Maverick-4B-Unity-XR-Agent) (1.0) | Model | Tool calling for Unity XR scenes, GGUF |
|
| 33 |
-
| [Maverick XR Agent for Unity](https://github.com/mcbu-xrlab/Maverick-Unity) | Unity package | Connects the model to your scene; installs from the Package Manager ([Hugging Face mirror](https://huggingface.co/ErenAta00/Maverick-Unity)) |
|
| 34 |
-
| [Offline Voice Control for Unity XR Scenes](https://huggingface.co/blog/ErenAta00/maverick-4b-unity-xr-agent) | Article | How Maverick was trained and tested |
|
| 35 |
-
| [XRLab-SDK](https://github.com/mcbu-xrlab/XRLab-SDK) | Python SDK | Shows which vision-language models fit on an XR headset and what they cost to run |
|
| 36 |
|
| 37 |
-
|
|
|
|
| 38 |
|
| 39 |
-
|
|
|
|
| 8 |
---
|
| 9 |
|
| 10 |
The Extended Reality Laboratory (XRLab) at Manisa Celal Bayar University works on virtual, augmented and mixed
|
| 11 |
+
reality, 3D virtual environments and educational technology, and on the AI models that run inside them. This page
|
| 12 |
+
collects what we release in the open.
|
| 13 |
|
| 14 |
+
[Website](https://xrlab.cbu.edu.tr) 路 [GitHub](https://github.com/mcbu-xrlab)
|
| 15 |
|
| 16 |
+
## Projects
|
| 17 |
+
|
| 18 |
+
### Maverick: language control for Unity XR scenes
|
| 19 |
+
|
| 20 |
+
<a href="https://huggingface.co/ErenAta00/Maverick-4B-Unity-XR-Agent-1.1"><img src="https://huggingface.co/ErenAta00/Maverick-4B-Unity-XR-Agent-1.1/resolve/main/assets/banner.jpg" alt="Maverick-4B-Unity-XR-Agent 1.1" width="55%"></a>
|
| 21 |
|
| 22 |
A 4B language model that turns spoken or typed English into actions in Unity XR scenes. It picks up, places and moves
|
| 23 |
objects, switches devices on and off, moves the user between rooms, and asks which object you mean when a command is
|
| 24 |
+
unclear. It runs offline through llama.cpp on a 4 GB laptop GPU. Version 1.1 adds undo and corrections, shared
|
| 25 |
+
attention for several people, actions across rooms and short plans.
|
|
|
|
|
|
|
| 26 |
|
| 27 |
+
On 499 instructions written by people for the ALFRED benchmark, 1.1 chose the right action on the right object 83.2% of
|
| 28 |
+
the time (1.0: 83.8%), ahead of the other models we tested, which scored up to 67.1%.
|
| 29 |
|
| 30 |
+
- [Maverick-4B-Unity-XR-Agent 1.1](https://huggingface.co/ErenAta00/Maverick-4B-Unity-XR-Agent-1.1), the current model
|
| 31 |
+
- [Maverick-4B-Unity-XR-Agent 1.0](https://huggingface.co/ErenAta00/Maverick-4B-Unity-XR-Agent), the first release
|
| 32 |
+
- [Maverick XR Agent for Unity](https://github.com/mcbu-xrlab/Maverick-Unity), the Unity package that connects the model to your scene ([Hugging Face mirror](https://huggingface.co/ErenAta00/Maverick-Unity))
|
| 33 |
+
- [Offline Voice Control for Unity XR Scenes](https://huggingface.co/blog/ErenAta00/maverick-4b-unity-xr-agent), the article on how 1.0 was trained and tested
|
| 34 |
|
| 35 |
+
### XRLab-SDK: vision-language models on XR headsets
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
|
| 37 |
+
A Python SDK that reports what a headset is, which vision-language model configurations fit on it, and what running
|
| 38 |
+
them costs. It is built from a study of about 10,000 inferences on a Meta Quest 3.
|
| 39 |
|
| 40 |
+
- [XRLab-SDK](https://github.com/mcbu-xrlab/XRLab-SDK) on GitHub
|