ErenAta00 commited on
Commit
06ba942
路
verified 路
1 Parent(s): aa75554

Present the laboratory and its projects, Maverick as one of them

Browse files
Files changed (1) hide show
  1. README.md +20 -19
README.md CHANGED
@@ -8,32 +8,33 @@ pinned: false
8
  ---
9
 
10
  The Extended Reality Laboratory (XRLab) at Manisa Celal Bayar University works on virtual, augmented and mixed
11
- reality, 3D virtual environments and educational technology, and on the AI models behind them. This page lists the
12
- models and tools we release.
13
 
14
- ## Maverick-4B-Unity-XR-Agent
15
 
16
- <a href="https://huggingface.co/ErenAta00/Maverick-4B-Unity-XR-Agent"><img src="https://huggingface.co/spaces/ExtendedRealityLabMCBU/README/resolve/main/maverick.jpg" alt="Maverick-4B-Unity-XR-Agent" width="100%"></a>
 
 
 
 
17
 
18
  A 4B language model that turns spoken or typed English into actions in Unity XR scenes. It picks up, places and moves
19
  objects, switches devices on and off, moves the user between rooms, and asks which object you mean when a command is
20
- unclear. It runs offline through llama.cpp and needs about 3 GB of GPU memory.
21
-
22
- On 499 instructions written by people for the ALFRED benchmark, it chose the right action on the right object 83.8% of
23
- the time, ahead of models with up to 120B parameters. The [article](https://huggingface.co/blog/ErenAta00/maverick-4b-unity-xr-agent) explains how it was trained and tested.
24
 
25
- Version 1.1 adds undo and corrections, shared attention for several people, actions across rooms and short plans: [Maverick-4B-Unity-XR-Agent 1.1](https://huggingface.co/ErenAta00/Maverick-4B-Unity-XR-Agent-1.1).
 
26
 
27
- ## Releases
 
 
 
28
 
29
- | Release | Type | Description |
30
- |---|---|---|
31
- | [Maverick-4B-Unity-XR-Agent 1.1](https://huggingface.co/ErenAta00/Maverick-4B-Unity-XR-Agent-1.1) | Model | Adds undo and corrections, shared attention, actions across rooms and short plans |
32
- | [Maverick-4B-Unity-XR-Agent](https://huggingface.co/ErenAta00/Maverick-4B-Unity-XR-Agent) (1.0) | Model | Tool calling for Unity XR scenes, GGUF |
33
- | [Maverick XR Agent for Unity](https://github.com/mcbu-xrlab/Maverick-Unity) | Unity package | Connects the model to your scene; installs from the Package Manager ([Hugging Face mirror](https://huggingface.co/ErenAta00/Maverick-Unity)) |
34
- | [Offline Voice Control for Unity XR Scenes](https://huggingface.co/blog/ErenAta00/maverick-4b-unity-xr-agent) | Article | How Maverick was trained and tested |
35
- | [XRLab-SDK](https://github.com/mcbu-xrlab/XRLab-SDK) | Python SDK | Shows which vision-language models fit on an XR headset and what they cost to run |
36
 
37
- ## Links
 
38
 
39
- [Website](https://xrlab.mcbu.edu.tr) 路 [GitHub](https://github.com/mcbu-xrlab)
 
8
  ---
9
 
10
  The Extended Reality Laboratory (XRLab) at Manisa Celal Bayar University works on virtual, augmented and mixed
11
+ reality, 3D virtual environments and educational technology, and on the AI models that run inside them. This page
12
+ collects what we release in the open.
13
 
14
+ [Website](https://xrlab.cbu.edu.tr) 路 [GitHub](https://github.com/mcbu-xrlab)
15
 
16
+ ## Projects
17
+
18
+ ### Maverick: language control for Unity XR scenes
19
+
20
+ <a href="https://huggingface.co/ErenAta00/Maverick-4B-Unity-XR-Agent-1.1"><img src="https://huggingface.co/ErenAta00/Maverick-4B-Unity-XR-Agent-1.1/resolve/main/assets/banner.jpg" alt="Maverick-4B-Unity-XR-Agent 1.1" width="55%"></a>
21
 
22
  A 4B language model that turns spoken or typed English into actions in Unity XR scenes. It picks up, places and moves
23
  objects, switches devices on and off, moves the user between rooms, and asks which object you mean when a command is
24
+ unclear. It runs offline through llama.cpp on a 4 GB laptop GPU. Version 1.1 adds undo and corrections, shared
25
+ attention for several people, actions across rooms and short plans.
 
 
26
 
27
+ On 499 instructions written by people for the ALFRED benchmark, 1.1 chose the right action on the right object 83.2% of
28
+ the time (1.0: 83.8%), ahead of the other models we tested, which scored up to 67.1%.
29
 
30
+ - [Maverick-4B-Unity-XR-Agent 1.1](https://huggingface.co/ErenAta00/Maverick-4B-Unity-XR-Agent-1.1), the current model
31
+ - [Maverick-4B-Unity-XR-Agent 1.0](https://huggingface.co/ErenAta00/Maverick-4B-Unity-XR-Agent), the first release
32
+ - [Maverick XR Agent for Unity](https://github.com/mcbu-xrlab/Maverick-Unity), the Unity package that connects the model to your scene ([Hugging Face mirror](https://huggingface.co/ErenAta00/Maverick-Unity))
33
+ - [Offline Voice Control for Unity XR Scenes](https://huggingface.co/blog/ErenAta00/maverick-4b-unity-xr-agent), the article on how 1.0 was trained and tested
34
 
35
+ ### XRLab-SDK: vision-language models on XR headsets
 
 
 
 
 
 
36
 
37
+ A Python SDK that reports what a headset is, which vision-language model configurations fit on it, and what running
38
+ them costs. It is built from a study of about 10,000 inferences on a Meta Quest 3.
39
 
40
+ - [XRLab-SDK](https://github.com/mcbu-xrlab/XRLab-SDK) on GitHub