---
license: apache-2.0
base_model: meta-models/Muse-Glimmer-30B
library_name: peft
tags: [robotics, program-synthesis, lora, qlora, manipulation]
---

# Intent2Action · Glimmer program writer (G1b)

A QLoRA adapter for [Muse Glimmer 30B](https://huggingface.co/meta-models/Muse-Glimmer-30B). Given a task intent
(explicit rules such as "every cube into the red box") and the robot's object list, it writes a short, checkable
Python program against the robot API of [Intent2Action](https://github.com/allwinlucious/Intent2Action): `find`,
`find_all`, `pick`, `place`, `verify`, and so on. The program is checked, dry-run and executed by the robot stack.
The model never issues motions itself.

## Training

*   **Data:** 711 expert rows from the planning simulator ("plan tier": the real motion planner and robot
    service, no physics):
    *   for every training task, every pick-order strategy was run;
    *   the label is the most reliable strategy that succeeded;
    *   5 task families (cluster, packing, stacking, reach, sorting), 562 rows plus 149 sorting rows.
*   **Method:** supervised fine-tuning with the model cookbook's QLoRA recipe:
    *   r 32, alpha 16, dropout 0.05;
    *   attention and MLP projections of the language model;
    *   NF4 base.

## Results

Held-out tasks: 250 tasks (50 per family, including the never-trained `mixed` family), 3 runs per program. 95%
intervals are a bootstrap over tasks.

| | success |
|---|---|
| Glimmer, no adapter | 61% [56–67] |
| **this adapter** | **86% [82–89]** |
| best fixed strategy per family (reference) | 84% |
| oracle (any strategy) | 91% |

*   **Stacking:** 47% → 97%.
*   **The unseen family:** 51% → 82%.
*   **Isaac Sim physics** (50 held-out tasks): 41/50, against 34/50 without the adapter.

What it learned is which strategy works for which kind of task. It did not learn per-scene judgement: the oracle
is not reached.

Full results, ablations (rejection sampling, DPO, GRPO, bf16 LoRA, seeds) and limitations:
`docs/designs/learning_results.md` in the repository.

## Use

Load it on the NF4 base, as the cookbook evaluates adapters: transformers + PEFT. The repository's
`learn/chat_server.py` serves the base model and this adapter behind an OpenAI-compatible API:

```
python learn/chat_server.py --root <dir containing runs/G1b> --adapters G1b
```

## Limitations

*   Synthetic tabletop tasks only (cubes, boxes, plates).
*   Strategy-level decisions only: grasps and motion stay with the robot stack.
*   Physics results are on 50 tasks.

## License

Apache-2.0, as the base model. This adapter is a derivative of Muse Glimmer 30B, fine-tuned as described above.