---
license: cc-by-nc-4.0
base_model: nvidia/Cosmos3-Nano
base_model_relation: finetune
datasets:
  - BonsaiRobotics/bonsai-vineyard-sample
pipeline_tag: image-to-video
tags:
  - cosmos
  - lora
  - agriculture
  - vineyard
  - robotics
  - video-generation
---

# Bonsai Vineyard Sample LoRA (Cosmos3-Nano)

A rank-32 LoRA on [`nvidia/Cosmos3-Nano`](https://huggingface.co/nvidia/Cosmos3-Nano) that turns a seed image, a
segmentation video, a depth video, or segmentation + depth together into a 720p, 15 fps video of a farm robot driving
through vineyard rows. It is the checkpoint produced by the Cosmos3-Nano multi-control transfer LoRA recipe in
[NVIDIA/cosmos-framework](https://github.com/NVIDIA/cosmos-framework) (`docs/transfer_multicontrol_lora_nano.md`),
trained for 150 iterations on the 50 training clips of
[`BonsaiRobotics/bonsai-vineyard-sample`](https://huggingface.co/datasets/BonsaiRobotics/bonsai-vineyard-sample).

This repository holds the exported checkpoint (base weights plus the LoRA tensors), in the format
`cosmos_framework.scripts.inference --checkpoint-path` loads. One checkpoint serves all four modes.

## Use

From a cosmos-framework checkout (see the recipe for the pinned commit and setup):

```bash
uvx hf@latest download BonsaiRobotics/bonsai-vineyard-sample-lora --local-dir examples/checkpoints/bonsai-vineyard-sample-lora
python -m cosmos_framework.scripts.inference --parallelism-preset=latency --seed 2026 \
    -i <spec>.json -o outputs/bonsai-vineyard-sample-lora \
    --checkpoint-path examples/checkpoints/bonsai-vineyard-sample-lora
```

The recipe's `examples/transfer_multicontrol_lora_nano/make_specs.py` writes the specs used for its results (seed
image -> video: 149 frames, guidance 6; control modes: 61 frames, guidance 3, control guidance 0.9, 35 steps,
shift 10). The Wan2.2 VAE is fetched automatically on first use.

## Results

Held-out day of the dataset, 6 clips, one seed, against the base checkpoint with identical inputs (mean over clips):

| Mode | PSNR (dB), base -> LoRA | LPIPS, base -> LoRA |
|---|---|---|
| Seg + depth -> video | 11.82 -> **15.65** | 0.463 -> **0.323** |
| Seg -> video | 12.13 -> **15.30** | 0.466 -> **0.332** |
| Depth -> video | 12.31 -> **15.32** | 0.476 -> **0.334** |
| Seed image -> video | 19.27 -> 19.39 | 0.333 -> 0.318 |

The gain is mostly appearance: the LoRA renders the site's bright, dry daylight where the base model renders a
generic low-sun vineyard. Full tables, the before/after comparison and a check on orchard and strawberry clips are in
the recipe.

## Limitations

- Trained on a small sample dataset: 50 ten-second clips from one camera rig in one vineyard, daylight only. The
  results show what a small dataset can do, not the best this recipe can reach.
- Objects the segmentation marks but the training clips rarely show (buildings at the row end, people) are not drawn.
- Where the segmentation leaves part of the frame unassigned, such as a distant horizon, the LoRA fills it with the
  training site's background (dry hills, a fence).
- In the control modes, leaves near the camera change more from frame to frame than in real video.

## Licence

The fine-tuned checkpoint is released under the
[Creative Commons Attribution-NonCommercial 4.0 International licence (CC BY-NC 4.0)](https://creativecommons.org/licenses/by-nc/4.0/),
the same terms as the dataset it was trained on: non-commercial use, with credit to Bonsai Robotics.

It is derived from `nvidia/Cosmos3-Nano`, which NVIDIA releases under
[OpenMDW-1.1](https://openmdw.ai/license/1-1/); the base model's terms and notices continue to apply to the parts of
the weights that come from it.

## Citation

```bibtex
@misc{bonsai_vineyard_sample_lora_2026,
  title        = {Bonsai Vineyard Sample LoRA (Cosmos3-Nano)},
  author       = {Zachariah, Sandeep Sam and {Bonsai Robotics}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/BonsaiRobotics/bonsai-vineyard-sample-lora}}
}
```