minicpm-robot-manip / README.md
yujiepan's picture
Upload folder using huggingface_hub
00a86bb verified
|
Raw History Blame Contribute Delete
2.67 kB
metadata
library_name: transformers
pipeline_tag: robotics
base_model:
  - openbmb/MiniCPM-RobotManip

This is a randomly initialized tiny model based on openbmb/MiniCPM-RobotManip for testing and debugging.

Feature Source This repo
Checkpoint tensor elements 1.30B BF16, 235.81M F32 8.29M BF16, 459.10K F32
Safetensors weight files 3.54 GB (model.safetensors) 18.45 MB (model.safetensors)
VLM text layers / width / head dimension 24 / 1024 / 256 8 / 32 / 32
VLM attention cadence 3 linear + 1 full, repeated Two 3-linear + 1-full cycles
Vision layers / width 27 / 1152 2 / 32
Action diffusion blocks / width 16 / 768 2 / 32
Action horizon / action-state dimensions 30 / 80-80 Same
Embodiment categories 32 Same
Text vocabulary rows 248094 Same
Stored VLM / action-head precision BF16 / FP32 Same
Input / output Image, instruction, state → action chunk Same
File path Size
model.safetensors 18.4MB

Example usage

Transformers (✅ verified)

def run_transformers(model_id: str):
    import torch
    from PIL import Image
    from transformers import AutoModel, AutoProcessor

    model = AutoModel.from_pretrained(model_id, trust_remote_code=True, dtype=torch.bfloat16).eval().cuda()
    processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
    image = Image.new("RGB", (112, 112), "blue")
    inputs = processor.apply_chat_template(
        [
            {
                "role": "user",
                "content": [{"type": "image", "image": image}, {"type": "text", "text": "Pick up the blue object."}],
            }
        ],
        tokenize=True,
        add_generation_prompt=True,
        return_dict=True,
        return_tensors="pt",
        processor_kwargs={"padding": False},
    ).to(model.device)
    action = model.predict_action(
        state=torch.zeros(1, 1, 80, device=model.device),
        embodiment_id=torch.zeros(1, dtype=torch.long, device=model.device),
        **inputs,
    )
    assert action.shape == (1, 30, 80)
    assert torch.isfinite(action).all()
    print(action.shape)


if __name__ == "__main__":
    run_transformers(model_id="tiny-random/minicpm-robot-manip")

Creating this repo

Change log

  • 2026-10-27: Created a tiny MiniCPM RobotManip vision-language-action policy.

This repo is generated by GPT-6.1-Sol.