| --- |
| license: apache-2.0 |
| library_name: pytorch |
| pipeline_tag: robotics |
| tags: |
| - robotics |
| - world-model |
| - cable-driven-parallel-robot |
| - stringman |
| - model-predictive-control |
| extra_gated_heading: Request access to the Stringman world model |
| extra_gated_prompt: Access is granted by the author on request. Tell me who you are and what you'd like to try. |
| extra_gated_fields: |
| Name: text |
| Affiliation: text |
| What would you like to test?: text |
| extra_gated_button_content: Request access |
| --- |
| |
| # Stringman world model |
|
|
| A small learned **dynamics model** of Neufangled's Stringman cable robot: give it the last second of states and commands, and the room's four mount points, and it predicts how the robot moves under any sequence of future commands. It knows nothing about tasks. In my work a sampling planner uses it to imagine a few hundred futures every control step, and each chore is written as costs on those futures. |
|
|
| - **28,401 parameters, 127 KB.** Runs on a CPU: 128 two-second futures in about 15 ms on a laptop. |
| - **One model for every room.** The room enters only through its calibrated mount points, so a room it never saw is just another set of four points. |
| - Trained without a GPU, on task-free simulated motion in 12,000 random rooms, with the robot itself varied from one trajectory to the next. |
|
|
| By Edgar Moreau · [huggingface.co/Ethgar](https://huggingface.co/Ethgar) |
|
|
| ## Try it |
|
|
| ```bash |
| pip install torch numpy |
| python example.py |
| ``` |
|
|
| ```python |
| import numpy as np |
| from stringman_wm import LearnedCDPRModel |
| |
| model = LearnedCDPRModel.load("stringman_wm_g1.pt", dt=0.2) # your control period, a multiple of 0.1 s |
| model.set_room(mounts) # (4, 3) metres, the firmware's line order |
| model.reset_history(x_now, a_now) # then model.observe(x, a) after every 0.1 s |
| future = model.rollout(None, None, commands) # (S, T, 5) commands -> (S, T, 14) states |
| ``` |
|
|
| ## Inputs and outputs |
|
|
| | | | units | |
| |---|---|---| |
| | **state** (14) | gantry position p (3), where the four lines meet, room frame | m | |
| | | gantry velocity v (3) | m/s | |
| | | pole swing s (2): x, y of the pole axis, = sin(tilt) per axis | – | |
| | | swing rate (2) | 1/s | |
| | | wrist angle, 0 = neutral (firmware degrees 0..1080 = 540 + degrees) | rad | |
| | | wrist rate | rad/s | |
| | | finger closure, 0 open .. 1 closed (firmware -90..90 = -90 + 180 × closure) | – | |
| | | closure rate | 1/s | |
| | **command** (5) | gantry velocity (3), room frame, as `move_direction_speed` takes it | m/s | |
| | | wrist rate | rad/s | |
| | | closure target | 0..1 | |
| | **room** | mount points (4, 3): where each free line span leaves from | m | |
|
|
| Model step 0.1 s, history 1 s (10 steps). Line lengths and tensions are not inputs. |
|
|
| ## How good it is (simulation) |
|
|
| Open-loop prediction, 2 s ahead, on 500 held-out windows in 310 random rooms it never saw: |
|
|
| | | grasp point error, median / p90 | with a payload on or changing | pole tilt error, median | |
| |---|---|---|---| |
| | this model | 19.6 / 60 mm | 23.2 mm | 0.89° | |
| | MuJoCo itself, nominal physics (reference) | 21.0 / 56 mm | 24.3 mm | 0.90° | |
|
|
| In two real rooms imported from public Stringman recordings (neither in training): 20.5 mm and 20.2 mm median, tilt 0.88° and 0.80°. The grasp point is between the finger pads, 0.515 m down the pole. |
|
|
| Planning with it in closed loop (sim): clearing 12 cubes into a bin in those two real rooms, 33/36 and 34/36 over 3 episodes each; in three furnished 3D rooms, 22, 20 and 24 of 24 over 2 episodes each, with no furniture contact. |
|
|
| ## Limits |
|
|
| - **Simulation only.** Trained and tested on a MuJoCo twin of Stringman. The twin's masses, line stretch, spool acceleration and swing damping are estimates, not measurements of a real robot. It has not run on real hardware yet. |
| - **Stringman's own two-finger gripper.** Another gripper changes the swing and needs a new model. |
| - **No contact.** Furniture, the floor and the objects are outside the model: that is the planner's and the task's business. |
| - Payloads up to 0.5 kg. The gantry's heading is assumed held by the lines. |
|
|
| ## License |
|
|
| Apache-2.0. |
|
|
