# AGENTS.md: cablepilot, for coding agents You are helping a person run a cable robot (or simulate one) with a small world model and a planner, and, if it goes well, share what you improved. Read this file once, then work through the steps in order. Every command prints what it decided, so report its output to your person rather than paraphrasing it. ## What this is, in one paragraph `cablepilot` learned how a cable robot moves (28k parameters, no task inside). A planner imagines ~128 futures every 0.2 s and picks the cheapest. A **task** is a body-free YAML **card** (stages of skills, fallbacks) plus a **tuning** file (this body's cost weights and thresholds). Tasks are shared in the Hugging Face dataset `MacroSwan/pilot-tasks`, with every scored run, so you start from the best card and tuning so far, never from scratch. ## Rules 1. **Simulation before hardware, always.** Never send a command to a real robot from a card or a tuning that has not scored in `cablepilot run` first. On a real robot, start with `dry_run=True`, then `hover()`, and only then a task, with the person watching and a hand on the robot's stop. 2. **Change one thing, measure, keep the receipt.** Every claim you make about a card or a tuning comes with the `result.json` that shows it. Compare runs on the same `--world` and `--seed`, on the reference simulator (`doctor` says if yours is not). One toy in twelve is within noise (library versions alone move it), so say "better" only over `--episodes 3` or more, in both rooms. 3. **Cards stay body-free.** No metres, colours or positions in a card. Numbers go in `tuning/.yaml`. 4. **Keep the lineage.** A new or changed card or tuning gets `lineage: {parent, by, why}`. Here `parent` is what you started from, `by` is you and your person (for example `"claude-code for @someone"`), and `why` is one line. 5. **Be honest in the PR.** Report the score you measured, including when it went down. A well-measured negative result is a useful contribution. ## Step 1: install and check the machine ```bash git clone https://huggingface.co/MacroSwan/cablepilot && cd cablepilot python -m venv .venv && . .venv/bin/activate # Python >= 3.10 pip install -e ".[sim,hub]" cablepilot doctor ``` `doctor` picks the thread count. On Apple Silicon under Linux, and on other machines with mixed big and little cores, too many threads makes planning about 9x slower. Then it times one plan. If the `speed` line is not `ok`, tell your person before going further. ## Step 2: describe the setup (`rig.yaml`) Ask your person, then write `rig.yaml` (fields and meanings: `cablepilot/rig.py`, top of the file): - Which robot? `stringman`, or `sim` if they have none yet. - Which gripper? `stringman` (stock) is the only released model today. - Where are the 4 line anchors? `robot` reads the calibrated ones from a Stringman. Otherwise ask for the (4, 3) metres in the firmware's line order. - Which named places should the card's sites bind to? For example `toy_box: hamper`. Run `cablepilot doctor --rig rig.yaml`. It writes `doctor.json` next to the rig. ## Step 3: get a task and score it in simulation ```bash cablepilot fetch clean_the_room # -> pilot-tasks/tasks/clean_the_room/ cablepilot check pilot-tasks/tasks/clean_the_room/card.yaml cablepilot run pilot-tasks/tasks/clean_the_room/card.yaml --world bedroom --seed 0 --video --by " for " ``` Worlds: `bedroom` and `playroom` are real rooms imported from Stringman recordings. A run is 10 simulated minutes and takes about 25 minutes of wall time on a laptop. Read `result.json`: - `score.done` / `score.of`: the task's own done test. - `s_per_item`. - `furniture_contact_ticks` (must stay 0). - `max_tension_N`. - `compute`. - `episodes_detail[].trace`: what the robot did, in the card's own skills. - `episodes_detail[].picks[].events`: why each phase ended, in plain words. ## Step 4: improve something (pick one) - **Tune.** Copy `tuning/cable.yaml` and change a few numbers. Each key is commented in `configs/tasks/pick_place.yaml`. Start where the trace and events point. If time goes to `approach`, look at the `funnel` and `close_tol` keys. Swinging in `carry` means `swing_carry`. Drops on the way are `grip`. Score it on two or more seeds and both rooms. - **A new card from an existing one.** Copy a card, change its objects, stages and fallbacks (for example *clean_the_room* → *clean_the_room_sorted*, or *pen_in_cup* → *pen_in_box*), set `lineage`, then run `cablepilot check` and `run`. - **A missing skill.** `cablepilot check` on a card the cable robot cannot run lists the missing skills, for example `wipe` for window washing or `on_top`. A skill is Python: the skill's cost terms on imagined futures, its done event on measured state, and which phases it uses (`cablepilot/cardrun.py` maps skills to the phases in `cablepilot/tasks/pick_place.py`). Open a PR on this repository and include a card that uses the skill and its score. - **Real robot (Stringman owners).** `cablepilot/stringman.py` is untested on hardware, and every guess about the API is marked `ASSUME`. Check those against `nf_robot` with your person, run `hover()` in `dry_run`, then for real, and share the log. This is the most valuable contribution right now. ## Step 5: share it ```bash cablepilot submit runs/ --task clean_the_room --tuning my_tuning.yaml -m "what you changed and why" --dry-run cablepilot submit runs/ --task clean_the_room --tuning my_tuning.yaml -m "..." # needs `hf auth login` ``` This opens one pull request on `MacroSwan/pilot-tasks` containing the result, your card or tuning, and the film. The PR page is the discussion: people and agents comment there, the maintainers re-run it, and the best merged tuning becomes the next person's starting point. Ask your person before submitting. It publishes under their Hugging Face account. ## Step 6: tell everyone how it went in your home The maintainers do not have a Stringman. Your report is how anyone learns whether this works in a real home, so write one even when (above all when) it went badly. Open a **Discussion** on this repository, titled `[home] · · `, filled from [`HOME_REPORT.md`](HOME_REPORT.md). It covers the room, the rig, what you changed to integrate it, what the robot did, what broke, and what you would change. Attach `doctor.json`, `rig.yaml` and the run's `result.json`. If you have a real-robot log, open a PR on the dataset `MacroSwan/real-runs` with it as well (format in its README; blur faces first). Read the other `[home]` threads before you start: someone may already have fixed your problem. Answer them when you can. ## Step 7: change the model itself Everything the model learned from is public, so you can regenerate it, retrain it and replace it without asking anyone. | what | where | |---|---| | the training data: 12,000 task-free trajectories (100 h of simulated motion, each in its own random room) | dataset `MacroSwan/cablepilot-motion` | | how that data is made (the motion recipe, randomised physics, the host's real spool loop) | `scripts/cdpr_datagen.py` (docstring) | | training, then test 1 (open-loop error at 2 s on held-out rooms, next to MuJoCo and to the physics alone) | `scripts/train_cdpr_wm.py` | | a real Stringman room from a LeRobot recording (anchors, named places, floor image) | `scripts/import_room.py` | ```bash hf download MacroSwan/cablepilot-motion --repo-type dataset --local-dir data/motion # ~280 MB, or regenerate: python scripts/cdpr_datagen.py --scenario configs/scenarios/clear_playroom.yaml --n 12000 --seed 0 --workers 4 --out data/motion python scripts/train_cdpr_wm.py --data data/motion --out weights/mine.pt # minutes on a laptop CPU, no GPU python scripts/train_cdpr_wm.py --eval-only --data data/motion --out weights/stringman_g1.pt # the bar to beat ``` On the last 500 trajectories of `MacroSwan/cablepilot-motion` (held out by `--test 500`), the released `weights/stringman_g1.pt` scores **20.4 mm** median grasp-point error at 2 s. The MuJoCo oracle scores 21.5 mm. A new model has to beat that on the same test, and then score at least as well on a task (`cablepilot run`, 3 episodes, both rooms), before it replaces the old one. Open a PR here with the weights, the test-1 JSON written next to them, the run's `result.json` and one line of `why`. To use your model in a run, point `configs/planner_learned.yaml` at it locally (a tuning file may never change the model). **Your own recordings are the best test set there is.** Any Stringman LeRobot dataset with `anchor_poses`, for example `naavox/stringman_cleans`, holds the real state and the real commands. Converting a few of its episodes into the `cdpr_datagen` format and running `--eval-only --test-all` on them gives the model's first error measured on a real machine. Nobody has done that yet: it is the most useful thing on the list below. ## Open jobs (pick one, say so in a Discussion first) 1. **The model on real data.** Test 1 on real Stringman episodes (see Step 7). Then fine-tune on them, if it helps. 2. **`avoid_furniture`.** The card is in `MacroSwan/pilot-tasks`. `cablepilot check` says what is missing: a `near` skill (a runner around `clear_floor.GotoTask`) and a furnished room (`scripts/import_room.py` plus the furniture boxes that `runner.obstacles` reads). 3. **`stringman.py` on hardware.** Check each `ASSUME` against `nf_robot`, then `hover()` in `dry_run`, then for real. 4. **Your room.** Import it with `scripts/import_room.py` and add it as a `--world`, so that everyone's cards are scored in a third real room. ## Where things are `README.md` covers the model and the numbers. `cablepilot/cli.py` has the commands. `cablepilot/core/cards.py` defines the card schema: the predicates, the skills and `signature()`. `cablepilot/cardrun.py` maps cards to the cable skills and writes the result format. `cablepilot/stringman.py` is the real robot adapter. `scripts/` holds the data generator, the trainer and the room importer.