Spaces:
Paused
Paused
| # RosDiff: project report and handover | |
| *RosDiff by Orcanion, Stage 1 tester build. State as of 28 Sept 2026, branch `claude/festive-pasteur-dy6nrl` | |
| (everything below is committed and pushed; CI green). Written so the work can be picked up in any environment: | |
| another computer, a server, another developer, or another AI coding session.* | |
| --- | |
| ## 1. What RosDiff is | |
| A web app for robotics students and engineers. You describe what you want in plain words and RosDiff builds it | |
| and tests it for real: | |
| - **Simulation:** "a Go2 robot dog with three boxes 1–2 m ahead" → a MuJoCo scene around a real robot model, | |
| checked by loading and stepping it in MuJoCo, then run on a cloud GPU with a video, 3D replay, stress test | |
| (MuJoCo Warp) or photoreal render (Isaac Sim). | |
| - **ROS 2 code:** "stop the robot when the laser sees something closer than 0.5 m" → a complete ROS 2 package | |
| (Humble or Jazzy, Python or C++), checked against the real message definitions, then built, tested and | |
| launched in a ROS 2 sandbox in the cloud. | |
| Stage 1 exists to measure quality with ~4 hand-picked testers before building anything commercial. | |
| ## 2. Where everything runs | |
| | Part | Where | Cost control | | |
| |---|---|---| | |
| | Web app + API (FastAPI), database (SQLite), documentation index (Chroma), OpenCode | One server: your PC today; a free Oracle Cloud machine for testing (`deploy/oracle/`) | ₹0 | | |
| | Simulations, sweeps, Isaac renders, ROS 2 checks, dataset export | Modal (serverless GPUs/CPUs), app `rosdiff-sim` (+ `worker/modal_isaac.py`) | $29.98/month, split equally: $7.495 per tester, strict | | |
| | Result files (videos, replays, trajectories) | Cloudflare R2 bucket `rosdiff-runs` | 9 GB total, files expire after 30 days | | |
| | Paid AI models | OpenRouter | ₹500 pool, ₹125 per tester | | |
| | Free AI models (prompt enhancement, first debugging fix) | OpenCode Zen via the `opencode` CLI on the server | free, recorded but not billed | | |
| ``` | |
| browser ──► RosDiff server ──► OpenRouter (paid models, routed by complexity) | |
| │ ├──────────► OpenCode Zen (free: prompt enhancement, first repair) | |
| │ └──────────► Modal (GPU/CPU jobs) ──► Cloudflare R2 (results) ──► browser (signed links) | |
| └── SQLite: testers' work, budgets, feedback · Chroma: MuJoCo/ROS 2 docs | |
| ``` | |
| ## 3. What has been built | |
| ### Generation (the core) | |
| - **Scene generation** (`api/simulate.py`, `api/mjcf_check.py`, `api/robots.py`): RAG over the MuJoCo docs and | |
| XML schema; robots included from MuJoCo Menagerie, never written by the model; robot facts read from the model | |
| file; MuJoCo load + 2 s step in a subprocess; repair loop with MuJoCo's error and the valid attributes. | |
| Robots: Unitree G1, H1, Go2, Franka Panda. | |
| - **ROS 2 code generation** (`api/codegen.py`, `api/ros_check.py`, `api/opencode_runner.py`): OpenCode headless in | |
| a throwaway workspace (or a direct chat call with `CODEGEN_BACKEND=direct`); static validation of the package | |
| layout, syntax, launch files and every message/service type against the real definitions; repair loop. | |
| - **Prompt enhancement** (`api/enhance.py`): each request is first rewritten into a precise brief by a free | |
| OpenCode Zen model; rejected if it drops any number the tester wrote; skipped on any failure. Routing and cost | |
| estimates use the tester's own words. | |
| ### Model routing and budgets | |
| - **Automatic routing** (`api/routing.py`): complexity rating → cheapest capable tier; repairs escalate. Separate | |
| lists for code and scenes (Chinese open-weight models on OpenRouter, chosen by benchmark and price): | |
| | Tier | Code | Scenes | | |
| |---|---|---| | |
| | low | `deepseek/deepseek-v4-flash` | (unused: scenes start at mid) | | |
| | mid | `deepseek/deepseek-v4-pro` | `z-ai/glm-5.2` | | |
| | high | `z-ai/glm-5.2` | `moonshotai/kimi-k2.6` | | |
| - **Free debugger** (`api/zen.py`): the first repair tries `opencode/deepseek-v4-flash-free` through OpenCode Zen; | |
| if that call fails, the same attempt falls back to the next paid tier. Code gets 3 attempts. | |
| - **Simulator choice**: MuJoCo by default, MuJoCo Warp for "how often / robust / random" questions, Isaac Sim for | |
| photoreal renders; Gazebo, Webots and PyBullet are shown as not available (RosDiff scenes are MJCF). | |
| - **Rupee AI budget** (`api/budget.py`): ledger of every model call using OpenRouter's reported cost; ₹125 per | |
| tester from a ₹500 pool; free/exempt calls recorded separately; estimate + confirmation above | |
| `CONFIRM_ABOVE_INR`; 402 when a budget is used up. | |
| - **GPU budget** (`api/jobs.py`): every job is admitted only if its worst case (run until timeout) fits in the | |
| tester's share and the team limit; running jobs count at their worst case, so parallel jobs cannot overshoot. | |
| - **Storage cap**: new jobs wait while R2 holds more than 9 GB; objects expire after 30 days. | |
| ### Cloud jobs (`worker/`, `api/jobs.py`, `api/runs.py`) | |
| - Simulation run with video, WebM, MCAP for Foxglove, trajectory; report page with the Foxglove embed. | |
| - Robustness sweep (MuJoCo Warp, 64 randomized worlds). | |
| - ROS 2 sandbox (`worker/ros/`): colcon build, lint, tests, 10 s launch with node/topic capture, rosbag; no | |
| network, no secrets. | |
| - Cosmos Reason video review (opt-in), Isaac Sim RTX render (daily limit), LeRobot dataset export to Hugging Face. | |
| - Modal by default; RunPod supported for runs and sweeps. | |
| ### Web app (`api/static/app/`, plain HTML/CSS/JS, no build step) | |
| - RosDiff by Orcanion branding (logo, favicon), startup animation (short for returning visitors, skippable), | |
| 10-step beginner tour with the LiDAR simulator example. | |
| - Simulation and code side by side; history; reopen past work; download scene / package zip; copy files. | |
| - Routing line on each result ("AI Routing: automatic · tier · Why?"), "✨ Enhanced prompt", | |
| "Use another simulator" menu, estimate card before costly tasks. | |
| - 👍 Helpful / 👎 Not helpful with an optional note, on scenes, packages and jobs, stored with context. | |
| - "My usage": rupee budget by category, GPU share, and a team view for `ADMIN_TESTERS`. Light/dark, mobile. | |
| ### Operations | |
| - Tester keys: `python -m api.keys add|list|revoke`. Review: `python -m api.review summary|list|show|usage`. | |
| - Hosting: `deploy/oracle/setup.sh` + `deploy/oracle/README.md` (free Oracle Cloud machine, HTTPS via Caddy on | |
| `<ip>.sslip.io`, systemd service, nightly DB backup). | |
| - CI (`.github/workflows/ci.yml`): lint + 165 offline tests; worker image smoke test; ROS 2 sandbox for Humble and | |
| Jazzy; LeRobot export. No CI job uses Modal or any paid service. | |
| ## 4. Decisions and why | |
| | Decision | Why | | |
| |---|---| | |
| | Models picked automatically, never by testers | Students shouldn't have to choose; cost stays predictable | | |
| | Chinese open-weight models on OpenRouter | Near-frontier coding benchmarks at a fraction of the price; one key for all | | |
| | GLM-5.2 rather than the newer GLM-5.3 | Stronger published benchmarks and cheaper | | |
| | Kimi K2.6 as the scene escalation | A different model family for the hard repair; top open-weight Intelligence Index | | |
| | DeepSeek V4 Flash not added to scene tiers | Only one source claims it beats Kimi K2.6; others disagree | | |
| | Free model only for enhancement and the first repair | Free offers are rate-limited and can disappear; paid fallback keeps quality | | |
| | Only MuJoCo, MuJoCo Warp, Isaac Sim offered | RosDiff scenes are MJCF; other simulators are shown, not faked | | |
| | GPU limits checked against each job's worst case | Strict: 4 testers can never exceed $29.98 together or $7.495 each | | |
| | Oracle Cloud Always Free for hosting | Free forever, Mumbai/Hyderabad regions, 24 GB RAM; Vercel can't run long requests or keep a database | | |
| | Create the Oracle machine ~1 week before testing | Oracle may reclaim Always Free machines idle for 7 days | | |
| ## 5. Moving to a different environment | |
| ### What is in git, and what is not | |
| | In the repository | NOT in the repository (copy or recreate) | | |
| |---|---| | |
| | All code, web app, workers, tests, CI, docs | `.env` (all keys and settings) | | |
| | `.env.example` (every setting, documented) | `testers.yaml` (tester key hashes) | | |
| | `deploy/oracle/` hosting script | `data/generations.sqlite3` (testers' work, budgets, feedback) | | |
| | | `data/chroma`, `data/sources` (rebuilt by `python -m rag.ingest`) | | |
| | | `~/.modal.toml` (Modal login) | | |
| ### New development machine | |
| ```bash | |
| git clone <repo> && cd rosdiff # or copy the folder | |
| python3 -m venv .venv && . .venv/bin/activate | |
| pip install -r requirements.txt -r requirements-dev.txt | |
| cp .env.example .env # fill in keys (section 6) | |
| set -a; source .env; set +a | |
| python -m rag.ingest # a few minutes | |
| python -m pytest -q # 165 tests, offline | |
| npm i -g opencode-ai # free models; optional but recommended | |
| uvicorn api.main:app --port 8000 # open http://localhost:8000/ | |
| ``` | |
| To keep existing testers and their history, also copy `testers.yaml` and `data/generations.sqlite3`. | |
| ### Server for testers | |
| Follow `deploy/oracle/README.md`. It copies the code plus `.env`, `testers.yaml` and the database once, then | |
| `setup.sh` does the rest. | |
| ### Another AI coding session | |
| Point it at this repository and this file. Useful starting points: `README.md` (full technical detail), | |
| `api/main.py` (every endpoint), `api/routing.py` (models), `api/budget.py` + `api/jobs.py` (limits), | |
| `api/static/app/app.js` (web app). Run `python -m pytest -q` before and after changes. | |
| ## 6. Accounts and secrets needed (names only, never commit values) | |
| | Service | Where it goes | Setting names | | |
| |---|---|---| | |
| | OpenRouter | `.env` | `LLM_API_KEY` (`LLM_BASE_URL` defaults to OpenRouter) | | |
| | Modal | `~/.modal.toml` (`modal token new`) or `.env` | `MODAL_TOKEN_ID`, `MODAL_TOKEN_SECRET` | | |
| | Cloudflare R2 | `.env` **and** Modal secret `rosdiff-r2` | `R2_ACCOUNT_ID`, `R2_ACCESS_KEY_ID`, `R2_SECRET_ACCESS_KEY`, `R2_BUCKET` | | |
| | Hugging Face (dataset export, Cosmos) | Modal secret `rosdiff-hf` | `HF_TOKEN`; `.env`: `HF_NAMESPACE` | | |
| | NVIDIA NGC (Isaac Sim image, if asked) | Modal secret named by `MODAL_NGC_SECRET` | `REGISTRY_USERNAME`, `REGISTRY_PASSWORD` | | |
| | OpenCode Zen (only if the free model asks) | `.env` | `ZEN_API_KEY` | | |
| | Report links | `.env` | `REPORT_SECRET` (random), `PUBLIC_BASE_URL` | | |
| Keys were pasted into chat during setup earlier; rotate the OpenRouter and R2 keys if that hasn't been done. | |
| ## 7. Key settings (all in `.env.example`) | |
| | Setting | Default | Meaning | | |
| |---|---|---| | |
| | `BUDGET_TOTAL_INR` / `BUDGET_PER_USER_INR` | 500 / 125 | Rupee AI budget | | |
| | `CONFIRM_ABOVE_INR` | 10 | Ask before tasks whose worst case is above this | | |
| | `ADMIN_TESTERS` | — | Testers who see the team view | | |
| | `MONTHLY_CREDIT_LIMIT_USD` / `COMPUTE_TESTERS` | 29.98 / 4 | GPU budget and how many share it | | |
| | `STORAGE_LIMIT_GB` | 9 | R2 limit for all testers together | | |
| | `ISAAC_DAILY_LIMIT` | 10 | Isaac renders per day | | |
| | `PROMPT_ENHANCE` / `ENHANCE_MODEL` | true / `opencode/deepseek-v4-flash-free` | Free prompt enhancement | | |
| | `FREE_DEBUG` / `DEBUG_MODEL` | true / `opencode/deepseek-v4-flash-free` | Free first repair | | |
| | `CODE_MODELS` / `SCENE_MODELS` | built-in lists | JSON to replace the model lists | | |
| | `CODEGEN_BACKEND` | opencode | `direct` = plain chat calls, no OpenCode | | |
| | `CODE_MAX_ATTEMPTS` / `MJCF_MAX_ATTEMPTS` | 3 / 3 | Drafts + repairs | | |
| | `AUTO_REVIEW` | false | Cosmos review of every run | | |
| ## 8. Not verified or not done yet | |
| - **Not checked live from the build environment:** OpenCode Zen's free model id | |
| (`deepseek-v4-flash-free`); OpenRouter prices and the ids `z-ai/glm-5.2`, `moonshotai/kimi-k2.6` (taken from | |
| third-party pages; billing uses OpenRouter's actual reported cost regardless); the Oracle setup script on a | |
| real Oracle machine (syntax and parts tested only). | |
| - **Not built:** follow-up edits on an existing package ("now add a parameter"), editing files in the browser, | |
| uploading your own project; Gazebo/Webots/PyBullet; per-tester storage split (storage is shared, 9 GB). | |
| - **Deployment state (as last reported by the owner):** Modal app deployed with R2 secrets; the server has run on | |
| the owner's PC; Oracle hosting not created yet (planned ~1 week before testers start). | |
| ## 9. History (commits, oldest first) | |
| 1. Stage 1 MuJoCo scene generation API; ROS 2 code generation via OpenCode | |
| 2. GPU simulation runs (RunPod worker, R2 storage, report page); Foxglove embed, WebM video, CI image build | |
| 3. GLM via OpenRouter; simulations on Modal (RunPod optional) | |
| 4. Job framework with per-job cost and a monthly budget | |
| 5. ROS 2 sandbox (build, lint, test, launch); `/code/check`; H1, Go2, Panda robots; hold controller | |
| 6. MuJoCo Warp sweeps; Cosmos Reason review; LeRobot export; Isaac Sim renders; tool docs and costs | |
| 7. Free-of-storage default Modal deploy; R2 near-full refusal; separate coding model | |
| 8. Web app for testers | |
| 9. Automatic model routing and the rupee budget | |
| 10. Fix for Go2/H1 scene generation (unnamed free joint) | |
| 11. Branding, tour, feedback, per-tester GPU shares, free prompt enhancement | |
| 12. 9 GB storage limit | |
| 13. Free debugger on OpenCode Zen; separate code and scene model lists | |
| 14. Oracle Cloud hosting script and guide | |