Spaces:
Running
Running
File size: 19,582 Bytes
a045435 49abe48 a045435 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 | <!doctype html><html><head><meta charset='utf-8'><title>Reference parity: `pymunk-reference` against the published Push-T</title><style>
body{max-width:980px;margin:2rem auto;padding:0 1.25rem;font:15px/1.55 -apple-system,Segoe UI,Roboto,sans-serif;color:#1c2330;background:#fafbfc}
h1{font-size:1.7rem;border-bottom:2px solid #d8dee8;padding-bottom:.4rem}h2{margin-top:2.2rem;border-bottom:1px solid #e3e8ef;padding-bottom:.25rem}
table{border-collapse:collapse;margin:1rem 0;font-size:13.5px;font-variant-numeric:tabular-nums}th,td{border:1px solid #d8dee8;padding:.35rem .6rem;text-align:left}
th{background:#eef2f7}td:not(:first-child){text-align:right}code{background:#eef2f7;padding:.1rem .3rem;border-radius:3px;font-size:.92em}
pre{background:#1c2330;color:#e6edf5;padding:.8rem 1rem;border-radius:6px;overflow:auto}pre code{background:none;color:inherit}
img{max-width:100%;border:1px solid #d8dee8;border-radius:4px;margin:.5rem 0}blockquote{border-left:3px solid #6aa9ff;margin:0;padding:.2rem 1rem;color:#4b5563}
nav{font-size:13px;margin-bottom:1.5rem}nav a{margin-right:1rem}
</style></head><body><nav><a href="index.html">index</a> <a href="contact_friction_study.html">contact friction study</a> <a href="convergence_study.html">convergence study</a> <a href="dataset_definition.html">dataset definition</a> <a href="ds_distribution_at_1000.html">ds distribution at 1000</a> <a href="ds_distribution_report.html">ds distribution report</a> <a href="episode_store.html">episode store</a> <a href="human_pushing_analysis.html">human pushing analysis</a> <a href="lewm_parity.html">lewm parity</a> <a href="reference_parity.html">reference parity</a> <a href="session_fidelity_and_behaviour.html">session fidelity and behaviour</a></nav><h1 id="reference-parity-pymunk-reference-against-the-published-push-t">Reference parity: <code>pymunk-reference</code> against the published Push-T</h1>
<p><code>pusht_sim.envs.pymunk_reference_env.ReferencePushTEnv</code> claims to <em>be</em> the
environment the Push-T literature was measured on: LeRobot's <code>gym_pusht</code>,
inherited unchanged from Diffusion Policy. This file records what that claim
was checked against, what was found to differ, and what was done about each
difference.</p>
<p>Reproduce with:</p>
<pre><code>.venv/bin/python scripts/reference_parity.py
</code></pre>
<p>Sources read line by line on 2026-09-24:</p>
<ul>
<li><code>gym_pusht/envs/pusht.py</code> from the installed <code>gym-pusht</code> 0.1.6, and the same
file on
<a href="https://raw.githubusercontent.com/huggingface/gym-pusht/main/gym_pusht/envs/pusht.py"><code>main</code></a>.
Line citations everywhere in this repo are the <code>main</code> numbering; the 0.1.6
file is the same code two lines further down.</li>
<li><code>diffusion_policy/env/pusht/pusht_env.py</code> (the original the LeRobot <code>pusht</code>
dataset was recorded in), to confirm the quirks below are not a gym-pusht
regression but were there when the data was collected.</li>
</ul>
<h2 id="why-the-harness-needs-two-interpreters">Why the harness needs two interpreters</h2>
<p>The published environment <strong>cannot run on pymunk 7</strong>: <code>_setup</code> calls
<code>Space.add_collision_handler</code>, removed in that major. This repo runs pymunk
7.3.0. So <code>scripts/reference_parity.py</code> runs our backend in this venv, and
drives the real <code>gym_pusht</code> in a separate virtualenv on pymunk 6.11.1
(<code>/tmp/refenv</code>, <code>--refenv</code> to move it) through a JSON pipe.</p>
<p>That makes every zero below say two things at once: our reproduction matches
the published environment, <em>and</em> the Chipmunk changes between pymunk 6 and 7 do
not move this scene.</p>
<pre><code>ours python 3.12.3 pymunk 7.3.0 numpy 2.5.3
theirs python 3.12.3 pymunk 6.11.1 numpy 2.5.3 gym-pusht 0.1.6
</code></pre>
<h2 id="measurements">Measurements</h2>
<p>Per seed the harness compares two things: the published <code>reset(seed=n)</code> (the
published draw put through the published legacy setter), and then an open-loop
rollout from one shared start with an identical action sequence generated by
the harness itself. All figures are the <strong>worst deviation over every step of
every episode</strong>.</p>
<h3 id="default-run-5-seeds-x-40-control-steps">Default run: 5 seeds x 40 control steps</h3>
<pre><code> seed steps start agent block angle keypts cover obs32 dn touch
-------------------------------------------------------------------------------------------------------------------
0 40 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 3.747e-16 1.493e-05 0 8
1 40 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 9.714e-17 1.525e-05 0 4
2 40 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 6.939e-18 1.469e-05 0 9
3 40 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 3.608e-16 1.510e-05 0 9
4 40 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 1.665e-16 1.480e-05 0 6
-------------------------------------------------------------------------------------------------------------------
worst 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 3.747e-16 1.525e-05 0
PASS: worst float64 deviation 3.747e-16 against tolerance 1.000e-09; worst contact-count gap 0; worst packed-observation gap 1.525e-05 against 6.200e-05 (float32)
</code></pre>
<h3 id="long-horizon-10-seeds-x-120-control-steps">Long horizon: 10 seeds x 120 control steps</h3>
<p><code>--seeds 0,1,2,3,4,5,6,7,8,9 --steps 120</code>. Seed 7 is in contact for 118 of its
120 steps, so this is not a measurement of two blocks sitting still.</p>
<pre><code> seed steps start agent block angle keypts cover obs32 dn touch
-------------------------------------------------------------------------------------------------------------------
0 120 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 1.943e-16 1.526e-05 0 14
1 120 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 4.857e-16 1.526e-05 0 12
2 120 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 2.602e-17 1.430e-05 0 7
3 120 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 3.608e-16 1.523e-05 0 74
4 120 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 5.551e-17 1.491e-05 0 11
5 120 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 1.665e-16 1.509e-05 0 10
6 120 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 8.674e-19 1.522e-05 0 10
7 120 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 1.516e-05 0 118
8 120 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 1.388e-17 1.516e-05 0 52
9 120 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 3.331e-16 1.512e-05 0 18
-------------------------------------------------------------------------------------------------------------------
worst 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 4.857e-16 1.526e-05 0
PASS: worst float64 deviation 4.857e-16 against tolerance 1.000e-09; worst contact-count gap 0; worst packed-observation gap 1.526e-05 against 6.200e-05 (float32)
</code></pre>
<h3 id="the-two-published-extra-options-block-cog-060-damping-09">The two published extra options: <code>--block-cog 0,60 --damping 0.9</code></h3>
<pre><code> seed steps start agent block angle keypts cover obs32 dn touch
-------------------------------------------------------------------------------------------------------------------
0 40 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 1.525e-05 0 9
1 40 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 1.388e-16 1.525e-05 0 8
2 40 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 1.469e-05 0 12
3 40 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 6.072e-17 1.505e-05 0 25
4 40 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 3.608e-16 1.480e-05 0 22
-------------------------------------------------------------------------------------------------------------------
worst 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 3.608e-16 1.525e-05 0
PASS: worst float64 deviation 3.608e-16 against tolerance 1.000e-09; worst contact-count gap 0; worst packed-observation gap 1.525e-05 against 6.200e-05 (float32)
</code></pre>
<h3 id="reading-the-two-non-zero-columns">Reading the two non-zero columns</h3>
<ul>
<li><strong><code>cover</code>, up to 4.9e-16.</strong> Coverage is the only quantity neither side reads
off a body. Upstream intersects two shapely <code>MultiPolygon</code>s built from the
engine's shapes; this repo intersects the union polygon <code>geometry.tee_polygon</code>
builds from the same vertices. Same area, different order of floating-point
additions. Four ulps of a number in <code>[0, 1]</code>.</li>
<li><strong><code>obs32</code>, up to 1.5e-05.</strong> <code>PushTState.to_vector</code> packs the observation as
float32 because it crosses a JSON/HTTP boundary to the viewer; upstream's
<code>get_obs</code> concatenates to float64. A float32 ulp at the top of the workspace
is 3.05e-05, so this column is one ulp, and it is checked against its own
float32 tolerance. <code>get_state()</code> and <code>info["state"]</code> keep full precision -
which is what every other column is measured from - and the LeRobot <code>pusht</code>
dataset stores float32 anyway.</li>
</ul>
<p>Everything else - pusher position, block position, block angle, all sixteen
keypoint coordinates, and the contact count - is <strong>bit-identical</strong>, at the
seeded start and at every step.</p>
<h2 id="every-gap-found-and-what-was-done">Every gap found, and what was done</h2>
<h3 id="1-observation-angle-convention-was-wrong-now-faithful">1. Observation angle convention - <em>was wrong, now faithful</em></h3>
<p><code>get_obs</code> reports <code>self.block.angle % (2 * np.pi)</code> (L383); this backend
reported the raw pymunk angle, so a block at -0.314 rad here read 5.969 rad
there. Every LeRobot <code>pusht</code> state was written by <code>get_obs</code>, so anything
comparing our state vector against the dataset saw a 2pi discontinuity.</p>
<p><strong>Fixed:</strong> <code>_poses</code> now reduces modulo a turn, so <code>get_state()</code>, the packed
observation and <code>info["state"]</code> all use the published convention.
<code>reset(state=...)</code> still round-trips as a <em>pose</em> - feed -0.314 in, get 5.969
out, feed that back in and nothing moves - and the unwrapped, continuously
winding angle is still on the body as <code>env._block.angle</code> for anything that
needs it. Nothing in this repo did: every angle comparison here already goes
through a wrap (<code>experiments._wrap_angle</code>, <code>policies._wrap</code>) or through
<code>cos</code>/<code>sin</code>.</p>
<h3 id="2-initial-state-sampling-was-wrong-now-faithful">2. Initial-state sampling - <em>was wrong, now faithful</em></h3>
<p>Upstream draws <code>integers(50, 450)</code>, <code>integers(50, 450)</code>, <code>integers(100, 400)</code>,
<code>integers(100, 400)</code>, <code>uniform(-pi, pi)</code> (L276-280) from the generator
gymnasium builds as <code>np.random.default_rng(seed)</code>. <code>BasePushTEnv.sample_state</code>
drew uniform floats over <code>50..462</code> and <code>100..412</code> - a different support, so a
seeded start here was not a start the published environment could produce.</p>
<p><strong>Fixed:</strong> <code>ReferencePushTEnv.sample_state</code> overrides the base draw with the
published one. Same constructor, same order, so seed <em>n</em> here selects the state
seed <em>n</em> selects there - the <code>start</code> column above is that equality, measured
end to end at 0.0.</p>
<p><strong>Consequence, deliberate:</strong> this is the one place the backend breaks the base
class's "one seed, one scenario on every backend" rule, so <code>make_env("pymunk",
seed=n)</code> and <code>make_env("pymunk-reference", seed=n)</code> no longer agree on where
the episode starts. Anything comparing engines has to sample once and pass the
state to the others with <code>reset(state=...)</code>; the published draw is integral and
strictly inside the SI backends' support, so it is always a legal start for
them. The web session does exactly this.</p>
<h3 id="3-legacy-_set_state-was-a-deviation-now-a-deviation-with-an-exact-entry-point">3. Legacy <code>_set_state</code> - <em>was a deviation, now a deviation with an exact entry point</em></h3>
<p>Upstream assigns the block <em>position</em> and then the angle (L457-462) and says
why: "for compatibility with legacy data". pymunk rotates about the centre of
gravity, which for this block is <code>(0, 45)</code>, so the pose asked for is not the
pose obtained - upstream's own docstring shows <code>[0, 10, 20, 30, 1]</code> coming back
as <code>(57.87, 50.69)</code>, a shift of 42 units. Recorded LeRobot/Diffusion Policy
episode starts were <em>all</em> written through that setter, so replaying one through
a setter that honours the pose puts the block in the wrong place.</p>
<p><strong>Both behaviours, explicitly:</strong></p>
<ul>
<li><code>reset(state=...)</code> keeps <code>add_tee</code>'s angle-first order (L510-511), so the
pose round-trips. This is what <code>PushTState</code>, <code>geometry.transform_points</code> and
both other backends mean by a pose, and the whole three-way comparison rests
on it.</li>
<li><code>set_reference_state(vector_or_state)</code> is the legacy setter, verbatim, and
reproduces the documented shift exactly. Use it for recorded data.</li>
<li><code>reset_reference(seed)</code> is the two published steps together - published draw,
legacy setter - i.e. gym-pusht's <code>reset(seed=n)</code> in one call.</li>
</ul>
<h3 id="4-missing-accessors-added">4. Missing accessors - <em>added</em></h3>
<ul>
<li><strong><code>get_keypoints()</code></strong> (L517-537): the eight block corners in world
coordinates, <code>(8, 2)</code>; <code>.flatten()</code> is the 16-value <code>environment_state</code>
LeRobot trains <code>environment_state_agent_pos</code> policies on. Taken from the
engine's own shapes, because the index order in upstream's diagram is
Chipmunk's re-winding of the input vertices and not <code>geometry.tee_vertices</code>'
order. Both pymunk majors were checked to return the same winding.</li>
<li><strong>Contact count</strong> (L425-426, L413-420): <code>info["n_contacts"]</code>, on pymunk 7's
<code>Space.on_collision(0, 0, post_solve=...)</code> in place of the removed
<code>add_collision_handler</code>. Only <code>post_solve</code> is registered, so the solver is
untouched - and the parity runs above, which include an episode in contact
for 118 of 120 steps, are the evidence that registering it changed no
physics.</li>
<li><strong><code>block_cog</code> and <code>damping</code></strong>: constructor arguments of <code>ReferencePushTEnv</code>,
exactly as they are constructor arguments of <code>PushTEnv</code> (L140-141, L165-166),
and measured to be exact above. They are deliberately <strong>not</strong> in <code>SimParams</code>:
that dataclass is the shared, swept, serialised schema of the two SI engines,
and neither of them has a "move the block's centre of gravity" or "multiply
every velocity by this each step" knob to map them onto. Putting them there
would add two permanently-inapplicable entries to every schema response,
every sweep and every UI form. <code>make_env</code> therefore keeps offering the
published defaults, and a caller who wants the options constructs the class.</li>
</ul>
<h3 id="5-found-while-reading-the-two-sources-side-by-side">5. Found while reading the two sources side by side</h3>
<ul>
<li><strong><code>info["n_contacts"]</code> is divided by the wrong number</strong> (L414). <code>_get_info</code>
computes <code>n_steps = int(1 / self.dt * self.control_hz)</code>, which is 1000, where
<code>step</code> (L240) computes the substep count as <code>int(1 / (self.dt *
self.control_hz))</code>, which is 10. So the published "contact points per step"
is the real count over a hundred times too large a divisor: it is 1 whenever
anything touched during the step and 0 otherwise. <strong>Reproduced bug-for-bug</strong>,
with the raw sum published alongside it as <code>info["n_contact_points"]</code>.</li>
<li><strong><code>get_keypoints</code> is broken on NumPy 2.</strong> L537 calls <code>np.row_stack</code>, removed
in NumPy 2.0, so <code>obs_type="environment_state_agent_pos"</code> raises
<code>AttributeError</code> on the pinned gym-pusht 0.1.6 + NumPy 2.5.3 install.
<strong>Deviation:</strong> ours uses <code>np.vstack</code>, which is what <code>row_stack</code> was an alias
for - same values, and it runs. (The parity harness restores the alias in the
reference subprocess so the published function can be called at all.)</li>
<li><strong>No action clipping upstream.</strong> <code>step</code> declares <code>Box(0, 512)</code> but never
enforces it; <code>BasePushTEnv.step</code> clips. Inside the declared range - every
action a policy or a dataset produces - the two are identical. Left as is.</li>
<li><strong>Observation dtype.</strong> Upstream concatenates to float64; <code>PushTState.to_vector</code>
packs float32. See "Reading the two non-zero columns" above.</li>
<li><strong>Termination comparison.</strong> Upstream terminates on <code>coverage > 0.95</code> (L256,
strict); the base class uses <code>>=</code>. They differ only on an episode sitting
exactly on the threshold.</li>
<li><strong>Truncation.</strong> Upstream never truncates (L263 <code>truncated = False</code>); the base
truncates at <code>max_episode_steps</code>. Base-owned, documented, left as is.</li>
<li><strong>The two physics quirks were already reproduced and still are</strong>: shape
friction is 0.0 everywhere because <code>body.friction = 1</code> (L476, L512) assigns
to a property <code>pymunk.Body</code> does not have, and the block's moment is twice
the <em>bar's</em> because L501 passes <code>vertices1</code> where it means <code>vertices2</code>. Both
are present in the Diffusion Policy original too, so the recorded dataset has
them, and <code>tests/test_env_reference.py</code> pins both.</li>
</ul>
<h2 id="tests">Tests</h2>
<p><code>tests/test_env_reference.py</code> covers all of the above without needing the
reference virtualenv - gym-pusht's semantics are re-stated there as literals
from its source rather than imported. The single test that runs the real
package shells out to <code>scripts/reference_parity.py</code> and skips when
<code>/tmp/refenv</code> is absent.</p>
<pre><code>.venv/bin/python -m pytest tests/test_env_reference.py -q
</code></pre></body></html> |