[Bug] Released CloseDrawer BC Transformer checkpoint has defective BatchNorm running buffers (Oracle 0/20)

[wenyilll]

[Bug] Released CloseDrawer BC Transformer checkpoint has defective BatchNorm running buffers (Oracle 0/20)

Summary

The publicly released CloseDrawer BC Transformer checkpoint produces 0/20 success under the official Oracle baseline (layout/style=[[0,0]], seed=123000, horizon=500), while the published reference reports near-perfect success. We isolated the failure to BatchNorm running buffers (running_mean / running_var / num_batches_tracked) via a same-environment single-variable A/B. Replacing only those buffers (diagnostic checkpoint; not a proposed official fix) recovers 12/20.

We request the exact BC checkpoint SHA used for the published reference results, and the correct BN running buffers (or a re-exported checkpoint) matching that reference.

Environment pins

Outer repo SHA: 950beaebf7d5efa93a63ab3b49b3bf69df3431e8

Submodules:

Path	SHA
env/mimicgen	36ef5b744eeac684e3e5c2809f403177b8fc15e3
env/robocasa	6bab98c97d6914cbb10cbbfcfb5e124bdde4e1d0
env/robosuite	cb173eb465089b1b4d7038dc8e913f18817f2b0f
policy/robomimic	53c9e8018ac067730be0244b47a3bb709309dd5c
predictor/N2M	8ec853b975d7e4a2ce17724c7e8ae830bbfeedd8
predictor/lelan	6ce7018887820708d8d31c3abe479e0ebc07aabf
predictor/mobipi	d3b122adeab89a890a3cb02bf589848450a40bae

Assets

Asset	SHA256
Official BC checkpoint CloseDrawer_BCtransformer.pth (Drive id 1EZZqefN4FTS9tgknexc5VFtscbaJo7SG)	896565a1610c3e9bbe6f8fb5eefcceb99ca1305e9369b746960bac118c3240e0
Official CloseDrawer HDF5 (Box; used as CloseDrawer_robomimic.hdf5)	292f0419aba6b5f6ada02ff4d1439701cf09f21d25c397ad2d34fe2f39fcd2df

Reproduction (Oracle)

python scripts/run_benchmark.py \
  env.name=CloseDrawer env.render=false \
  'env.layout_and_style_ids=[[0,0]]' \
  policy=bc_transformer predictor=oracle \
  benchmark=evaluation benchmark.num_episodes=20

Causal evidence (frozen as D2_CAUSAL_LOOP_CLOSED)

Condition	Checkpoint	Environment	Success
A0-pristine	official BC	official-pristine	0/20
A0-trace	official BC (same SHA)	official-trace (same assets/runner/seed/layout as A1)	0/20
A1-trace	BN-only diagnostic (buffers re-estimated on train split; weights/γ/β untouched)	official-trace	12/20

Additional constraints / checks:

• SE2 (se2_initial / se2_randomized / se2_predicted) matches across A0-pristine, A0-trace, and A1 for all 20 episodes.
• Strict tensor diff: only 180 BN buffers changed (running_mean x60, running_var x60, num_batches_tracked x60); 0 illegal non-buffer changes.
• Official BC anomaly sample: running_var max ~ 1.87e6, num_batches_tracked=500001 across BN layers.
• Visual encoder features collapse under eval BN stats; recover after BN-buffer-only patch.
• Diagnostic checkpoint SHA (for evidence only; not an official release candidate): 8d070a909a33c99b2cf512c817cdb59401ca4f36ea4c0430f9ffc798f2bbb177

Request

Please provide:

1. The SHA256 of the BC checkpoint used to generate CloseDrawer_0_0_bc_transformer_oracle reference results.
2. The correct BatchNorm running buffers for that checkpoint (or a re-uploaded official .pth whose eval-mode BN stats match the reference run).
3. Confirmation whether the Drive file CloseDrawer_BCtransformer.pth (1EZZqefN4FTS9tgknexc5VFtscbaJo7SG) is expected to match the paper/reference artifact.

Notes

• We are not asking reviewers to accept our diagnostic BN re-estimate as the official fix; it is only a causal probe.
• A separate Diffusion Policy Oracle gate on the released CloseDrawer_diffusion.pth also failed (0/20) and is tracked independently; this issue is specifically about BC.