Wait for validation readiness without consuming rendered frames

[bkaradzic-microsoft]

Summary

• Wait for scene readiness, GUI image readiness, and clean material defines with ready effects in the active camera's render pass. Include utility scenes using the main scene's camera.
• Restore the prior render pass on every exit, including errors.
• Refresh render IDs without rendering extra animation/particle frames; preserve renderCount, screenshot indices, and RenderDoc capture/finalization budgets.
• Allow up to 240 non-rendering wait ticks; on the next readiness check, render if ready or fail if still unready (tick 241 for a continuously unready scene). Preserve once-only cleanup and suite continuation; reject stale callbacks after readiness timeout or failure.
• Restore the original seeded Math.random function as well as its seed between scenes.
• Fail fast on terminal engine effect-error notifications while preserving fallbacks and ignoring disposed effects.

The standalone simulated-host Node test and its dedicated CI job have been removed. The production runner fixes remain unchanged.

Independence

Based on upstream master. Uses existing published Babylon.js 9.21.2 APIs. No JavaScript engine changes, dependency updates, protocol changes, engine-option changes, fixture/reference changes, capture-count changes, or tolerance changes.

Validation

Previously completed native validation of the unchanged production runner:

• Stock Windows x64 / D3D11 / Chakra / RelWithDebInfo Playground: Native Canvas, setParent, and GUI Gradient Linear with transparency passed with unchanged fixtures and gates.
• After the utility-scene readiness correction, Near Menu, Native Canvas, setParent, and the transparent GUI gradient passed together on Linux QuickJS/OpenGL/llvmpipe and Windows Chakra/D3D11.
• An actual invalid GLSL ShaderMaterial followed by a black-frame control on stock Windows x64 / D3D11 / Chakra / Release produced ran=2 passed=1 failed=1 missingRef=0 in 2.121 seconds; the expected process exit was -1.

These are native runtime results, not claims of retained standalone Node test coverage.

Linux CI follow-up

The initial readiness wait now includes associated utility scenes. Previously, Linux QuickJS Near Menu exhausted 240 convergence checks in 114 ms with five utility-scene loads still pending and zero rendered frames because executeWhenReady covered only the main scene.

The existing initial readiness wait and 10-minute timeout now cover associated utility scenes too. Resource loads therefore complete before bounded convergence polling begins. The 240-check limit, canonical Near Menu render count (60), references, and pixel tolerances are unchanged.

Terminal shader error follow-up

• Fail pending scene creation or scene processing outside the compiler's notification callback.
• Ignore remaining fallbacks, retained ready pipelines, and disposed effects.
• Remove observers/timers and invalidate pending readiness/render/screenshot callbacks; late-loaded scenes cannot overwrite the next test.
• Keep the existing readiness budgets, render counts, references, tolerances, dependency pins, and Babylon.js 9.21.2 unchanged.

This does not claim unsupported WGSL preprocessing now reports an error on stock Babylon.js. Early WGSL rejection and asynchronous shader-preparation fixes remain separate Babylon.js changes. This PR consumes terminal errors that the released engine already emits.

[Copilot]

🟡 Changes recommended

Unresolved moderate issues remain in render-pass handling and asynchronous framebuffer evaluation.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds readiness polling for native validation without consuming rendered frames, preserving deterministic state and adding standalone regression coverage.

Changes:

• Waits for scene, GUI, material, effect, and utility-scene readiness.
• Preserves render budgets, render-pass state, cleanup, and seeded randomness.
• Adds documentation, Node tests, and CI execution.

File summaries

File	Summary
Documentation/AddingNewValidationTests.md	Documents readiness and deterministic capture behavior.
Apps/Playground/Tests/validation_native.test.cjs	Adds standalone host-independent runner regression tests.
Apps/Playground/Scripts/validation_native.js	Implements readiness polling and lifecycle handling; findings remain regarding pass ordering/restoration and stale asynchronous evaluation.
.github/workflows/ci.yml	Runs the standalone validation tests in CI.

Review details

Suppressed comments (1)

Apps/Playground/Scripts/validation_native.js:318

• scene.isReady() runs before currentRenderPassId is switched to the active camera pass. Babylon's readiness path calls mesh.isReady(true), which prepares/checks pass-specific draw wrappers; the later materialDefines/effect reads can therefore see an uninitialized camera-pass wrapper and treat the missing values as ready, allowing the first captured frame before that pass's effect is compiled. Select the camera pass before the entire readiness check and keep it inside the existing try/finally.

        if (!scene.isReady()) {

• Files reviewed: 4/4 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

[Copilot]

🟡 Changes recommended

Initial readiness can hang indefinitely when a pending associated utility scene is disposed or detached.

Get a fresh assessment by requesting another Copilot review.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

Utility layers created before the main camera can bypass both readiness phases.

Get a fresh assessment by requesting another Copilot review.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Balanced

[Copilot]

🔵 Needs a closer look

The asynchronous lifecycle handling and private utility-layer instrumentation warrant final human validation across supported native engines.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Copilot]

Copilot review overview

🔵 Needs a closer look

Native asynchronous lifecycle handling and private utility-layer instrumentation warrant final human validation.

Review effort: Balanced
Findings: None

[bghgary]

AI has more comments, but feel free to ignore them since this will be replaced soon.