Skip to content

getModuleFileEntryAsBytes / .bin byte-exact bundled data does not work on web (silent wrong data, not an error) #128

Description

@clholgat

Summary

The byte-exact bundled-data mechanism added for getModuleFileEntryAsBytes/.bin srcs files (docs: docs/docs/advanced-images.md#byte-exact-raw-bitmap-data, example: apps/helloworld/src/valdi/hello_world/src/ByteExactImageExample.tsx) does not work on the web target, and fails silently rather than erroring.

The docs state:

getModuleFileEntryAsBytes returns the file's exact bytes, on every platform.

This is not true for web.

Root cause

Two separate gaps, at two different layers:

  1. Compiler: .bin srcs files are never emitted into the web build output at all. I confirmed this empirically — after building a consuming app's web target, its .ayab_web_cache output directory (the Valdi-compiled JS/asset tree consumed by webpack) contains zero .bin files anywhere, even though the same module builds and bundles correctly for iOS/Android/macOS. Looking at bzl/valdi/valdi_compiled.bzl, _get_srcs_js_paths only maps .ts/.tsx/.js sources to .js web outputs — any other extension is silently skipped and never declared as a web output. The .valdimodule archive packing that makes .bin retrievable at runtime (_will_generate_valdimodule, valdi/src/valdi/runtime/Resources/Bundle.cpp) is only wired for iOS/Android/native/standalone targets; there is no web equivalent. (There is a working precedent for "verbatim-copy a raw file into a per-platform web output path" — the strings-JSON pipeline, e.g. _get_web_string_resource_paths in the same file — but that copy is performed by the Swift local_valdi_compiler binary itself as part of its declared ctx.actions.run outputs, not by a Bazel/shell step, so extending it to .bin requires compiler-side (Swift) changes, not just Bazel/TS.)

  2. Runtime: getModuleEntry in src/valdi_modules/src/valdi/web_renderer/src/ValdiWebRuntime.ts (~line 293) is a hardcoded stub:

    // Stubbed — was using jsonContext (removed). Localized strings now use
    // _strings_preload.js generated by collapse_web_paths instead.
    getModuleEntry(module: string, pathStr: string, asString: boolean) {
      return '{}';
    }

    It always returns the literal string '{}' regardless of arguments, and never throws. getModuleFileEntryAsBytes casts this to Uint8Array, so callers get back the 2 bytes {/} with no error, no warning — code that then tries to decode those bytes as image data (e.g. decodeBitmap) doesn't throw either; it just silently produces a bogus/degenerate result (in our repro, a 1x1 bitmap instead of the real 60x10 source image). This is the more actionable half of the bug: even without the compiler-side fix, getModuleEntry should fail loudly on web instead of returning stub data that downstream code can't distinguish from success.

Repro

  1. Add a module with a .bin src per the advanced-images.md byte-exact pattern (srcs = glob([..., "src/**/*.bin"]), inline_assets = True).
  2. Call getModuleFileEntryAsBytes(module, path) for that file from a component that also renders on web (e.g. via webpack serve / npm start in a consuming app's web/ dir).
  3. Native builds (iOS/Android/macOS-standalone bazel test) return the correct bytes. The web build returns Uint8Array bytes for the string '{}' instead, and no error is surfaced anywhere (console, exception, or otherwise).

Suggested fix

  • Short term (low risk, high value): make getModuleEntry's web stub throw instead of returning '{}', so this fails loudly instead of masquerading as success. This alone would have made the underlying bug in our app immediately obvious instead of costing significant debugging time to trace through decoded-bitmap-looks-wrong -> traced back through three layers to this stub.
  • Longer term: either (a) extend the compiler to emit .bin srcs verbatim into a web output path (mirroring the strings-JSON precedent) plus a webpack-visible registry (mirroring the existing _image_registry.js / _strings_preload.js generation in bzl/valdi/valdi_collapse_web_paths.bzl), giving .bin real web support; or (b) if web support isn't planned, update advanced-images.md to explicitly scope the "on every platform" claim to native platforms only, so consumers don't reach for this API expecting web parity.

Environment

  • Found via a downstream consumer app pinned to Valdi SHA 919f23c8ebdd3582b58821a9ff252b93ead5eb75 (the commit that introduced the byte-exact .bin feature / ByteExactImageExample.tsx).
  • Repro path: web dev server via webpack serve (webpack 5), bazel test (native JS engine / valdi_standalone) does not reproduce this — it only shows up in an actual browser/webpack build.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions