Fix the JS test shard dropping results and never awaiting the snapshot tests [CLF-454] - #1571
Merged
Merged
Conversation
KGP's Karma reporter writes its ##teamcity[...] service messages to stdout back-to-back with no newline between them. KGP's Gradle-side parser (TCServiceMessageOutputStreamHandler) buffers stdout until it sees a newline and drops any "line" longer than 1 MiB, logging "too long teamcity service message (more than 1Mb). Event was lost" and "IllegalStateException: no running test". Since the whole run is one line, every 1 MiB chunk is thrown away and only the final remainder is parsed: Gradle was recording roughly 2,000 of the 27,955 workflow-runtime JS tests, and any failure in a dropped chunk was invisible. Add a karma.config.d script that wraps the reporter so every message ends with a newline, and have KotlinMultiPlatformConventionPlugin append that directory to every generated karma.conf.js. The plugin hooks KotlinJsTest.onTestFrameworkSet rather than calling useKarma so that it doesn't replace KGP's default Karma configuration (and its ChromeHeadless browser) with an empty one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Burst 2.13.0 only generates specializations that return runTest(...) when the parameterized test function's body calls runTest itself. Since #1568 moved the body of RenderWorkflowInSnapshotSaveRestoreTest into a helper to dodge a Kotlin/Native compiler crash, the generated per-value test functions have been calling the original as a bare statement and returning nothing. On JS that return value is the Promise Mocha awaits, so all 4,096 saves_to_and_restores_from_snapshot_* tests completed instantly and passed without their bodies finishing. The un-awaited bodies then ran concurrently with the rest of the suite and starved the event loop, which is what made unrelated tests hit Mocha's 2 s timeout on #1442. Call runTest directly again, but without any argument that refers to `this` (the shape that crashes the Kotlin/Native backend), and launch both runtimes in explicit TestScope(dispatcherUsed) scopes instead. The class KDoc documents both constraints. Verified locally on main: jvmTest and iosSimulatorArm64Test for this class each record 16,384 tests with 0 failures, and the full jsBrowserTest run records 27,955 tests with 0 failures. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
zach-klippenstein
enabled auto-merge
September 4, 2026 22:36
handstandsam
approved these changes
Sep 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The "JS Unit Tests for KMP Modules" shard has not been testing what it claims to. Two independent problems, both present on
main:1. Gradle throws away most of the Karma results
KGP's Karma reporter (
kotlin-web-helpers/dist/karma-kotlin-reporter.js) writes its##teamcity[...]service messages to stdout back-to-back with no newline between them. KGP's Gradle-side parser (TCServiceMessageOutputStreamHandler) buffers stdout until it sees a newline and drops any "line" longer than 1 MiB, logging:Since the whole run is a single line, every 1 MiB chunk is discarded and only the final remainder is parsed. On recent
mainruns Gradle recorded roughly 2,000 of the 27,955workflow-runtimeJS tests that actually run. Any failure inside a dropped chunk is invisible, so the shard is green regardless.Fix:
build-logic/karma.config.d/newline-delimited-kotlin-reporter.jswraps KGP's reporter so every message ends with a newline, andKotlinMultiPlatformConventionPluginappends that directory to every generatedkarma.conf.jsviaKotlinKarma.useConfigDirectory. After this change all 27,955workflow-runtimeand 50workflow-coretests are recorded and the overflow message is gone.2. Burst never awaits
RenderWorkflowInSnapshotSaveRestoreTeston JSBurst 2.13.0 only generates specializations that
return runTest(...) { … }when the parameterized test function's body callsrunTestitself. WhenrunTestis behind a helper (which #1568 introduced to dodge a Kotlin/Native compiler crash), the generated per-value functions call the original as a bare statement and return nothing. On JS that return value is the Promise Mocha has to await, so all 4,096saves_to_and_restores_from_snapshot_*tests completed instantly and "passed" without their bodies ever finishing. Those thousands of un-awaited bodies then ran concurrently with the rest of the suite and starved the event loop, which is what made unrelated tests hit Mocha's 2 s timeout on #1442.Fix: the test calls
runTestdirectly again, but with no arguments that referencethis(the shape that crashes Kotlin/Native), and launches both runtimes in explicitTestScope(dispatcherUsed)scopes instead. The class KDoc documents both constraints. Burst itself should also be fixed to return the delegate's value; that's a follow-up.Verification
Locally, with these changes on
main::workflow-runtime:jvmTest(this class):workflow-runtime:iosSimulatorArm64Test(this class):workflow-runtime:jsBrowserTest(full):workflow-core:jsBrowserTest(full)Ref: CLF-454
🤖 Generated with Claude Code