Conversation
✅ Deploy Preview for golemcloud canceled.
|
a stream of schema values between agents, per the guest-bridge streaming contract in cli/golem-cli/src/bridge_gen/README.md. clean completion is end of input and a producer failure never is: ProduceStream abandons the writer rather than closing it, because a consumer cannot tell a clean close apart from success. recoverable failures belong in the item type, as AgentStream[Result[T, E]]. endpoints are affine. copies share one state, so a transfer through any of them is seen by all; a second transfer, or a read after one, reports ErrStreamTransferred. forwarding a stream that was already read is refused — the items taken cannot be put back — while forwarding an unread one is a pure endpoint move that pumps nothing through this component, which is why decode takes the endpoint without lifting it. writes block for backpressure and keep going when the host accepts fewer items than offered; a dropped reader is reported as a value so a producer can stop cleanly. Trigger and Schedule now refuse stream-bearing methods, including when the only stream is in the output, since both return before the call completes. that is what codec.containsStream is for, propagated through composites. treeSource and treeSink are structs of functions rather than interfaces: an interface call here builds an itab, and the linker then retains the method sets of every structurally-matching type — *StreamReader[SchemaValueTree] exists in the untagged generated package — which drags host-call methods into a native link that has no bodies for them.
so a group declaration can sit in a package-level var alongside the commands it groups, as every other registration call already does.
a new stdlib-only module holding the schema model in the recursive form the REST and RPC surfaces use: a graph is a set of named definitions plus a root type, and a reference names a definition by its string id. the guest SDK's WebAssembly bindings carry a flattened variant — one pool of nodes addressed by index — and will convert at the boundary; nothing here knows about that. this exists so an external program calling golem does not have to pull in the guest SDK's wasm bindings to speak the wire format, and so there is exactly one implementation of canonical json in go. GOL-653 is what two implementations look like. sum types are interfaces with one struct per case, so a reader dispatches with an ordinary type switch and each case carries exactly its own fields; optional values are pointers. host-managed handles (secret, quota-token, permission-card, stream) are narrow interfaces, since core cannot know whether one is a wasm resource or an id on a wire — together with ReleaseAll, which closes the leak when a value fails part-way through being built and the handles already taken would otherwise never come back. Ref carries the graph alongside the node so resolution stays inside it, bounds reference-following so a malformed cycle is reported rather than hanging, and renders a named definition as its id — which is also what stops a recursive type rendering forever.
ports the codec from the guest SDK's index-walking form to node-walking. the rules transfer verbatim — GOL-653 names Go as one of the three SDKs that has them right — while the traversal does not: a record holds its fields rather than indices into a pool, so packing returns a value instead of appending to one, and a failed union branch leaves nothing behind to discard. rendering now also checks cardinality as it goes (record arity, tuple length, flags width, fixed-list length, variant payload agreement), which the index form had to defer. the tests state the contract rather than the implementation: wide integers are canonical base-10 strings and a JSON number is refused; non-canonical spellings (leading zeroes, +, -0) are refused because receivers compare these as strings; a duration is an object of nanoseconds, not ISO-8601; binary is base64url without padding; a map is an array of pairs; flags are the selected names; an absent option is null. those are exactly the points MoonBit and Scala diverge on.
the rendered document has to describe the canonical json exactly, or a reader
told "integer" sends a number the host refuses. so s64 and u64 render as
strings with a pattern and the exact range as metadata, a duration as an object
with a required nanoseconds string, a quantity as {mantissa, scale, unit}, a
map as an array of pairs, and flags as a unique array of names. capability
types render writeOnly rather than as something a caller could construct.
named definitions become $defs entries referenced by an rfc 6901-escaped
pointer, which is what terminates a recursive type. union branches are inlined
under oneOf and narrowed by the rule that selects them, so the oneOf stays
decidable; that is the guest-side convention, and the host synthesising
per-branch definitions instead is equally valid for the same values.
an invocation's input is an ordered named parameter list rather than one value,
so PackParameters builds the record the wire carries and reports every missing
or unexpected argument together — a caller assembling arguments should learn
all of its mistakes at once. BoolParameter covers a tool flag, whose type the
wire fixes rather than names.
the two forms describe the same thing differently: the wit form is flat, one pool of nodes addressed by index, which is what a component-model record can carry; the core form is recursive, with references naming a definition by its string id, which is what the REST surface uses and what a go program can read without an index in hand. a reference is converted by name and NOT followed — the definition it names is converted in its own right — which is what makes a recursive type terminate. the walk is depth-bounded anyway, since a malformed graph can nest without end. Converted keeps the index side table. reflection selects sub-schemas by wit node index (a method's output, a tool's result, each parameter) and the recursive form has no indices at all, so callers stay on indices while everything downstream works in core terms. go switches are not exhaustive, so a wit case added later would fall through and convert to nothing. body() ends in an error naming the tag, and TestEveryWitBodyTagConverts walks every declared tag against a pinned count so adding one is a deliberate act rather than a silent loss. the sdk module now requires core; the guest SDK still uses its own schema package, which the next step swaps out.
flattening what was unflattened gives back the same value, which is the property the guest SDK will depend on when it moves onto core: nothing is lost crossing the boundary either way. wide integers in particular cross exactly, with no float in the middle. conversion is total rather than lazy, and ValueToWit releases on failure. a value is built in one pass, so if a later field fails the host-managed handles already taken for the earlier ones have no other owner — only the caller holding the whole tree can give them back. the four handle adapters are the only place a generated resource is bound to an interface, and they are build-tagged for that reason: binding one materializes its Drop method, which is a host call. off the wasm target a handle conversion says so rather than inventing one.
things found while building the SDK that are deliberately not fixed yet, each with what it is, why it was left, and what fixing it would take. the two that cost the most time are first: go type switches are not exhaustive, so a new WIT case converts to nothing silently; and natively-linked code may import the generated bindings but must never make a host import reachable, which trips in three distinct ways that each took a debugging session to find.
The guest SDK now takes its schema model from sdks/go/core instead of carrying its own copy, converting to and from the flat WIT form at the boundary through internal/witschema. sdks/go/golem/schema is deleted outright rather than kept as an alias. Reflection holds the converted graph beside the WIT one: ReflectedAgentType and ReflectedTool are built by converting constructors that record a malformed graph instead of raising, so discovery stays a lookup and the failure surfaces where the schema is used. Parameters() returns []core.Parameter, resolved through the converter's index side table, since the wire selects sub-schemas by node index and the shared model has none. Every component's go.mod has to name core as well: a replace in a dependency's go.mod is ignored, so a component that resolves the SDK from a checkout must point at core itself. The CLI derives the core path from the existing Go SDK path override, emits both require and replace lines from the component template, and reconciles both on every build.
An external bridge speaks the server's REST API, which carries values as
{"kind": "<case>", "value": <payload>} — the serde shape of the Rust
SchemaType and SchemaValue, and what openapi/golem-service.yaml is
generated from. That is a different thing from canonical JSON: it is
structural rather than schema-directed, a record is a positional list of
tagged nodes, a s64 is a number, and binary is an array of byte numbers.
Marshal*/Unmarshal* here, Pack*/Unpack* in json.go.
Values travel both ways. A type graph only arrives: a generated client
embeds the schema it was generated against, and nothing outside the server
invents one, so there is no marshaller for it to drift from.
The four host-managed handles — secret, quota token, permission card,
stream — are refused in both directions with an error that says why, since
an external caller has no host to hold them.
Both switches carry a pinned case count, so a kind added to the model
without a case here fails a test rather than landing in the "unsupported"
arm in production.
A new module generated bridge clients call an external Golem server through. It speaks the worker service's agent REST API over net/http and depends on nothing but the standard library and the shared schema model, so a program calling Golem from outside gets the wire format without the guest SDK's WebAssembly bindings. Two shapes cross that API and they are not interchangeable: method parameters and results are schema-native SchemaValue nodes, while a configuration override is a NormalizedJsonValue — ordinary canonical JSON, which the server applies the schema to itself. The tests assert the request bodies rather than the Go call shape, since the bodies are the contract. An agent captures its configuration when it is built, so repointing the ambient one does not move calls already in flight. Calls are synchronous and take a context: an external caller has no supervisor above it, so a failure is an ordinary (T, error), and the error says whether the server answered or was never reached. Streaming methods are not wired yet; they need the invocation-session protocol, which lands next.
core and bridge are separate Go modules, so the build-golem-go job's steps never reached them — the shared schema model and the external runtime were going in untested. Both now lint and test in the same job, with a .golangci.yml matching the guest SDK's so a rule holds the same way across the three modules. Neither builds for wasip1, so neither gets a wasm step. Running the linter turned up real findings, fixed here: a malformed numeric bound was decoded with the error discarded, so a bad bound silently became zero; the ten numeric type kinds are now one case that splits on the kind after decoding, rather than ten near-identical ones; and a response body was closed without checking.
Every other SDK documents itself in an AGENTS.md; Go was the only one without. The durable rules move there — the module layout, the build and test commands for all three modules, the native-linking rule and its three traps, why exhaustive switches carry a pinned case count, the difference between canonical JSON and the wire form, and the toolchain traps — and the root guide now points at it. The deferred work that was in FOLLOW-UPS.md is tracked in Linear instead (GOL-655 through GOL-660), which is where this repository's follow-ups live. The one item that was a real defect is already fixed: the test component's go.mod named an absolute path.
The first piece of the Go bridge generator: turning a schema name into a Go identifier. Go differs from the other targets in two ways that matter here. Its reserved set is not just the 25 keywords — the predeclared identifiers (any, error, string, len, new, nil, …) are ordinary names, so a parameter called `len` compiles and quietly takes the builtin away from every line emitted beside it. And exportedness is spelled in the casing, so repairing a name must not change its visibility: a leading digit is prefixed with N or n rather than with an underscore, which would silently unexport it. Go also drops the separator, so `first-name`, `first_name` and `firstName` all land on `FirstName` — names that stay distinct in every other target collide here, and the uniquing pass has more work to do.
Maps a schema type onto a generated Go type name, and decides which schema types need one. Go needs no reserved-runtime-name list, unlike the Scala and MoonBit generators. The runtime is always referenced through its package qualifier, so a generated Option is client.Option and cannot shadow the runtime's, and Go's predeclared names are all lowercase while every generated type name is exported. Named declarations are needed for exactly the schema types Go cannot spell inline: record, variant, enum, flags and union. A list is []T, a fixed list [N]T, an option *T, a result Result[Ok, Err], a tuple TupleN[…], and a map a slice of entries, since a schema map's keys need not be strings.
Option, Result, TupleN, Char, Text, Binary, Path, URL and Quantity now live in the shared core module, so a guest agent and an external bridge client generated from the same schema get the same Go types and a program can hold one domain package for both sides. golem's are aliases — golem.Option[T] *is* values.Option[T], not a copy. Go has no alias for a function, so the constructors forward in one line each. The codec plumbing could not follow as-is. Its interfaces are sealed by unexported methods, which means another module can assert against the interface but cannot call through it, and exporting the methods would put four reflect-flavoured methods on Option's public surface. core/values exposes them as package-level functions instead — OptionGet, ResultSetOk, QuantityParts and friends — each reporting whether the value was of the expected kind, so a caller classifies and unpacks in one step. A reader of golem.Option still sees IsSome, Get and Unwrap. No go.mod fan-out: values is a package inside the module components already require.
The writer collects imports as the body is written rather than taking a fixed header: an unused import is a compile error in Go, not a warning, so a generator that emits a type conditionally cannot know its imports up front. Standard library and module imports are grouped and sorted the way gofmt preserves, and the file carries Go's exact generated-code marker so tooling treats it as generated. The type mapping is the same in both modes, which is what putting the value vocabulary in core/values bought: only the call code differs. Every distinction the schema draws survives, because the item codecs are derived from the source schema and not from the Go type — a text is values.Text and not string, a char is values.Char and not rune, which is only an alias for int32. A fixed list keeps its length in the type. A map is a slice of MapEntry rather than a Go map, since a schema map is ordered and its keys need not be comparable; the Rust bridge spells the same thing Vec<(K, V)>.
The five schema types Go cannot spell inline. A record is a struct and flags are a struct of booleans; the other three needed a decision. An enum becomes a named string type with one constant per case, holding the schema's own case name. The wire carries an index, so the codec maps between the two — an integer-backed enum would make the codec marginally simpler and every log line and debugger session worse. A variant becomes a sealed interface with one struct per case. The alternative, one struct with a tag and a pointer per case, lets a caller build a value that names one case and carries another; with an interface a case *is* its payload. A union gets the same shape, since it is also a closed sum and only the wire's way of recognising a branch differs. The sealing method is named after its own type, so two variants in one package cannot satisfy each other's interface. Names are transliterated mechanically: order-id becomes OrderId, not Go's conventional OrderID. Applying Go's initialism list reads better in isolation but makes the mapping back to the schema name depend on knowing that list, and none of the other five generators do it.
I had the Go bridge spell map<K, V> as a slice of entries, reasoning from the Rust bridge's Vec<(K, V)> and from the schema model, whose MapType puts no constraint on the key. Schema well-formedness does constrain it: classify_map_key restricts a map key to bool, an integer, a float, char or string, and every one of those is comparable in Go. So a Go map always holds a well-formed schema map. Rust spells it Vec<(K, V)> for a reason that does not apply here — Rust floats are neither Hash nor Eq. This is the better shape twice over. It is what a Go programmer expects, and it is what the guest SDK's reflection codec already derives map<K, V> from, so a guest-mode client needs no generated map codec at all. Go randomizes map iteration, but the codec sorts keys before encoding, so what travels is deterministic. values.MapEntry existed only for the slice form and is gone. Also records the dependency policy the WebSocket client will land under: core and golem take no third-party dependency, bridge may, and what bridge takes must not be reachable from the other two.
CI's build-golem-go job has been failing on golangci-lint. Two doc comments did not start with the identifier they document, which ST1020/ST1021 catch, and seven symbols were reported unused. Five of the seven are used only by the wasip1 build, which a native lint run cannot see: a stream source's reader field, toolRPCErrorMessage, underlyingErrorToGo, toolErrorMessage and nextStdin's payload. Each now carries a //nolint:unused naming the file that uses it, matching the directive already in registry_test.go. A two-target codebase always looks partly unused to a single-target linter; saying where the use is beats widening the exclusion. The other two were genuinely dead and are removed: treeSink.valid, where only treeSource.valid is ever called, and errNoHostStreams, which the off-target stream constructors never returned.
# Conflicts: # docs/src/content/next/how-to-guides.mdx
The Go SDK could not express a payloadless variant case. compileVariant gave
every case Some(payload), so the SDK's own doc example — `type Cash struct{}`
— published `cash(record {})` rather than `cash`, and decoding a case that
arrived with no payload was an error.
That is not only a fidelity problem for Go agents. A Go guest calling a Rust
or TypeScript agent whose variant has a payloadless case could neither send
that case in the shape the callee declares nor decode it coming back — which
the generated guest bridge would have hit on the first such schema.
An empty struct now means "no payload": the schema carries None, the value
travels with None, and a payload arriving for such a case is reported as the
schema mismatch it is. The existing round-trip test passed throughout, since
the old encoding round-tripped with itself; the new tests check the schema
and the wire value, and fail on the old code.
A variant case type is its payload, which works until the payload cannot also carry the variant's marker method without becoming a different type. `type EventAt time.Time` is not a time.Time to the SDK and publishes a struct; `type EventNote golem.Text` publishes string, not text; a defined type over an Option or Result drops the methods the codec needs; and an interface — a nested variant — cannot have methods at all. Each silently publishes the wrong schema, which is exactly the erased-distinction failure the bridge contract warns generated code against. WrappedCase[T] and WrappedBranch[T] register a one-field struct whose field is the payload. The schema node is the field's own codec, so graph building and named-type references are untouched, and encoding and decoding go through the field. It is opt-in: no existing agent's schema changes. A wrapper without exactly one exported field is a definition error rather than a guess about which part is the payload. The generated Go bridge will use it for every payload-carrying case, which gives it one uniform shape regardless of what the payload is.
Checking the generated declarations against the guest SDK they have to register with found two mismatches. The SDK's DefineEnum accepts only an integer type, so an enum is now a uint32 with iota constants — the wire carries the case index anyway — and a String() method returns the schema's case name, so a log line still reads "in-transit" rather than 1. And a variant case wraps its payload in a Value field, which is the shape the new WrappedCase registers; that shape was already right, the SDK just had no way to say it. The output is also gofmt-canonical now, which is how a generated file is told apart from an edited one. Struct field types align within gofmt's runs — a doc comment line ends a run, which aligned_widths reproduces after checking gofmt's behaviour directly — one-line interface and struct bodies take gofmt's spacing, and a type is separated from its marker method. Checked by hand for now: a sample of every declaration kind compiles for wasip1 against the real SDK, gofmt -l is silent on it, and each value round-trips through EncodeTypedValue and DecodeTypedValue. The permanent version of that check belongs in the CLI integration suite, since unit tests must not spawn the Go toolchain.
A Go guest client sits directly on the guest SDK: the target is declared with golem.DeclareRemoteAgent, each method gets a typed descriptor, and a thin client struct gives every method a Go signature. The SDK's own reflective codec does every conversion, so the generator emits no codec at all. The sum types are registered exactly as a hand-written Go agent would — DefineVariant with WrappedCase, DefineEnum, DefineFlags, DefineUnion with WrappedBranch — which is only possible because of the SDK fixes that land alongside: payloadless variant cases and wrapped payloads. Each client is a module of its own, golem.local/bridge/<client-dir>. The .local suffix can never resolve on a module proxy, so a consumer that forgets the replace pointing at the generated directory fails loudly rather than fetching something else. As in every other SDK, the consuming component adds that require and replace itself; nothing reconciles it. The agent-level names — the Id, the client, the constructors and one input struct per method — share Go's single package namespace with the generated types, so they are reserved before any type is named. Checked by the Go toolchain itself in the integration suite, through ensure_go_toolchain rather than whatever go is on PATH: gofmt must be silent (failing with its own diff), go vet must pass for wasip1, and a native go test round-trips a value of every generated type through the SDK's codec — the only check that sees a malformed registration, which the SDK reports at run time. External mode is refused with a clear error until its codecs exist.
BridgeSdkTargetKind::supports now reports Go guest agent bridges, and gen_bridge dispatches them to the Go generator, so a Go component that lists another component's agent under dependencies.agents gets a typed guest client from golem build. External Go clients and guest tool clients are still not generated and still say so. The consumer names the generated module in its go.mod itself, as every other SDK's consumer does; the golem-call-another-agent-go skill documents it. A method without parameters now takes golem.Unit, as a hand-written Go agent's would, rather than an empty input struct of its own. bridge_sdk_support_matrix_matches_current_capabilities was already failing before this: it omitted Effect, which supports() reports. It now lists the supported languages per capability, which is also the shape Go's partial support needs. test_go_agent_guest_bridge_e2e builds a Rust provider and a Go consumer, deploys both, and invokes the consumer, which calls the provider through the generated client. It calls the provider directly once first so its component has finished compiling: a Go component compiles faster than the Rust provider, and a call that waits on that compilation outlasts the executor's idle window, where a Go caller — unlike a Rust one — is suspended and never resumed. That is a separate Go SDK bug, reproduced with a slow provider and a Rust control, and it is not in the bridge.
A Go agent whose MethodDef.Call waited longer than the executor's RPC idle window hung forever. The executor suspended it, the target finished, and the caller never returned — for any Go call, hand-written or generated, same- or cross-component. The Go guest bridge e2e hit it on a fresh server, where the Go consumer compiles faster than a Rust provider and its first call waits on the provider's compilation. Call used the synchronous invoke-and-await import. Every other SDK awaits agent RPC through async-invoke-and-await and future-invoke-result.get. The executor's debug log showed suspension and the scheduled wakeup both working: the caller was reconstructed, replayed to its incomplete invoke_and_await, re-entered it — and never came back. On the async path the identical sequence completes, so Call is now CallAsync followed by Get, and the reflected agent client awaits the same way. Call is therefore a yield point, like Future.Get; its doc and the parallel-workers skill say so. go_rpc_caller_resumes_after_suspending_mid_call reproduces it deterministically: the target blocks on a promise, the RPC idle window is two seconds, and the test completes the promise only once the caller is Suspended. It failed before this change and passes now, alongside an async variant and the rest of the Go executor suite (25/25). The bridge e2e no longer calls the provider first to dodge the bug. The executor defect on the synchronous path remains. Nothing uses that import for agent RPC any more, and fixing replay in the executor belongs in a change of its own.
An external client has no reflective codec — it does not import the guest
SDK — so a generated one converts between its Go types and schema values
explicitly. These are the pieces it composes: an encode/decode pair per leaf
kind, and per composite a pair that takes the element conversions as
arguments. A generated record encoder is then one expression per field, and
nesting is just nesting: a list of options of records is EncodeList over
EncodeOption over the record's own encoder.
Decoding checks the kind of every node and says what it expected, rather
than trusting a server value to match the schema the client was generated
from. A result arm that carries no value is spelled struct{} and takes a nil
conversion; a value arriving for it is an error. Go cannot abstract over an
array's length, so a fixed list travels as the array's slice and its length
is checked on the way back. The tuple pairs, identical but for the arity,
are generated for all seven arities the value vocabulary provides.
Tested through the actual wire form: every helper round-trips via
MarshalWireValue and UnmarshalWireValue, including the extremes of s64 and
u64.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.