Skip to content

fix(codegen): preserve serializer manifest target metadata for NativeAOT - #11374

Open
ReubenBond wants to merge 25 commits into
dotnet:mainfrom
ReubenBond:rb-nativeaot-manifest-metadata
Open

ReubenBond wants to merge 25 commits into
dotnet:mainfrom
ReubenBond:rb-nativeaot-manifest-metadata

Conversation

@ReubenBond

@ReubenBond ReubenBond commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Fixes #11368.

The serializer manifest registers implementation types and rediscovers their target contracts by enumerating interfaces at startup. Under NativeAOT, preserving an implementation's constructors and interfaces does not reliably preserve the generic target definitions needed by that reflection path. The default manifest consequently fails resolving System.ValueTuple`8 during serializer initialization.

The generator now emits explicit implementation/target registrations for field and base codecs, value serializers, deep and base copiers, activators, and converters. Target-taking TypeManifestOptions.Add* APIs preserve implementation constructors and target interface metadata. CodecProvider and TypeConverter consume these mappings directly. Converter activation uses the registered surrogate. Existing single-type registrations and collection membership/order/removal retain their compatibility behavior.

Declarative SerializationType descriptions preserve fixed and reordered generic arguments and structural array shapes. Lookup selects exact closed entries first, then matching named generic entries, then array and bare-parameter patterns in reverse registration order. Implementation closure uses the winning registration's parameter bindings. Accessible targets use typeof; inaccessible targets resolve a generated constant assembly-qualified name as part of their registration.

SerializationType.Array is a structural target-matching pattern. Executable array types require source-known closed descriptors such as SerializationType.Create(typeof(MyValue[])) or explicit closed converter/surrogate registrations. The generator emits concrete descriptors for fully known array nodes, including those nested inside otherwise generic contracts. Resolving an array pattern as an executable type produces NotSupportedException with registration guidance on both JIT and NativeAOT. This intentionally narrows the new, unreleased recipe API: metadata Type construction alone can succeed while its native representation is unavailable. Runtime array construction and its scoped AOT suppression are removed. Typed closed code supplies the native representations used by supported registrations.

The Metadata scenario uses the shared harness and discovery matrix merged in #11367. It initializes the default metadata consumers, resolves tuple definitions of arity 1–8, verifies inaccessible and inherited target metadata, and round-trips primitives, tuples, and a custom array codec. It also exercises registration priority, reordered parameter binding, generated and explicit concrete-array surrogate descriptors, immutable-array implementation closure, and uniform rejection of unresolved executable array recipes. Its descriptor retains legacy diagnostics while rejecting IL diagnostics from generated manifest initialization and explicit registration APIs.

The generated public API surface is verified with GenAPI. Coverage includes interface-inspection traps, legacy/explicit precedence and removal, generic shape binding, selected-registration closure, source-known array emission, closed converter witnesses, validation, and registration annotations. Normal Release package validation remains enabled.

The independent nested-generic NativeAOT Type.Namespace inconsistency is tracked in dotnet/runtime#135106. The private-target probe here verifies registration and full-name resolution.

Microsoft Reviewers: Open in CodeFlow

Copilot AI balanced review requested due to automatic review settings October 1, 2026 16:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The NativeAOT smoke program does not compile due to parameter shadowing, and its CI warning configuration does not enforce the stated metadata-root diagnostics.

Review effort: Balanced
Findings: 1 High severity · 1 Medium severity

Open (2)
What changed in this PR

Adds source-generated metadata roots so NativeAOT serializer initialization can resolve manifest target types and their interfaces.

Changes:

  • Collects and deterministically emits serializer metadata dependencies.
  • Adds generator tests, snapshots, documentation, and a NativeAOT smoke scenario.
  • Adds the smoke scenario to static-analysis CI.
File Description
.github/​workflows/​analyzer-audit.yml Adds NativeAOT manifest smoke job.
src/​Orleans.CodeGenerator/​MetadataGenerator.cs Emits annotated metadata roots.
src/​Orleans.CodeGenerator/​MetadataSourceOutputGenerator.cs Passes metadata-support capability.
src/​Orleans.CodeGenerator/​Model/​ReferenceAssemblyModelExtractor.cs Extracts registered codec dependencies.
src/​Orleans.CodeGenerator/​Model/​RegisteredCodecModel.cs Stores codec metadata dependencies.
src/​Orleans.CodeGenerator/​Model/​SerializableTypeModel.cs Stores serializable metadata dependencies.
src/​Orleans.CodeGenerator/​Model/​SerializableTypeModelExtractor.cs Collects serializable dependencies.
src/​Orleans.CodeGenerator/​Model/​TypeMetadataDependencyCollector.cs Traverses metadata dependency types.
src/​Orleans.CodeGenerator/​OrleansSourceGenerator.cs Detects annotation availability.
src/​Orleans.Serialization/​README.md Documents generated metadata preservation.
test/​Orleans.CodeGenerator.Tests/​ManifestMetadataTests.cs Tests collection, emission, and invalidation.
test/​Orleans.NativeAotSmoke/​Metadata.cs Adds metadata and serialization checks.
test/​Orleans.NativeAotSmoke/​NativeAotEnvironment.cs Verifies execution under NativeAOT.
test/​Orleans.NativeAotSmoke/​Orleans.NativeAotSmoke.csproj Defines the reusable smoke executable.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestAlias.verified.cs Updates alias snapshot roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestBasicClass.verified.cs Updates basic-class roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestBasicClassWithAnnotatedFields.verified.cs Updates annotated-field roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestBasicClassWithDifferentAccessModifiers.verified.cs Updates accessibility roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestBasicClassWithInheritance.verified.cs Updates inheritance roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestBasicClassWithoutNamespace.verified.cs Updates global-namespace roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestBasicGrain.verified.cs Updates grain and invokable roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestBasicStruct.verified.cs Updates struct roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestClassesWithGeneratedActivatorConstructorAnnotation.verified.cs Updates activator-constructor roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestClassNestedTypes.verified.cs Updates nested-type roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestClassPrimitiveTypes.verified.cs Updates primitive-member roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestClassPrimitiveTypesUsingFullName.verified.cs Updates full-name primitive roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestClassReferenceProperties.verified.cs Updates reference-property roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestClassWithGenerateMethodSerializersAnnotation.verified.cs Updates method-invokable roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestClassWithGenerateSerializerAnnotation.verified.cs Updates serializer and enum roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestClassWithInterfaceConstructorParameter.verified.cs Updates constructor-target roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestClassWithNoPublicConstructors.verified.cs Updates non-public-constructor roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestClassWithOptionalConstructorParameters.verified.cs Updates optional-constructor roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestClassWithParameterizedConstructor.verified.cs Updates parameterized-constructor roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestCompoundTypeAlias.verified.cs Updates compound-alias roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestGenericClass.verified.cs Updates generic-class roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestGenericClassWithConstructorParameters.verified.cs Updates generic-constructor roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestGrainComplexGrain.verified.cs Updates complex-grain roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestGrainMethodAnnotatedWithInvokableBaseType.verified.cs Updates custom-invokable roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestGrainMethodAnnotatedWithResponseTimeout.verified.cs Updates response-timeout roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestGrainWithDifferentKeyTypes.verified.cs Updates keyed-grain roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestGrainWithMultipleInterfaces.verified.cs Updates multi-interface invokable roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestRecords.verified.cs Updates record roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestRecordsWithParameterIdAttributes.verified.cs Updates parameter-ID record roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestWithOmitDefaultMemberValuesAnnotation.verified.cs Updates omitted-default roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestWithSuppressReferenceTrackingAttribute.verified.cs Updates reference-tracking roots.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestWithUseActivatorAnnotation.verified.cs Updates custom-activator roots.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread test/Orleans.NativeAotSmoke/Metadata.cs Outdated
Comment thread .github/workflows/analyzer-audit.yml Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The new Metadata smoke scenario is not currently discovered or executed by the linked CI runner.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Resolved since last review (2)

Comment thread test/Orleans.NativeAotSmoke/Metadata.smoke.json
Copilot AI balanced review requested due to automatic review settings October 1, 2026 17:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The new Metadata NativeAOT scenario and diagnostic gates are not wired into any current runner or workflow.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Code coverage

Metric Pull request Current main Variance
Lines 82.94% (116,187 / 140,081) 82.97% (115,618 / 139,353) -0.0250 pp
Branches 72.34% (33,958 / 46,944) 72.37% (33,638 / 46,479) -0.0352 pp

Report-only conclusion: regressed.

The current-main baseline is commit d114d7522a and uses the same reviewed coverage matrix.

Coverage combines every CI test matrix job, including providers, CodeGen, .NET 8/10, Linux, Windows, and macOS, using canonical physical source and branch identities.

The comparison remains report-only while normal line and branch variance is calibrated.

Coverage details

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The implementation addresses the reported NativeAOT failure with focused unit, snapshot, and native smoke coverage.

Review effort: Balanced
Findings: None

Resolved since last review (1)

@ReubenBond
ReubenBond force-pushed the rb-nativeaot-manifest-metadata branch from 4baabac to e17ebf9 Compare October 2, 2026 02:56
Copilot AI balanced review requested due to automatic review settings October 2, 2026 02:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The metadata collection and emission paths are deterministic and comprehensively covered by generator and NativeAOT regression tests.

Review effort: Balanced
Findings: None

Copilot AI balanced review requested due to automatic review settings October 2, 2026 03:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

NativeAOT linker behavior and source-generator caching warrant final human validation despite comprehensive regression coverage.

Review effort: Balanced
Findings: None

Copilot AI balanced review requested due to automatic review settings October 2, 2026 23:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Structural array targets are not selectable, and shared serializer contract storage breaks field-codec collection removal semantics.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Separate field-codec and serializer contract mappings

src/​Orleans.Serialization/​Configuration/​TypeManifestOptions.cs:379

Field-codec and serializer registrations share SerializerContracts even though their membership sets are independently mutable. If the same implementation is also present in Serializers (for example via AddBaseCodec), removing it from FieldCodecs does not remove its field-codec mapping: the field contract is consumed again while iterating SerializerTypes. Preserve the originating collection for each contract (or use separate contract maps) so existing collection removal semantics remain effective.

Copilot AI balanced review requested due to automatic review settings October 2, 2026 23:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Plain open-target registrations can incorrectly fail when the same implementation also has a described target pattern.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Resolved since last review (1)

Comment thread src/Orleans.Serialization/Serializers/CodecProvider.cs Outdated
Copilot AI balanced review requested due to automatic review settings October 2, 2026 23:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Contract removal, global registration ordering, and constrained generic pattern selection remain incorrect.

Review effort: Balanced
Findings: None

Resolved since last review (1)
Previously missed (3)

In code that hasn't changed since last review

Medium severity Removed registrations can be resurrected by later re-registration

src/​Orleans.Serialization/​Configuration/​TypeManifestOptions.cs:568

Removing an explicitly registered implementation from the public HashSet only makes these contracts temporarily inactive: the dictionary entries remain. If the same type is later re-added (for example, register A for T1, remove A from Copiers, then register A for T2), both T1 and T2 are consumed, so the removed registration is resurrected. Track contract lifetime with collection removal/re-addition (or clear stale contracts on a fresh membership generation) so mutable-collection removal remains effective.

Medium severity Per-implementation ordering breaks reverse registration precedence

src/​Orleans.Serialization/​Configuration/​TypeManifestOptions.cs:577

Storing contracts in a list per implementation loses global registration order because consumption iterates each implementation once in the implementation HashSet. For example, registering pattern A, then overlapping pattern B, then another overlapping pattern for A causes all of A's entries to be consumed before B, so B wins even though A was registered last. This contradicts the documented reverse-registration-order selection. Preserve a category-wide ordered registration sequence rather than grouping order solely by implementation.

Medium severity Generic constraint violations cause candidate selection exceptions

src/​Orleans.Serialization/​Serializers/​CodecProvider.cs:805

Pattern matching does not account for generic constraints before selecting a candidate. A valid registration such as ReferenceCopier<T> : IDeepCopier<T> where T : class produces a bare-parameter pattern which also matches int; CloseImplementation then calls MakeGenericType(typeof(int)) and throws instead of trying the next candidate or the normal shallow-copy fallback. Reject candidates whose bound arguments do not satisfy the implementation constraints during selection.

Copilot AI balanced review requested due to automatic review settings October 3, 2026 02:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

It introduces cross-cutting public API, source-generation, runtime resolution, and NativeAOT behavior that warrants final human validation.

Review effort: Balanced
Findings: None

Copilot AI balanced review requested due to automatic review settings October 3, 2026 03:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Interleaved registrations can lose their documented global priority and select the wrong implementation.

Review effort: Balanced
Findings: 1 High severity

Open (1)

Comment thread src/Orleans.Serialization/Serializers/CodecProvider.cs Outdated
Copilot AI balanced review requested due to automatic review settings October 3, 2026 04:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Structural recipe deduplication and mixed legacy/described discovery can alter priority or omit valid contracts.

Review effort: Balanced
Findings: 2 Medium severity

Open (2)
Resolved since last review (1)

Comment thread src/Orleans.Serialization/Configuration/TypeManifestOptions.cs
Comment thread src/Orleans.Serialization/Serializers/CodecProvider.cs Outdated
Copilot AI balanced review requested due to automatic review settings October 3, 2026 06:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Constraint-incompatible generic candidates can be selected and throw instead of allowing an applicable fallback registration.

Review effort: Balanced
Findings: None

Resolved since last review (2)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Validate generic constraints before selecting generated registrations

src/​Orleans.Serialization/​Serializers/​CodecProvider.cs:854

A structurally matching candidate is selected without checking whether its bound arguments satisfy the implementation's generic constraints. For example, [RegisterCopier] StructCopier<T> : IDeepCopier<T> where T : struct emits a bare Parameter(0) pattern; a request for string selects it here, then MakeGenericType(typeof(string)) throws instead of considering an earlier applicable candidate. Validate closure while scanning candidates (and continue when constraints reject the binding) so constrained generated registrations do not hijack unsupported targets.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NativeAOT serializer startup fails resolving ValueTuple`8 in TypeConverter metadata

2 participants