Skip to content

Migrate on-device APK tests from NUnit to MSTest #12721

Description

@jonathanpeppers

Android framework version

net11.0-android (Preview)

Affected platform version

.NET 11 / MSTest 4.4.0

Description

MSTest 4.4.0 supports NativeAOT: https://www.nuget.org/packages/MSTest/4.4.0

Migrate all on-device APK tests from NUnit to MSTest. This scope includes:

  • tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj, including tests loaded from Java.Interop-Tests.NET.csproj
  • tests/CodeGen-Binding/Xamarin.Android.JcwGen-Tests/Xamarin.Android.JcwGen-Tests.csproj
  • The shared Android instrumentation runner in tests/TestRunner.Core/
  • Related CI configuration, test documentation, and package/version configuration

The current shared runner directly uses NUnit APIs for discovery, filtering, execution, result counting, and TRX generation. The migration must preserve:

  • Android instrumentation startup and the Microsoft.Android.Run result protocol
  • Multiple test assembly support
  • Include/exclude category behavior from instrumentation arguments and RuntimeHostConfigurationOption
  • Runtime-specific exclusions for Mono, CoreCLR, trimmable type maps, LLVM, and NativeAOT
  • Fully-qualified test-name exclusions for external test assemblies
  • Execution on the instrumentation/JNI-attached thread where required
  • dotnet test, MTP, and TRX reporting behavior

The NativeAOT configuration currently contains NUnit-specific reflection/rooting workarounds (IlcGenerateCompleteTypeMetadata and NativeAOT.rd.xml, including roots for NUnit async adapters). Remove any workarounds that become unnecessary after the migration.

This issue does not include host-side NUnit test projects such as MSBuildDeviceIntegration or the Android SDK/build-task unit tests.

Acceptance criteria

  • The two on-device APK test applications and TestRunner.Core no longer reference or use NUnit.
  • Existing on-device tests are migrated to MSTest without losing coverage; any intentional semantic differences are documented.
  • Existing include/exclude filters and runtime-specific skip behavior continue to work.
  • The following CI lanes continue to build, install, run, and publish results:
    • Mono.Android.NET_Tests-Debug
    • Mono.Android.NET_Tests-Release
    • Mono.Android.NET_Tests-NoAab
    • Mono.Android.NET_Tests-Mono
    • Mono.Android.NET_Tests-CoreCLRTrimmable
    • Mono.Android.NET_Tests-NativeAOT
    • Xamarin.Android.JcwGen_Tests
  • Local documented dotnet-local build/test commands continue to work and produce TRX results.
  • NUnit-specific NativeAOT metadata/rooting workarounds are removed when they are no longer required.

Steps to Reproduce

  1. Inspect tests/TestRunner.Core/TestInstrumentation.cs; it uses NUnitTestAssemblyRunner and NUnit filters/results directly.
  2. Inspect the two APK test projects; both reference NUnit and use NUnit test attributes/assertions.
  3. Build the NativeAOT lane with -p:TestsFlavor=NativeAOT -p:PublishAot=true; the project currently requires NUnit-specific runtime directives.

/cc @Evangelink

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-triageIssues that need to be assigned.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions