Swift GitHub Actions Workflows Unification.

[rnro]

Swift GitHub Actions Workflows Unification.

This PR aims to bring the best aspects of the swiftlang GitHub Actions workflows (this repository) and the SwiftNIO (apple/swift-nio) workflows together. It does this chiefly through a new re-usable workflow which sits beside the existing workflow, and a hierarchy of supporting infrastructure.

Some key design features:

1. GitHub-release-based versioning
2. No skipped jobs
3. Scripts are cloned not curled
4. Custom matrix builds
5. Minimum Swift version detection

The unified design provides one workflow which can be adopted to run a range of common test and build configurations on a variety of platforms against multiple Swift versions. This workflow sits atop a "matrix generation" layer which takes inputs and produces a canonical work definition (in YAML or JSON). That definition is then expanded into one job per entry, each executing one slice of the work. Benchmarking is a further consumer of the same layer rather than part of the recommended test workflow.

The design is intended to be layered, so that adopters may use the whole stack, or customize portions beyond the level which the workflow allows. Adopters may:

• Use the package_test.yml workflow for the full suite of conveniences
• Use a custom workflow (perhaps with custom inputs) which calls toolchain_matrix.yml for the matrix and execute_matrix.yml to run its own command across it
• Generate the work definition in YAML or JSON by some other means, or hard-code it into a workflow, and pass it to execute_matrix.yml

The matrix generation layer is a Swift script, generate-matrix.swift, run by the matrix job on ubuntu-latest, which comes with the Swift, jq and yq it needs. It reads the workflow's inputs from the environment, applies the defaults and the minimum version filter, and writes the work definition to standard output. In SwiftNIO it was a bash script, which executes faster, but the unified version is much more complex, so Swift seemed like the more maintainable option. A test package under tests/MatrixGeneratorValidator covers the inputs, their defaults and failure modes.

As part of this work an effort was also made to reduce execution times, defaulting to running natively on the runner rather than inside a container where possible. The unified workflows install the toolchain with swiftly on the runner rather than pulling a full Docker image. Containers remain available for a package that needs a particular distribution or system dependencies, either through linux_use_docker or by naming a distribution in linux_os. swiftly was previously used only for macOS toolchains, so using it on Linux is new to both upstreams.

Migrating from this repository's existing workflow

The new workflow is currently available alongside existing infrastructure so a repository can control when they perform the migration.

When adopting the new workflow the most visible change is that the *_exclude_swift_versions inputs are gone, replaced with explicit supported Swift version lists and minimum-version detection which drops anything the package manifest cannot support. A repository that was only excluding old versions based on its Swift tools version can simplify its config.

Before:

  tests:
    name: Test
    uses: swiftlang/github-workflows/.github/workflows/swift_package_test.yml@0.0.14
    with:
      enable_cross_pr_testing: true
      linux_exclude_swift_versions: "[{\"swift_version\": \"6.1\"}, {\"swift_version\": \"6.2\"}]"
      windows_exclude_swift_versions: "[{\"swift_version\": \"6.1\"}, {\"swift_version\": \"6.2\"}]"

After:

  tests:
    name: Test
    uses: swiftlang/github-workflows/.github/workflows/package_test.yml@0.0.16
    with:
      enable_cross_pr_testing: true

The new architecture means that jobs which are not intended to be run will no longer appear as skipped:

Before:

After:

Migrating from the SwiftNIO workflows

Changing from the SwiftNIO workflows means a change in spelling but no reduction in flexibility. Per-version inputs are now supplied in a map which reduces churn and synchronization problems when a new Swift version is released.

Before:

  unit-tests:
    uses: apple/swift-nio/.github/workflows/unit_tests.yml@main
    with:
      linux_6_1_arguments_override: "-Xswiftc -warnings-as-errors --explicit-target-dependency-import-check error"
      linux_6_2_arguments_override: "-Xswiftc -warnings-as-errors --explicit-target-dependency-import-check error"
      linux_6_3_arguments_override: "-Xswiftc -warnings-as-errors --explicit-target-dependency-import-check error"
      linux_nightly_next_arguments_override: "-Xswiftc -warnings-as-errors --explicit-target-dependency-import-check error"
      linux_nightly_main_arguments_override: "--explicit-target-dependency-import-check error"
  benchmarks:
    uses: apple/swift-nio/.github/workflows/benchmarks.yml@main
  cxx-interop:
    uses: apple/swift-nio/.github/workflows/cxx_interop.yml@main
  construct-integration-test-matrix:   # hand-written: checkout, then build a matrix in a script
    runs-on: ubuntu-latest
    steps: [...]
  integration-tests:
    needs: construct-integration-test-matrix
    uses: apple/swift-nio/.github/workflows/swift_test_matrix.yml@main

After:

  tests:                                     # unit tests and Cxx interop
    uses: swiftlang/github-workflows/.github/workflows/package_test.yml@0.0.16
    with:
      linux_version_overrides: |
        6.1: -Xswiftc -warnings-as-errors --explicit-target-dependency-import-check error
        6.2: -Xswiftc -warnings-as-errors --explicit-target-dependency-import-check error
        6.3: -Xswiftc -warnings-as-errors --explicit-target-dependency-import-check error
        nightly-release: -Xswiftc -warnings-as-errors --explicit-target-dependency-import-check error
        nightly-main: --explicit-target-dependency-import-check error
  benchmarks:
    uses: swiftlang/github-workflows/.github/workflows/benchmarks.yml@0.0.16
  integration-test-toolchains:               # replaces the hand-written matrix job
    uses: swiftlang/github-workflows/.github/workflows/toolchain_matrix.yml@0.0.16
  integration-tests:
    needs: integration-test-toolchains
    uses: swiftlang/github-workflows/.github/workflows/execute_matrix.yml@0.0.16

Functionality new to this repository

The unification also brings some new functionality to this repository. Some inspired by SwiftNIO, some entirely new:

• Simulator testing. iOS was build-only; xcodebuild now builds and tests on simulators, and tvOS, watchOS and visionOS join it. Mac Catalyst is new to both upstreams.
• Benchmarks across the matrix. benchmarks.yml measures performance with Swift Package Benchmark, checking committed thresholds under Thresholds/<swift-version>/ on every platform and Swift version in the matrix, failing the job and printing a diff when one moves. This differs from the existing performance_test.yml, which compares a PR against its merge base in one container and posts the table as a PR comment. The existing workflow tells you what a change did to performance; the new one holds a package to a standard it has written down.
• A C++ interoperability check. A release-configuration build needs no dedicated input: it is a second labeled command, so Release-Build Linux Swift 6.3 runs alongside Debug-Test Linux Swift 6.3 from one call.
• A Semantic Version PR label check (pull_request_label.yml), which fails a PR that carries no SemVer label.
• Importable Swift version matrix, toolchain_matrix.yml, provides the current supported Swift version matrix for custom workflows to utilize. The output is plain YAML, so it can be filtered or extended with yq before it is executed.
• Per-version overrides, so one Swift version can take extra arguments, or a different command, without splitting the matrix by hand.

Known limitations

• A pinned tag still does not pin the scripts. This is behavior inherited from the existing swiftlang workflows. swift_package_test.yml and soundness.yml check this repository out with no ref:, so they resolve to its default branch, and one variant hardcodes ref: main; this means a caller on a tag e.g. @0.0.14 has always run that tag's YAML against the default branch's scripts. The unified workflows inherit that default. Resolving this should be possible but would require stamping versions as part of the release process.