Skip to content

Add end-to-end smoke tests for the CsWinRT NuGet package#2456

Open
Sergio0694 wants to merge 14 commits into
staging/winrt-runtime-ref-assemblyfrom
user/sergiopedri/smoke-tests
Open

Add end-to-end smoke tests for the CsWinRT NuGet package#2456
Sergio0694 wants to merge 14 commits into
staging/winrt-runtime-ref-assemblyfrom
user/sergiopedri/smoke-tests

Conversation

@Sergio0694

Copy link
Copy Markdown
Member

Summary

Adds minimal, isolated end-to-end smoke tests that consume the real Microsoft.Windows.CsWinRT NuGet package — a consumption app and an authoring component — to verify the package works correctly in a real production environment, independent of the repository's own build infrastructure.

Motivation

The CsWinRT 3.0 build pipeline is complex (reference projections, forwarder assemblies, several post-build generators, and the dual reference/implementation WinRT.Runtime assembly added in this branch). The existing unit and functional tests all build against the local build infrastructure (SimulateCsWinRTNugetReference, project references to the in-repo projections, etc.), so they never exercise the actual packaged ref/lib assemblies, the packaged generators, or the NuGet build targets the way a real customer consumes them. These smoke tests close that gap: they reference the packed NuGet package exactly like an external app or component author would, validating that the correct assemblies are referenced and that all generators run correctly end-to-end.

Changes

The smoke tests live under src/Tests/SmokeTests/ and are intentionally kept out of cswinrt.slnx (the package they consume only exists after the build packs it).

  • src/Tests/SmokeTests/Directory.Build.props / Directory.Build.targets: blank files that isolate the smoke tests from the repository build infrastructure, so they inherit nothing from src/Directory.Build.*.
  • src/Tests/SmokeTests/Directory.Packages.props: disables central package management for the smoke tests (and shadows the repository Directory.Packages.props), so each project pins the CsWinRT package version directly.
  • src/Tests/SmokeTests/.gitignore: ignores the default bin/obj output (these projects use the default output folders rather than the repo _build redirect).
  • src/Tests/SmokeTests/Consumption/: an Exe that calls JsonObject.Parse(...) then Stringify() from Windows.Data.Json, exercising the Windows SDK projection, the interop generator, and the WinRT.Runtime ref/impl assemblies.
  • src/Tests/SmokeTests/Authoring/: a CsWinRTComponent library exposing a minimal Greeter class, exercising WinMD generation, the reference projection, and the forwarder assembly.
  • src/Tests/SmokeTests/run-smoke-tests.ps1: builds and runs the consumption test (asserting a clean exit code), then builds the authoring test and verifies that the generated Authoring.winmd is a valid Windows Runtime metadata file defining Authoring.Greeter.
  • src/build.cmd: runs the smoke tests after the local nuget pack step (x64 only; skippable via cswinrt_run_smoke_tests=false).
  • build/AzurePipelineTemplates/CsWinRT-PublishToNuGet-Steps.yml: runs the smoke tests after the CI NuGet pack step, against the freshly packed package.

Both projects override RestoreSources to restore the package being tested from the local build output (or the CI pack output) and everything else — notably the preview Windows SDK ref pack — from public NuGet. They use the real .NET SDK targeting pack via a pinned WindowsSdkPackageVersion and the .1 Windows TFM revision, mirroring src/WinRT.Internal. The package version and source default to the local build.cmd x64 Release output and are overridden by the build or CI that produced the package.

Validation

Verified end-to-end against a real CsWinRT 3.0 package using the committed configuration: the consumption test builds and runs (round-tripping the JSON to {"a":42} with a clean exit), and the authoring component's generated Authoring.winmd is confirmed to define Authoring.Greeter. In CI the same runner executes against the freshly packed package.

Sergio0694 and others added 6 commits June 18, 2026 12:20
Adds blank Directory.Build.props/.targets and a Directory.Packages.props that disables central package management, so the smoke tests can consume the real CsWinRT NuGet package in full isolation from the repository build infrastructure. A .gitignore keeps their default bin/obj output out of source control.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A .NET app that parses a JSON object from Windows.Data.Json and round-trips it via Stringify, exercising the Windows SDK projection, the interop generator, and the WinRT.Runtime ref/impl assemblies against the real NuGet package.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A Windows Runtime component library exposing a minimal class, exercising WinMD generation, the reference projection, and the forwarder assembly against the real NuGet package.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds a runner that builds and runs the consumption test and builds the authoring test against a given package source and version, and wires it into build.cmd (after the pack step, on x64) and the PublishToNuGet CI steps (after packing).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ed type

After building the component, the runner now locates the generated Authoring.winmd and asserts it is a Windows Runtime metadata file that defines the Authoring.Greeter type, using a dependency-free inspection that works in any PowerShell host.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove explanatory comments and extra blank lines from SmokeTests projects and source files to reduce noise. Changes touch Authoring.csproj, Greeter.cs, Consumption.csproj, Program.cs, Directory.Build.props, Directory.Build.targets, and Directory.Packages.props. No functional code changes — only formatting and comment cleanup to keep the smoke test files minimal.
@Sergio0694 Sergio0694 added testing Related to testing infrastructure CsWinRT 3.0 labels Jun 18, 2026
@Sergio0694 Sergio0694 requested a review from manodasanW June 18, 2026 20:46
@Sergio0694 Sergio0694 force-pushed the user/sergiopedri/smoke-tests branch from 83a4142 to 42e58d7 Compare June 18, 2026 20:55
Moves the target framework, Windows SDK ref pack pin, NuGet restore sources, CsWinRT package version/source, and the CsWinRT package reference (all previously duplicated in both smoke test projects) into the shared SmokeTests Directory.Build.props. Each project now only declares what makes it different: the consumption app's output type and the authoring library's component flag.

Co-Authored-By: Copilot <223556219+Copilot@users.noreply.github.com>
@Sergio0694 Sergio0694 force-pushed the user/sergiopedri/smoke-tests branch from 42e58d7 to de07e31 Compare June 18, 2026 20:57
Sergio0694 and others added 7 commits June 18, 2026 14:52
Updates the testing skill and docs/structure.md to account for the new SmokeTests projects, and fixes other outdated test documentation: expands the SourceGenerator2Test analyzer test list (now CSWINRT2009-2017), corrects the AuthoringTest status and TFM, fixes the TestComponentCSharp IDL filename and the cswinrt.slnx reference, and lists the missing test projects in the repository structure doc. Also adds a smoke-tests verification step to the update-testing-instructions skill.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CI agents cannot reach public NuGet (api.nuget.org), which broke the smoke test restore. Restore the preview Windows SDK ref pack from the repository's standard CsWinRTDependencies feed instead (the same feed the rest of the repo uses, e.g. src/WinRT.Internal), keeping the local CsWinRT build output as the source for the package under test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The type map group types (WindowsRuntimeComWrappersTypeMapGroup, WindowsRuntimeMetadataTypeMapGroup, and DynamicInterfaceCastableImplementationTypeMapGroup) were stripped from the reference assembly as implementation-only types. However, they are referenced by code the CsWinRT source generator emits into consumer assemblies (the '[assembly: TypeMapAssemblyTarget<T>]' attributes that register assemblies for the interop type map infrastructure), which is compiled against the reference assembly. Stripping them broke consumers with CS0234.

Keep these three types in the reference assembly, and instead mark them '[Obsolete]' (with the new CSWINRT3002 diagnostic) and '[EditorBrowsable(Never)]', only in the reference assembly, to discourage direct use in user code. This mirrors the reference-assembly-only obsolete pattern already used for the parameterless 'WindowsRuntimeObject' constructor (CSWINRT3001). The marker is reference-assembly-only because the implementation assembly has internal '[assembly: TypeMap<T>]' usages that would otherwise trip the obsolete warning. The projection writer's generated-file header now also suppresses CSWINRT3002, since the generated projection code references these types. Adds the docs/diagnostics/cswinrt3002.md page.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Previously all smoke tests ran in a single CI step, so a failure didn't make it obvious which one failed. Following the unit test pattern in CsWinRT-Test-Steps.yml, each smoke test now runs as its own step with 'continueOnError' (an individual failure only marks the job as 'SucceededWithIssues'), and a final gate step turns any such issue into an actual failure. This makes it clear exactly which smoke test failed while still letting every test run and report.

The runner script gains a '-Test' parameter (Consumption, Authoring, or All) so CI can invoke a single test per step; local builds keep using the default 'All'.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…n assembly

The NuGet package ships WinRT.Runtime as both a reference assembly (ref/net10.0, with implementation-only types stripped) and an implementation assembly (lib/net10.0). The projection and interop generators compile and inspect generated implementation projection code that references those implementation-only types (e.g. WindowsRuntimeObjectReferenceValue, RestrictedErrorInfo, the marshallers), so they must use the implementation assembly rather than the reference assembly that MSBuild resolves for compilation.

Introduce a CsWinRTRuntimeImplementationAssemblyPath property (defaulting to the package lib layout) and swap WinRT.Runtime for it in each affected generator target. Repo-internal builds override the property to point at the locally built implementation assembly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The prior commit made the projection and interop generators unconditionally swap the WinRT.Runtime reference assembly for an implementation assembly, and overrode CsWinRTRuntimeImplementationAssemblyPath in src/Directory.Build.props to point at src/WinRT.Runtime2/bin/<config>/net10.0/WinRT.Runtime.dll. The official build rebuilds that bin output as the stripped reference assembly (CsWinRTBuildReferenceAssembly=true), so the override pointed the generators at the stripped assembly and broke repo builds.

The package always lays out both the reference assembly (ref/net10.0) and the implementation assembly (lib/net10.0), so the generators can simply assume the implementation assembly is present and swap to it; if it were ever missing, the smoke tests (and any real consumer) would fail loudly. The swap is now controlled by CsWinRTSwapRuntimeReferenceAssembly (default true). This repo references WinRT.Runtime directly as its implementation assembly, so it opts out via src/Directory.Build.props. The fragile bin-pointing override is removed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ssembly (CSWINRT3003)

The authoring smoke test failed to build because the CsWinRT source generator emits '[assembly: WindowsRuntimeComponentAssembly]' and '[assembly: WindowsRuntimeComponentAssemblyExportsType(...)]' into the component (in 'ManagedExports.g.cs'), but both attributes were stripped from the 'WinRT.Runtime' reference assembly (they were marked as implementation-only files), so the component compilation could not resolve them.

Keep both attributes in the reference assembly and mark them, only there, with a reference-assembly-only '[Obsolete]' (new diagnostic CSWINRT3003) plus '[EditorBrowsable(Never)]', exactly like the type map group types (CSWINRT3002). The generated code that applies them already suppresses warnings, so normal builds are unaffected, while direct use in user code is discouraged. Add a 'docs/diagnostics/cswinrt3003.md' page describing the diagnostic.

Also fix a related leak: the 'ReferenceVftbl' struct in the projection writer's 'InspectableVftbl' baseline declared its 'GetTrustLevel' slot as 'TrustLevel*', referencing the implementation-only 'Windows.Foundation.TrustLevel' enum. The slot is layout-only (the vtable is populated by copying 'IInspectableImpl.Vtable' and only 'get_Value' is set), so type it as 'int*' to match 'IInspectableVftbl.GetTrustLevel' and keep 'TrustLevel' out of the generated component projection.

Co-Authored-By: Copilot <223556219+Copilot@users.noreply.github.com>
@Sergio0694 Sergio0694 force-pushed the user/sergiopedri/smoke-tests branch from 85022f7 to 9b524b3 Compare June 20, 2026 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CsWinRT 3.0 testing Related to testing infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant