Add end-to-end smoke tests for the CsWinRT NuGet package#2456
Open
Sergio0694 wants to merge 14 commits into
Open
Add end-to-end smoke tests for the CsWinRT NuGet package#2456Sergio0694 wants to merge 14 commits into
Sergio0694 wants to merge 14 commits into
Conversation
Adds blank Directory.Build.props/.targets and a Directory.Packages.props that disables central package management, so the smoke tests can consume the real CsWinRT NuGet package in full isolation from the repository build infrastructure. A .gitignore keeps their default bin/obj output out of source control. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A .NET app that parses a JSON object from Windows.Data.Json and round-trips it via Stringify, exercising the Windows SDK projection, the interop generator, and the WinRT.Runtime ref/impl assemblies against the real NuGet package. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A Windows Runtime component library exposing a minimal class, exercising WinMD generation, the reference projection, and the forwarder assembly against the real NuGet package. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds a runner that builds and runs the consumption test and builds the authoring test against a given package source and version, and wires it into build.cmd (after the pack step, on x64) and the PublishToNuGet CI steps (after packing). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ed type After building the component, the runner now locates the generated Authoring.winmd and asserts it is a Windows Runtime metadata file that defines the Authoring.Greeter type, using a dependency-free inspection that works in any PowerShell host. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove explanatory comments and extra blank lines from SmokeTests projects and source files to reduce noise. Changes touch Authoring.csproj, Greeter.cs, Consumption.csproj, Program.cs, Directory.Build.props, Directory.Build.targets, and Directory.Packages.props. No functional code changes — only formatting and comment cleanup to keep the smoke test files minimal.
83a4142 to
42e58d7
Compare
Moves the target framework, Windows SDK ref pack pin, NuGet restore sources, CsWinRT package version/source, and the CsWinRT package reference (all previously duplicated in both smoke test projects) into the shared SmokeTests Directory.Build.props. Each project now only declares what makes it different: the consumption app's output type and the authoring library's component flag. Co-Authored-By: Copilot <223556219+Copilot@users.noreply.github.com>
42e58d7 to
de07e31
Compare
Updates the testing skill and docs/structure.md to account for the new SmokeTests projects, and fixes other outdated test documentation: expands the SourceGenerator2Test analyzer test list (now CSWINRT2009-2017), corrects the AuthoringTest status and TFM, fixes the TestComponentCSharp IDL filename and the cswinrt.slnx reference, and lists the missing test projects in the repository structure doc. Also adds a smoke-tests verification step to the update-testing-instructions skill. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CI agents cannot reach public NuGet (api.nuget.org), which broke the smoke test restore. Restore the preview Windows SDK ref pack from the repository's standard CsWinRTDependencies feed instead (the same feed the rest of the repo uses, e.g. src/WinRT.Internal), keeping the local CsWinRT build output as the source for the package under test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The type map group types (WindowsRuntimeComWrappersTypeMapGroup, WindowsRuntimeMetadataTypeMapGroup, and DynamicInterfaceCastableImplementationTypeMapGroup) were stripped from the reference assembly as implementation-only types. However, they are referenced by code the CsWinRT source generator emits into consumer assemblies (the '[assembly: TypeMapAssemblyTarget<T>]' attributes that register assemblies for the interop type map infrastructure), which is compiled against the reference assembly. Stripping them broke consumers with CS0234. Keep these three types in the reference assembly, and instead mark them '[Obsolete]' (with the new CSWINRT3002 diagnostic) and '[EditorBrowsable(Never)]', only in the reference assembly, to discourage direct use in user code. This mirrors the reference-assembly-only obsolete pattern already used for the parameterless 'WindowsRuntimeObject' constructor (CSWINRT3001). The marker is reference-assembly-only because the implementation assembly has internal '[assembly: TypeMap<T>]' usages that would otherwise trip the obsolete warning. The projection writer's generated-file header now also suppresses CSWINRT3002, since the generated projection code references these types. Adds the docs/diagnostics/cswinrt3002.md page. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Previously all smoke tests ran in a single CI step, so a failure didn't make it obvious which one failed. Following the unit test pattern in CsWinRT-Test-Steps.yml, each smoke test now runs as its own step with 'continueOnError' (an individual failure only marks the job as 'SucceededWithIssues'), and a final gate step turns any such issue into an actual failure. This makes it clear exactly which smoke test failed while still letting every test run and report. The runner script gains a '-Test' parameter (Consumption, Authoring, or All) so CI can invoke a single test per step; local builds keep using the default 'All'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…n assembly The NuGet package ships WinRT.Runtime as both a reference assembly (ref/net10.0, with implementation-only types stripped) and an implementation assembly (lib/net10.0). The projection and interop generators compile and inspect generated implementation projection code that references those implementation-only types (e.g. WindowsRuntimeObjectReferenceValue, RestrictedErrorInfo, the marshallers), so they must use the implementation assembly rather than the reference assembly that MSBuild resolves for compilation. Introduce a CsWinRTRuntimeImplementationAssemblyPath property (defaulting to the package lib layout) and swap WinRT.Runtime for it in each affected generator target. Repo-internal builds override the property to point at the locally built implementation assembly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The prior commit made the projection and interop generators unconditionally swap the WinRT.Runtime reference assembly for an implementation assembly, and overrode CsWinRTRuntimeImplementationAssemblyPath in src/Directory.Build.props to point at src/WinRT.Runtime2/bin/<config>/net10.0/WinRT.Runtime.dll. The official build rebuilds that bin output as the stripped reference assembly (CsWinRTBuildReferenceAssembly=true), so the override pointed the generators at the stripped assembly and broke repo builds. The package always lays out both the reference assembly (ref/net10.0) and the implementation assembly (lib/net10.0), so the generators can simply assume the implementation assembly is present and swap to it; if it were ever missing, the smoke tests (and any real consumer) would fail loudly. The swap is now controlled by CsWinRTSwapRuntimeReferenceAssembly (default true). This repo references WinRT.Runtime directly as its implementation assembly, so it opts out via src/Directory.Build.props. The fragile bin-pointing override is removed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ssembly (CSWINRT3003) The authoring smoke test failed to build because the CsWinRT source generator emits '[assembly: WindowsRuntimeComponentAssembly]' and '[assembly: WindowsRuntimeComponentAssemblyExportsType(...)]' into the component (in 'ManagedExports.g.cs'), but both attributes were stripped from the 'WinRT.Runtime' reference assembly (they were marked as implementation-only files), so the component compilation could not resolve them. Keep both attributes in the reference assembly and mark them, only there, with a reference-assembly-only '[Obsolete]' (new diagnostic CSWINRT3003) plus '[EditorBrowsable(Never)]', exactly like the type map group types (CSWINRT3002). The generated code that applies them already suppresses warnings, so normal builds are unaffected, while direct use in user code is discouraged. Add a 'docs/diagnostics/cswinrt3003.md' page describing the diagnostic. Also fix a related leak: the 'ReferenceVftbl' struct in the projection writer's 'InspectableVftbl' baseline declared its 'GetTrustLevel' slot as 'TrustLevel*', referencing the implementation-only 'Windows.Foundation.TrustLevel' enum. The slot is layout-only (the vtable is populated by copying 'IInspectableImpl.Vtable' and only 'get_Value' is set), so type it as 'int*' to match 'IInspectableVftbl.GetTrustLevel' and keep 'TrustLevel' out of the generated component projection. Co-Authored-By: Copilot <223556219+Copilot@users.noreply.github.com>
85022f7 to
9b524b3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds minimal, isolated end-to-end smoke tests that consume the real
Microsoft.Windows.CsWinRTNuGet package — a consumption app and an authoring component — to verify the package works correctly in a real production environment, independent of the repository's own build infrastructure.Motivation
The CsWinRT 3.0 build pipeline is complex (reference projections, forwarder assemblies, several post-build generators, and the dual reference/implementation
WinRT.Runtimeassembly added in this branch). The existing unit and functional tests all build against the local build infrastructure (SimulateCsWinRTNugetReference, project references to the in-repo projections, etc.), so they never exercise the actual packagedref/libassemblies, the packaged generators, or the NuGet build targets the way a real customer consumes them. These smoke tests close that gap: they reference the packed NuGet package exactly like an external app or component author would, validating that the correct assemblies are referenced and that all generators run correctly end-to-end.Changes
The smoke tests live under
src/Tests/SmokeTests/and are intentionally kept out ofcswinrt.slnx(the package they consume only exists after the build packs it).src/Tests/SmokeTests/Directory.Build.props/Directory.Build.targets: blank files that isolate the smoke tests from the repository build infrastructure, so they inherit nothing fromsrc/Directory.Build.*.src/Tests/SmokeTests/Directory.Packages.props: disables central package management for the smoke tests (and shadows the repositoryDirectory.Packages.props), so each project pins the CsWinRT package version directly.src/Tests/SmokeTests/.gitignore: ignores the defaultbin/objoutput (these projects use the default output folders rather than the repo_buildredirect).src/Tests/SmokeTests/Consumption/: anExethat callsJsonObject.Parse(...)thenStringify()fromWindows.Data.Json, exercising the Windows SDK projection, the interop generator, and theWinRT.Runtimeref/impl assemblies.src/Tests/SmokeTests/Authoring/: aCsWinRTComponentlibrary exposing a minimalGreeterclass, exercising WinMD generation, the reference projection, and the forwarder assembly.src/Tests/SmokeTests/run-smoke-tests.ps1: builds and runs the consumption test (asserting a clean exit code), then builds the authoring test and verifies that the generatedAuthoring.winmdis a valid Windows Runtime metadata file definingAuthoring.Greeter.src/build.cmd: runs the smoke tests after the localnuget packstep (x64 only; skippable viacswinrt_run_smoke_tests=false).build/AzurePipelineTemplates/CsWinRT-PublishToNuGet-Steps.yml: runs the smoke tests after the CINuGet packstep, against the freshly packed package.Both projects override
RestoreSourcesto restore the package being tested from the local build output (or the CI pack output) and everything else — notably the preview Windows SDK ref pack — from public NuGet. They use the real .NET SDK targeting pack via a pinnedWindowsSdkPackageVersionand the.1Windows TFM revision, mirroringsrc/WinRT.Internal. The package version and source default to the localbuild.cmd x64 Releaseoutput and are overridden by the build or CI that produced the package.Validation
Verified end-to-end against a real CsWinRT 3.0 package using the committed configuration: the consumption test builds and runs (round-tripping the JSON to
{"a":42}with a clean exit), and the authoring component's generatedAuthoring.winmdis confirmed to defineAuthoring.Greeter. In CI the same runner executes against the freshly packed package.