diff --git a/CHANGELOG.md b/CHANGELOG.md index 958ef82..9b43cf4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,10 @@ These are conditional source/runtime compatibility changes from `3.0.0`, even th - **Null structural elements (#26):** built-in structural collection mapping now preserves null for nullable element targets. For nonnullable element targets under `Throw`, it raises `InvalidOperationException` before constructing the target object or reading source members, rather than dereferencing null in the helper. Higher-precedence nullable converters still receive null. Recheck null handling, side effects, and exception expectations. - **Enum exception metadata (#27):** nullable-root by-name enum failures now identify the declared input parameter instead of a generated capture name. Update assertions or error handling that inspect `ParamName` or related exception text. +### Performance + +- Cycle tracking stores the first active source inline and creates reference-identity spill storage only for a second simultaneous source. Spill storage is retained for later siblings. This reduces measured depth-1 tracked allocations from 232 to 56 bytes per mapping on Windows x64; measured depths 2, 16 and 128 retain their previous allocations. Timing comparisons showed unstable controls and deeper slowdown signals, so this change does not establish an overall throughput improvement or release performance acceptance. Public cycle behavior and statically generated mapping remain unchanged. + ### Fixed - Issue #51: set mappings use the destination default comparer after custom element conversion, including same-type converters, preventing incorrect collapse of distinct mapped values. Identity-mapped sets retain compatible source comparers. [#51](https://github.com/PrimordialCode/Mammoth.LiteMapper/issues/51) diff --git a/PERFORMANCE_OPTIMIZATION_NOTES.md b/PERFORMANCE_OPTIMIZATION_NOTES.md index a50902d..765ec5a 100644 --- a/PERFORMANCE_OPTIMIZATION_NOTES.md +++ b/PERFORMANCE_OPTIMIZATION_NOTES.md @@ -2,6 +2,25 @@ These notes are non-authoritative follow-up candidates. `SPECIFICATION.md` remains the product contract. +## Compact cycle tracker allocation evidence (2026-10-13) + +The private generated tracker has one object slot. Its own instance denotes empty state, an ordinary first source is stored inline (including null), and a private sealed HashSet subtype denotes spill storage. Reserved sentinel/type-tag sources escape into a fresh set. Promotion populates a local set before publishing it; failed duplicate entry leaves occupancy intact. Spill sets use reference identity and remain available after unwind. Existing Enter-before-try/finally-Exit calls and lazy exception paths are unchanged. No public API, mapping reflection, runtime code generation or specification semantics change. + +Source comparison: merged develop `221459f715f2c4a864212ea6838eebacdf067157`, the earlier inline-first prototype, and this compact design. On SPECTRE_GRD (Windows 11 x64, i7-1260P, SDK 10.0.401/runtime 10.0.12), a separate typed allocation probe used 5,000 warmups and five rounds of 1,000 mappings per depth; every round validated mapped shape, cycle failures and tracker reuse. Source construction and assertions were outside measurement. + +| Chain depth | Develop bytes/op | Inline-first prototype bytes/op | Compact bytes/op | +|---:|---:|---:|---:| +| 1 | 232 | 72 | 56 | +| 2 | 264 | 280 | 264 | +| 16 | 1,744 | 1,760 | 1,744 | +| 128 | 15,528 | 15,544 | 15,528 | + +These are total managed allocations per mapping, not peak memory. The fieldless private spill subtype matches the original HashSet allocation footprint; direct constructor probes measured 24 bytes for the tracker and 64 for either empty set type. Generator tests verify deferred storage and reuse relative to runtime allocation controls. + +A separate six-pass BenchmarkDotNet experiment used develop/inline-first/compact/compact/inline-first/develop, 16 cases per pass, two launches, five warmups and ten measured iterations. It retained default CLR policies, affinity mask 4, a 45-second cooldown and a start-CPU check of at most 10%. `DontEnforcePowerPlan` (Guid.Empty) kept Balanced unchanged. Actual iterations were at least 164.905 ms. Timing controls drifted by up to 94.14%; compact depth-1 mappings were faster than develop in all four cross-pass comparisons, while deeper cases produced slowdown signals. Compact versus inline-first shallow ordering reversed between passes. The repository gate blocked all four compact/develop comparisons. This is allocation evidence with inconclusive throughput attribution, not a general speed win or release acceptance. The earlier 3.0.0/develop shallow tracked slowdown remains a separate historical concern; measurements from other PCs are not pooled. + +Raw JSON, normalized results, all twelve gate reports, launch statistics, environment, hashes, source patch and test evidence are preserved in `LiteMapper-compact-state-SPECTRE_GRD-20261013.zip` (SHA256 `1D78057FA23247A5A3637132A9B9A5FFBE0A4E445C4C7F4A91588EDA6B995F59`). No diagnostic harness, binaries or benchmark thresholds are part of this shipping change. Final-head CI and stable same-binary controls remain necessary before a release performance decision. + ## Issue #32: lazy cycle-path allocation evidence (2026-10-09) - Baseline: unchanged develop `7b19def74753df90cb3770c7410ff5d7a3951244`. Candidate: issue #32's private inline-leaf/immutable-link diagnostic path representation. diff --git a/STATUS.md b/STATUS.md index f1f0d22..c56f306 100644 --- a/STATUS.md +++ b/STATUS.md @@ -1,5 +1,12 @@ # Mammoth.LiteMapper Status +## Compact cycle tracker production checkpoint (2026-10-13) + +- Scope: promote the reviewed compact tracker from isolated experiments onto develop `221459f715f2c4a864212ea6838eebacdf067157`. One private object slot keeps the first source inline, safely escapes sentinel/type-tag collisions, promotes atomically to identity-based spill storage and retains that storage after unwind. Public API, cycle exception metadata, static mapping, finally cleanup and specification semantics remain unchanged. +- Tests: the prototype's red-first compact state cases yielded 15 failures and four passing controls on the unchanged emitter; allocation regressions also failed before implementation. Production-branch Release build has zero warnings/errors; 1,056 generator, five runtime and one integration tests pass with zero failures/skips. Independent seven-file review found no blocking findings. Local package validation is running with benchmark smoke and Native AOT explicitly deferred; exact-head Windows/Linux, Roslyn, package, trim and required Native AOT CI evidence will be recorded in the draft PR. Prototype results are not substituted for final-head checks. +- Performance: Windows x64 allocation probes show 56 versus 232 bytes per mapping at depth 1; depths 2/16/128 remain 264/1,744/15,528. Six-pass timing controls drifted up to 94.14%, with deeper slowdown signals and all four compact/develop gates blocked. No overall throughput or release acceptance claim. See PERFORMANCE_OPTIMIZATION_NOTES.md for controls, limits and the preserved evidence archive. +- Boundary: user authorized a scoped commit, push and draft PR. Diagnostic harnesses/raw binaries stay outside the shipping diff; benchmark gates remain unchanged. No merge, tag, release, publication or persistent machine-setting changes. The installed AgentStack CLI expects the obsolete flat asset layout and cannot operate on this repository's module layout; direct Git/GitHub CLI is used for the authorized PR, without issue creation or claim/comment mutations. + ## Consumer skill review correction (2026-10-12) - User-directed correction: the consumer skill and mapping reference describe the current library release contract directly. Previous versions appear only in breaking-change migration; normal recipes no longer frame supported behavior as upcoming or unreleased. The package command uses the consumer's configured feed and does not invent an available version. All skill document links and references stay inside its distributable folder; essential guidance is bundled instead of redirecting to repository documentation. This supersedes the initial skill framing in the checkpoint below. diff --git a/src/Mammoth.LiteMapper.Generator/LiteMapperGenerator.cs b/src/Mammoth.LiteMapper.Generator/LiteMapperGenerator.cs index 596ca19..2537d6d 100644 --- a/src/Mammoth.LiteMapper.Generator/LiteMapperGenerator.cs +++ b/src/Mammoth.LiteMapper.Generator/LiteMapperGenerator.cs @@ -4649,19 +4649,55 @@ private static void AppendCycleTracker(StringBuilder builder) builder.AppendLine(); builder.AppendLine(" private sealed class __LiteMapperCycleTracker"); builder.AppendLine(" {"); - builder.AppendLine(" private readonly global::System.Collections.Generic.HashSet _active = new global::System.Collections.Generic.HashSet(__ReferenceIdentityComparer.Instance);"); + builder.AppendLine(" private object? _state;"); + builder.AppendLine(" public __LiteMapperCycleTracker() { _state = this; }"); builder.AppendLine(); builder.AppendLine(" public void Enter(object source, global::System.Type sourceType, global::System.Type destinationType, string mappingMethod, Path memberPath)"); builder.AppendLine(" {"); - builder.AppendLine(" if (!_active.Add(source))"); + builder.AppendLine(" if (_state is __ActiveReferences active)"); builder.AppendLine(" {"); - builder.AppendLine(" throw new global::Mammoth.LiteMapper.LiteMapperCycleException(sourceType, destinationType, mappingMethod, memberPath.ToString());"); + builder.AppendLine(" if (active.Add(source)) return;"); builder.AppendLine(" }"); + builder.AppendLine(" else if (object.ReferenceEquals(_state, this))"); + builder.AppendLine(" {"); + builder.AppendLine(" if (object.ReferenceEquals(source, this) || source is __ActiveReferences)"); + builder.AppendLine(" {"); + builder.AppendLine(" var escaped = new __ActiveReferences();"); + builder.AppendLine(" escaped.Add(source);"); + builder.AppendLine(" _state = escaped;"); + builder.AppendLine(" }"); + builder.AppendLine(" else"); + builder.AppendLine(" {"); + builder.AppendLine(" _state = source;"); + builder.AppendLine(" }"); + builder.AppendLine(" return;"); + builder.AppendLine(" }"); + builder.AppendLine(" else if (!object.ReferenceEquals(_state, source))"); + builder.AppendLine(" {"); + builder.AppendLine(" var promoted = new __ActiveReferences();"); + builder.AppendLine(" promoted.Add(_state!);"); + builder.AppendLine(" promoted.Add(source);"); + builder.AppendLine(" _state = promoted;"); + builder.AppendLine(" return;"); + builder.AppendLine(" }"); + builder.AppendLine(" throw new global::Mammoth.LiteMapper.LiteMapperCycleException(sourceType, destinationType, mappingMethod, memberPath.ToString());"); builder.AppendLine(" }"); builder.AppendLine(); builder.AppendLine(" public void Exit(object source)"); builder.AppendLine(" {"); - builder.AppendLine(" _active.Remove(source);"); + builder.AppendLine(" if (_state is __ActiveReferences active)"); + builder.AppendLine(" {"); + builder.AppendLine(" active.Remove(source);"); + builder.AppendLine(" }"); + builder.AppendLine(" else if (object.ReferenceEquals(_state, source))"); + builder.AppendLine(" {"); + builder.AppendLine(" _state = this;"); + builder.AppendLine(" }"); + builder.AppendLine(" }"); + builder.AppendLine(); + builder.AppendLine(" private sealed class __ActiveReferences : global::System.Collections.Generic.HashSet"); + builder.AppendLine(" {"); + builder.AppendLine(" public __ActiveReferences() : base(__ReferenceIdentityComparer.Instance) { }"); builder.AppendLine(" }"); builder.AppendLine(); builder.AppendLine(" public readonly struct Path"); diff --git a/tests/Mammoth.LiteMapper.Generator.Tests/CompactCycleTrackerTests.cs b/tests/Mammoth.LiteMapper.Generator.Tests/CompactCycleTrackerTests.cs new file mode 100644 index 0000000..644703d --- /dev/null +++ b/tests/Mammoth.LiteMapper.Generator.Tests/CompactCycleTrackerTests.cs @@ -0,0 +1,555 @@ +using System; +using System.IO; +using System.Linq; +using System.Reflection; +using Mammoth.LiteMapper; +using Microsoft.CodeAnalysis; +using Microsoft.CodeAnalysis.CSharp; +using Microsoft.CodeAnalysis.CSharp.Syntax; +using Microsoft.VisualStudio.TestTools.UnitTesting; + +namespace Mammoth.LiteMapper.Generator.Tests +{ + [TestClass] + public sealed class CompactCycleTrackerTests + { + [TestMethod] + [DataRow(false)] + [DataRow(true)] + public void FirstReferenceAndDuplicateNeedNoSpillEvenWhenFirstIsNull(bool nullSource) + { + var state = new TrackerState(Compile(Generate(ChainFixture(true)))); + var first = nullSource ? null : new object(); + state.Enter(first); + state.AssertFirst(first); + Assert.IsNull(state.Spill, "One active reference needs no hash set."); + state.AssertDuplicate(first); + state.AssertFirst(first); + Assert.IsNull(state.Spill, "Failed Enter occurs outside try and must not create spill storage."); + state.Exit(first); + state.AssertEmpty(); + } + + [TestMethod] + [DataRow(false, false)] + [DataRow(true, false)] + [DataRow(false, true)] + public void SpillContainsEveryActiveReferenceAndReusesStorageAcrossSiblingsAndTraversals(bool nullFirst, bool nullSecond) + { + var state = new TrackerState(Compile(Generate(ChainFixture(true)))); + var first = nullFirst ? null : new object(); + var second = nullSecond ? null : new object(); + state.Enter(first); + state.Enter(second); + var spill = state.Spill!; + Assert.AreEqual(2, spill.Count, "Promotion retains both the first and second active references."); + Assert.IsTrue(spill.Contains(first!)); + Assert.IsTrue(spill.Contains(second!)); + state.AssertDuplicate(first); + state.AssertDuplicate(second); + state.AssertFirst(first); + Assert.AreSame(spill, state.Spill); + Assert.AreEqual(2, spill.Count, "Failed duplicate Enter must preserve active state."); + Assert.IsTrue(spill.Contains(second!)); + state.Exit(second); + Assert.AreEqual(1, spill.Count); + state.AssertFirst(first); + var sibling = new object(); + state.Enter(sibling); + Assert.AreSame(spill, state.Spill, "Sibling traversal must reuse emptied spill storage."); + Assert.AreEqual(2, spill.Count); + state.Exit(sibling); + state.Exit(first); + state.AssertEmpty(); + state.Enter(second); + state.AssertFirst(second); + Assert.AreSame(spill, state.Spill); + Assert.AreEqual(1, spill.Count, "Once promoted, every new traversal reuses the retained spill storage without demotion."); + state.Exit(second); + state.AssertEmpty(); + } + + [TestMethod] + public void UserEqualityAndHashOverridesNeverParticipateInInlineOrSpillTracking() + { + var source = ChainFixture(true).Replace("public static Node Chain(int depth)", @" + public override bool Equals(object? other) => throw new InvalidOperationException(""Equals must not run""); + public override int GetHashCode() => throw new InvalidOperationException(""GetHashCode must not run""); + public static Node Chain(int depth)"); + var assembly = Compile(Generate(source)); + var state = new TrackerState(assembly); + var first = Activator.CreateInstance(assembly.GetType("Node")!)!; + var second = Activator.CreateInstance(assembly.GetType("Node")!)!; + state.Enter(first); + state.Enter(second); + state.AssertDuplicate(first); + state.AssertDuplicate(second); + state.Exit(second); + state.Exit(first); + state.AssertEmpty(); + var chain = assembly.GetType("Node")!.GetMethod("Chain")!.Invoke(null, new object[] { 16 }); + Assert.IsNotNull(assembly.GetType("Mapper")!.GetMethod("Map")!.Invoke(null, new[] { chain })); + } + + [TestMethod] + [DataRow(1, false)] + [DataRow(4, false)] + [DataRow(1, true)] + [DataRow(4, true)] + public void ShallowAndDeepGetterOrConverterFailuresReleaseAllEnteredReferences(int depth, bool converter) + { + var source = ChainFixture(true).Replace(" public int Value { get; set; }", @" + public static readonly InvalidOperationException Failure = new InvalidOperationException(""mapping failed""); + public bool ThrowGetter; + private int _value; + public int Value { get => ThrowGetter ? throw Failure : _value; set => _value = value; }"); + if (converter) source = source.Replace("public static partial NodeDto Map(Node source);", @" +public static partial NodeDto Map(Node source); +[MappingConverter] private static int Convert(int value) => value < 0 ? throw Node.Failure : value;"); + var assembly = Compile(Generate(source)); + var state = new TrackerState(assembly); + var helper = assembly.GetType("Mapper")!.GetMethods(BindingFlags.NonPublic | BindingFlags.Static) + .Single(method => method.Name.StartsWith("MapNested_", StringComparison.Ordinal)); + var node = assembly.GetType("Node")!; + var root = node.GetMethod("Chain")!.Invoke(null, new object[] { depth })!; + var tail = root; + while (node.GetProperty("Child")!.GetValue(tail) is object child) tail = child; + if (converter) node.GetProperty("Value")!.SetValue(tail, -1); + else node.GetField("ThrowGetter")!.SetValue(tail, true); + var failure = Assert.ThrowsExactly(() => helper.Invoke(null, new[] { root, state.Tracker, state.EmptyPath, "Map" })); + Assert.AreSame(node.GetField("Failure")!.GetValue(null), failure.InnerException); + state.AssertEmpty(); + var nullFailure = Assert.ThrowsExactly(() => helper.Invoke(null, new[] { null, state.Tracker, state.EmptyPath, "Map" })); + Assert.IsInstanceOfType(nullFailure.InnerException, "GuardNonNullSource defaults to false; source! is not a runtime guard."); + state.AssertEmpty(); + if (converter) node.GetProperty("Value")!.SetValue(tail, 0); + else node.GetField("ThrowGetter")!.SetValue(tail, false); + Assert.IsNotNull(helper.Invoke(null, new[] { root, state.Tracker, state.EmptyPath, "Map" })); + state.AssertEmpty(); + } + + [TestMethod] + [DataRow(1)] + [DataRow(4)] + public void ReentrantPublicCallsOnTheSameSourceOwnIndependentTrackers(int depth) + { + var source = ChainFixture(true).Replace("public static partial NodeDto Map(Node source);", @" +public static partial NodeDto Map(Node source); +public static Node Root = null!; +private static bool _inside; +[MappingConverter] private static int Convert(int value) +{ + if (!_inside) + { + _inside = true; + try { if (Map(Root).Value != 0) throw new InvalidOperationException(""reentrant result""); } + finally { _inside = false; } + } + return value; +}"); + var assembly = Compile(Generate(source)); + var root = assembly.GetType("Node")!.GetMethod("Chain")!.Invoke(null, new object[] { depth }); + var mapper = assembly.GetType("Mapper")!; + mapper.GetField("Root")!.SetValue(null, root); + Assert.IsNotNull(mapper.GetMethod("Map")!.Invoke(null, new[] { root })); + } + + [TestMethod] + [DataRow(1)] + [DataRow(4)] + public void ClassStructClassCyclesTrackReferencesWithoutBoxingAndKeepFullPaths(int depth) + { + var result = Generate(@" +using System; +using System.Linq; +using Mammoth.LiteMapper; +[LiteMapper(ReferenceHandling = ReferenceHandling.ThrowOnCycle)] +public static partial class Mapper { public static partial NodeDto Map(Node source); } +public sealed class Node { public Link Link { get; set; } } +public struct Link { public Node? Next { get; set; } } +public sealed class NodeDto { public LinkDto Link { get; set; } } +public struct LinkDto { public NodeDto? Next { get; set; } } +public static class Probe +{ + public static bool Run(int depth) + { + var root = new Node(); var tail = root; + for (var i = 1; i < depth; i++) { var next = new Node(); tail.Link = new Link { Next = next }; tail = next; } + if (Mapper.Map(root) == null) return false; + tail.Link = new Link { Next = root }; + try { Mapper.Map(root); return false; } + catch (LiteMapperCycleException error) + { + if (error.SourceType != typeof(Node) || error.DestinationType != typeof(NodeDto) || + error.MappingMethod != ""Map"" || error.MemberPath != string.Join(""."", Enumerable.Repeat(""Link.Next"", depth))) return false; + } + tail.Link = default; + return Mapper.Map(root) != null; + } +} +"); + Assert.AreEqual(true, Compile(result).GetType("Probe")!.GetMethod("Run")!.Invoke(null, new object[] { depth })); + var methods = CSharpSyntaxTree.ParseText(result.Source).GetRoot().DescendantNodes().OfType(); + foreach (var method in methods.Where(method => method.ParameterList.Parameters.FirstOrDefault()?.Type?.ToString() == "Link")) + Assert.IsFalse(method.ToString().Contains(".Enter(", StringComparison.Ordinal), "Struct bridges must forward the tracker without boxing."); + foreach (var method in methods.Where(method => method.Body?.ToString().Contains(".Enter(", StringComparison.Ordinal) == true)) + { + var body = method.Body!; + var attempt = body.Statements.OfType().Single(); + Assert.IsTrue(body.Statements.TakeWhile(statement => statement != attempt).Any(statement => statement.ToString().Contains(".Enter(", StringComparison.Ordinal)), + "Enter must run before try so a failed duplicate cannot trigger Exit."); + Assert.IsNotNull(attempt.Finally); + StringAssert.Contains(attempt.Finally.ToString(), ".Exit("); + } + } + + [TestMethod] + public void TrackerSelfSourceEscapesTheEmptySentinelWithoutLosingOccupancy() + { + var state = new TrackerState(Compile(Generate(ChainFixture(true)))); + state.Enter(state.Tracker); + var spill = state.Spill!; + Assert.IsNotNull(spill, "The reserved empty sentinel must escape into tagged spill storage as an ordinary element."); + Assert.AreEqual(1, spill.Count); + Assert.IsTrue(spill.Contains(state.Tracker)); + state.AssertDuplicate(state.Tracker); + state.Exit(state.Tracker); + state.AssertEmpty(); + Assert.AreSame(spill, state.Spill); + state.Enter(null); + Assert.AreEqual(1, spill.Count); + Assert.IsTrue(spill.Contains(null!)); + state.AssertDuplicate(null); + state.Exit(null); + state.AssertEmpty(); + } + + [TestMethod] + [DataRow(false)] + [DataRow(true)] + public void NullAndTrackerSelfRemainDistinctSimultaneouslyActiveSources(bool selfFirst) + { + var state = new TrackerState(Compile(Generate(ChainFixture(true)))); + var first = selfFirst ? state.Tracker : null; + var second = selfFirst ? null : state.Tracker; + state.Enter(first); + state.AssertFirst(first); + var initialSpill = state.Spill; + state.Enter(second); + var spill = state.Spill!; + Assert.IsNotNull(spill); + if (selfFirst) Assert.AreSame(initialSpill, spill, "The second source must reuse the self-escape spill."); + else Assert.IsNull(initialSpill, "A null first source stays raw until promotion."); + Assert.AreEqual(2, spill.Count, "Null and the reserved self reference must occupy distinct active entries."); + Assert.IsTrue(spill.Contains(null!)); + Assert.IsTrue(spill.Contains(state.Tracker)); + state.AssertDuplicate(null); + state.AssertDuplicate(state.Tracker); + state.Exit(second); + Assert.AreEqual(1, spill.Count); + Assert.IsFalse(spill.Contains(second!)); + Assert.IsTrue(spill.Contains(first!)); + Assert.AreSame(spill, state.Spill, "Exiting one reserved source must preserve the other and retain tagged storage."); + state.Exit(first); + state.AssertEmpty(); + Assert.AreSame(spill, state.Spill); + } + + [TestMethod] + public void PrivateSpillSourceEscapesItsReservedStateTagAsAnOrdinaryReference() + { + var state = new TrackerState(Compile(Generate(ChainFixture(true)))); + var reserved = (System.Collections.Generic.HashSet)Activator.CreateInstance(state.SpillType, nonPublic: true)!; + state.Enter(reserved); + var spill = state.Spill!; + Assert.IsNotNull(spill); + Assert.AreNotSame(reserved, spill, "The source must not become active storage merely because it has the private spill type."); + Assert.AreEqual(0, reserved.Count, "Tracking must not mutate a reserved-tag source collection."); + Assert.AreEqual(1, spill.Count); + Assert.IsTrue(spill.Contains(reserved)); + state.AssertDuplicate(reserved); + state.Exit(reserved); + state.AssertEmpty(); + Assert.AreSame(spill, state.Spill); + } + + [TestMethod] + public void CurrentPrivateSpillStorageCanItselfBeAnActiveSource() + { + var state = new TrackerState(Compile(Generate(ChainFixture(true)))); + var first = new object(); var second = new object(); + state.Enter(first); + state.Enter(second); + var spill = state.Spill!; + state.Enter(spill); + Assert.AreSame(spill, state.Spill, "An already promoted tracker must track its storage reference as an ordinary element."); + Assert.AreEqual(3, spill.Count); + Assert.IsTrue(spill.Contains(spill)); + state.AssertDuplicate(spill); + state.Exit(spill); + Assert.AreEqual(2, spill.Count); + state.Exit(second); + state.Exit(first); + state.AssertEmpty(); + Assert.AreSame(spill, state.Spill); + } + + [TestMethod] + [DataRow(false)] + [DataRow(true)] + public void UserHashSetsAndThrowingUserSubclassesRemainRawSourcesUntilPromotion(bool subclass) + { + var state = new TrackerState(Compile(Generate(ChainFixture(true)))); + System.Collections.Generic.HashSet source = subclass ? new ThrowingSet() : new System.Collections.Generic.HashSet(); + state.Enter(source); + Assert.AreSame(source, state.State, "Only the private sealed spill subtype is tagged; user sets are ordinary raw references."); + Assert.IsNull(state.Spill); + state.AssertDuplicate(source); + var second = new object(); + state.Enter(second); + var spill = state.Spill!; + Assert.AreEqual(2, spill.Count); + Assert.IsTrue(spill.Contains(source)); + Assert.IsTrue(spill.Contains(second)); + Assert.AreEqual(0, source.Count, "Tracking must not mutate a user HashSet."); + state.AssertDuplicate(source); + state.Exit(second); + state.Exit(source); + state.AssertEmpty(); + } + + private sealed class ThrowingSet : System.Collections.Generic.HashSet + { + public override bool Equals(object? other) => throw new InvalidOperationException("User set equality must not run."); + public override int GetHashCode() => throw new InvalidOperationException("User set hashing must not run."); + } + + [TestMethod] + public void CompactLayoutAndGeneratedTraversalAllocationsMatchOriginalOncePromoted() + { + var result = Generate(ChainFixture(true) + @" +public static class Probe +{ + public static long[] Measure() + { + var depths = new[] { 1, 2, 16, 128 }; + var values = new long[depths.Length]; + for (var d = 0; d < depths.Length; d++) + { + var root = Node.Chain(depths[d]); + for (var warmup = 0; warmup < 80; warmup++) GC.KeepAlive(Mapper.Map(root)); + var before = GC.GetAllocatedBytesForCurrentThread(); + for (var i = 0; i < 200; i++) GC.KeepAlive(Mapper.Map(root)); + values[d] = (GC.GetAllocatedBytesForCurrentThread() - before) / 200; + } + return values; + } +} +"); + var wide = Generate(ChainFixture(true).Replace("NodeDto Map(Node source)", "NodeDto[] Map(Node[] source)") + @" +public static class Probe +{ + public static long Measure() + { + var shared = Node.Chain(2); + var roots = Enumerable.Repeat(shared, 256).ToArray(); + for (var warmup = 0; warmup < 80; warmup++) GC.KeepAlive(Mapper.Map(roots)); + var before = GC.GetAllocatedBytesForCurrentThread(); + for (var i = 0; i < 200; i++) GC.KeepAlive(Mapper.Map(roots)); + return (GC.GetAllocatedBytesForCurrentThread() - before) / 200; + } +} +"); + var originalBytes = (long[])Compile(OriginalStyleTracker(result)).GetType("Probe")!.GetMethod("Measure")!.Invoke(null, null)!; + var compactBytes = (long[])Compile(result).GetType("Probe")!.GetMethod("Measure")!.Invoke(null, null)!; + var originalWide = (long)Compile(OriginalStyleTracker(wide)).GetType("Probe")!.GetMethod("Measure")!.Invoke(null, null)!; + var compactWide = (long)Compile(wide).GetType("Probe")!.GetMethod("Measure")!.Invoke(null, null)!; + Console.WriteLine("Original-style bytes/op at 1/2/16/128/wide: " + string.Join(", ", originalBytes) + ", " + originalWide); + Console.WriteLine("Compact bytes/op at 1/2/16/128/wide: " + string.Join(", ", compactBytes) + ", " + compactWide); + Assert.IsTrue(compactBytes[0] < originalBytes[0], "One ordinary active reference must avoid all spill allocations."); + for (var d = 1; d < compactBytes.Length; d++) Assert.AreEqual(originalBytes[d], compactBytes[d], + "Promotion must match the original active-reference allocation."); + Assert.AreEqual(originalWide, compactWide, "Wide shared siblings must reuse spill storage without allocating a tracker or set per sibling."); + + // Instrument only the test compilation so the allocation loop can construct the private spill directly. + var layoutTree = result.Compilation.SyntaxTrees.Single(tree => tree.GetText().ToString() == result.Source); + var layoutRoot = layoutTree.GetRoot(); + var tracker = layoutRoot.DescendantNodes().OfType().Single(type => type.Identifier.ValueText == "__LiteMapperCycleTracker"); + var instrumentedTracker = tracker.AddMembers(SyntaxFactory.ParseMemberDeclaration("public static object CreateSpillForLayout() => new __ActiveReferences();")!); + var options = (CSharpParseOptions)layoutTree.Options; + var instrumentedTree = CSharpSyntaxTree.Create((CSharpSyntaxNode)layoutRoot.ReplaceNode(tracker, instrumentedTracker), options); + var layout = result.Compilation.ReplaceSyntaxTree(layoutTree, instrumentedTree).AddSyntaxTrees(CSharpSyntaxTree.ParseText(@" +using System; +public static partial class Mapper +{ + public static long[] LayoutBytes() + { + Func[] factories = { + () => new __LiteMapperCycleTracker(), + () => __LiteMapperCycleTracker.CreateSpillForLayout(), + () => new System.Collections.Generic.HashSet(__ReferenceIdentityComparer.Instance) + }; + var values = new long[factories.Length * 2]; + for (var d = 0; d < factories.Length; d++) + { + for (var warmup = 0; warmup < 80; warmup++) GC.KeepAlive(factories[d]()); + var before = GC.GetAllocatedBytesForCurrentThread(); + for (var i = 0; i < 1000; i++) GC.KeepAlive(factories[d]()); + values[d + factories.Length] = GC.GetAllocatedBytesForCurrentThread() - before; + values[d] = values[d + factories.Length] / 1000; + } + return values; + } +} +", options)); + var sizes = (long[])Compile(new GeneratorRun(layout, result.Source)).GetType("Mapper")!.GetMethod("LayoutBytes")!.Invoke(null, null)!; + Console.WriteLine("Tracker/private spill/base HashSet bytes per object: " + string.Join(", ", sizes.Take(3))); + Console.WriteLine("Raw constructor totals for 1000 objects each: " + string.Join(", ", sizes.Skip(3))); + for (var d = 3; d < sizes.Length; d++) Assert.AreEqual(0L, sizes[d] % 1000, "Direct allocation measurements must be integral without hiding counter noise."); + Assert.AreEqual(3L * IntPtr.Size, sizes[0], "One object field must have the expected compact CLR allocation size."); + Assert.AreEqual(sizes[2], sizes[1], "The sealed fieldless spill subtype must not grow HashSet allocation size."); + } + + private static GeneratorRun OriginalStyleTracker(GeneratorRun result) + { + var tree = result.Compilation.SyntaxTrees.Single(candidate => candidate.GetText().ToString() == result.Source); + var root = tree.GetRoot(); + var tracker = root.DescendantNodes().OfType().Single(type => type.Identifier.ValueText == "__LiteMapperCycleTracker"); + var baseline = CSharpSyntaxTree.ParseText(@" +private sealed class __LiteMapperCycleTracker +{ + private readonly global::System.Collections.Generic.HashSet _active = new global::System.Collections.Generic.HashSet(__ReferenceIdentityComparer.Instance); + public void Enter(object source, global::System.Type sourceType, global::System.Type destinationType, string mappingMethod, Path memberPath) + { + if (!_active.Add(source)) + { + throw new global::Mammoth.LiteMapper.LiteMapperCycleException(sourceType, destinationType, mappingMethod, memberPath.ToString()); + } + } + public void Exit(object source) { _active.Remove(source); } +} +").GetRoot().DescendantNodes().OfType().Single(); + baseline = baseline.WithMembers(baseline.Members.AddRange(tracker.Members.OfType())); + var replaced = CSharpSyntaxTree.Create((CSharpSyntaxNode)root.ReplaceNode(tracker, baseline), (CSharpParseOptions)tree.Options); + return new GeneratorRun(result.Compilation.ReplaceSyntaxTree(tree, replaced), replaced.GetText().ToString()); + } + + private sealed class TrackerState + { + public TrackerState(Assembly assembly) + { + var type = assembly.GetType("Mapper")!.GetNestedType("__LiteMapperCycleTracker", BindingFlags.NonPublic)!; + Tracker = Activator.CreateInstance(type)!; + _state = type.GetField("_state", BindingFlags.NonPublic | BindingFlags.Instance)!; + Assert.IsNotNull(_state, "One object field must represent empty, raw-first, and spilled states."); + Assert.AreEqual(typeof(object), _state.FieldType); + Assert.AreEqual(1, type.GetFields(BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance | BindingFlags.DeclaredOnly).Length, + "Compact tracker must not grow beyond one instance field."); + SpillType = type.GetNestedTypes(BindingFlags.NonPublic).Single(nested => nested.BaseType == typeof(System.Collections.Generic.HashSet)); + Assert.IsTrue(SpillType.IsSealed, "Only the exact private spill subtype is a state tag."); + Assert.AreEqual(0, SpillType.GetFields(BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance | BindingFlags.DeclaredOnly).Length, + "The spill tag must not add instance allocation overhead."); + Assert.AreEqual(0, SpillType.GetProperties(BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance | BindingFlags.DeclaredOnly).Length); + Assert.IsFalse(SpillType.GetMethods(BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.DeclaredOnly).Any(method => method.Name == "Finalize")); + Assert.AreSame(Tracker, State, "The tracker itself is the empty sentinel."); + _enter = type.GetMethod("Enter")!; + _exit = type.GetMethod("Exit")!; + var path = _enter.GetParameters()[4].ParameterType; + EmptyPath = Activator.CreateInstance(path)!; + _path = Activator.CreateInstance(path, new object[] { "Parent.Child" })!; + } + public object Tracker { get; } + public object EmptyPath { get; } + public Type SpillType { get; } + public object? State => _state.GetValue(Tracker); + public System.Collections.Generic.HashSet? Spill => State?.GetType() == SpillType ? (System.Collections.Generic.HashSet)State : null; + private readonly FieldInfo _state; + private readonly MethodInfo _enter, _exit; + private readonly object _path; + public void Enter(object? source) => _enter.Invoke(Tracker, new[] { source, typeof(object), typeof(string), "Entry", _path }); + public void Exit(object? source) => _exit.Invoke(Tracker, new[] { source }); + public void AssertFirst(object? source) + { + if (Spill is System.Collections.Generic.HashSet spill) Assert.IsTrue(spill.Contains(source!)); + else Assert.AreSame(source, State); + } + public void AssertDuplicate(object? source) + { + var before = State; + var count = Spill?.Count; + var failure = Assert.ThrowsExactly(() => Enter(source)); + Assert.IsInstanceOfType(failure.InnerException); + var cycle = (LiteMapperCycleException)failure.InnerException!; + Assert.AreEqual(typeof(object), cycle.SourceType); + Assert.AreEqual(typeof(string), cycle.DestinationType); + Assert.AreEqual("Entry", cycle.MappingMethod); + Assert.AreEqual("Parent.Child", cycle.MemberPath); + Assert.AreSame(before, State, "Failed Enter cannot replace raw state or spill storage."); + Assert.AreEqual(count, Spill?.Count, "Failed Enter cannot mutate the caller's active references."); + } + public void AssertEmpty() + { + if (Spill is System.Collections.Generic.HashSet spill) Assert.AreEqual(0, spill.Count, "Every promoted reference must unwind."); + else Assert.AreSame(Tracker, State, "Raw-reference exit restores the self sentinel, including after a null source."); + } + } + + private const string ChainFactory = @" + public static Node Chain(int depth) + { + var root = new Node(); var tail = root; + for (var i = 1; i < depth; i++) { tail.Child = new Node { Value = i }; tail = tail.Child; } + return root; + }"; + + private static string ChainFixture(bool tracking) => @" +using System; +using System.Linq; +using Mammoth.LiteMapper; +[LiteMapper(ReferenceHandling = ReferenceHandling." + (tracking ? "ThrowOnCycle" : "None") + @")] +public static partial class Mapper { public static partial NodeDto Map(Node source); } +public sealed class Node +{ + public int Value { get; set; } + public Node? Child { get; set; } +" + ChainFactory + @" +} +public sealed class NodeDto { public int Value { get; set; } public NodeDto? Child { get; set; } } +"; + + private static Assembly Compile(GeneratorRun result) + { + var diagnostics = result.Compilation.GetDiagnostics().Where(d => d.Severity >= DiagnosticSeverity.Warning).ToArray(); + Assert.AreEqual(0, diagnostics.Length, string.Join(Environment.NewLine, diagnostics.Select(d => d.ToString()))); + using var stream = new MemoryStream(); + var emitted = result.Compilation.Emit(stream); + Assert.IsTrue(emitted.Success, string.Join(Environment.NewLine, emitted.Diagnostics)); + return Assembly.Load(stream.ToArray()); + } + + private static GeneratorRun Generate(string source) + { + var options = CSharpParseOptions.Default.WithLanguageVersion(LanguageVersion.CSharp9); + var references = AppContext.GetData("TRUSTED_PLATFORM_ASSEMBLIES")!.ToString()!.Split(Path.PathSeparator) + .Select(static path => MetadataReference.CreateFromFile(path)) + .Concat(new[] { MetadataReference.CreateFromFile(typeof(LiteMapperAttribute).Assembly.Location) }); + var compilation = CSharpCompilation.Create("CompactCycleTrackerTests", new[] { CSharpSyntaxTree.ParseText(source, options) }, references, + new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary, nullableContextOptions: NullableContextOptions.Enable, + optimizationLevel: OptimizationLevel.Release)); + var inputErrors = compilation.GetDiagnostics().Where(d => d.Severity == DiagnosticSeverity.Error && d.Id != "CS8795").ToArray(); + Assert.AreEqual(0, inputErrors.Length, "Invalid test input: " + string.Join(Environment.NewLine, inputErrors.Select(d => d.ToString()))); + GeneratorDriver driver = CSharpGeneratorDriver.Create(new[] { new LiteMapperGenerator().AsSourceGenerator() }, parseOptions: options); + driver = driver.RunGeneratorsAndUpdateCompilation(compilation, out var output, out _); + var result = driver.GetRunResult(); + Assert.AreEqual(0, result.Diagnostics.Count(d => d.Severity >= DiagnosticSeverity.Warning), string.Join(Environment.NewLine, result.Diagnostics)); + return new GeneratorRun(output, result.GeneratedTrees.Single().GetText().ToString()); + } + + private sealed class GeneratorRun + { + public GeneratorRun(Compilation compilation, string source) { Compilation = compilation; Source = source; } + public Compilation Compilation { get; } + public string Source { get; } + } + } +} diff --git a/tests/Mammoth.LiteMapper.Generator.Tests/HelperSemanticIsolationTests.cs b/tests/Mammoth.LiteMapper.Generator.Tests/HelperSemanticIsolationTests.cs index 3b7c111..0a4f5f1 100644 --- a/tests/Mammoth.LiteMapper.Generator.Tests/HelperSemanticIsolationTests.cs +++ b/tests/Mammoth.LiteMapper.Generator.Tests/HelperSemanticIsolationTests.cs @@ -355,8 +355,9 @@ private static bool ReportsCycle(Func map, string entry) AssertNoDiagnostics(result); var tracker = result.RunResult.GeneratedTrees.SelectMany(tree => tree.GetRoot().DescendantNodes().OfType()) .Single(type => type.Identifier.ValueText == "__LiteMapperCycleTracker"); - Assert.AreEqual(1, tracker.Members.OfType().Count(), - "Entry metadata must be forwarded without growing the per-call tracker state."); + CollectionAssert.AreEqual(new[] { "object? _state" }, tracker.Members.OfType() + .Select(field => field.Declaration.Type + " " + field.Declaration.Variables.Single().Identifier.ValueText).ToArray(), + "Only compact active-reference state belongs in the tracker; entry metadata remains forwarded through calls."); Assert.AreEqual(1, Helpers(result, "MapNested_").Length, GeneratedSource(result)); // Keep a missing-tracker regression from invoking a cyclic graph without protection. foreach (var entry in GeneratedMethods(result).Where(method => method.Modifiers.Any(SyntaxKind.PublicKeyword) && method.Modifiers.Any(SyntaxKind.PartialKeyword))) diff --git a/tests/Mammoth.LiteMapper.Generator.Tests/LazyCyclePathTests.cs b/tests/Mammoth.LiteMapper.Generator.Tests/LazyCyclePathTests.cs index ef18f42..cd6cd4e 100644 --- a/tests/Mammoth.LiteMapper.Generator.Tests/LazyCyclePathTests.cs +++ b/tests/Mammoth.LiteMapper.Generator.Tests/LazyCyclePathTests.cs @@ -79,15 +79,33 @@ public static long[] Measure(bool tracking) } return values; } + public static long SpillStorageBytes() + { + var source = new object(); + for (var warmup = 0; warmup < 40; warmup++) GC.KeepAlive(Spill(source)); + var before = GC.GetAllocatedBytesForCurrentThread(); + for (var i = 0; i < 100; i++) GC.KeepAlive(Spill(source)); + return (GC.GetAllocatedBytesForCurrentThread() - before) / 100; + } + private static object Spill(object source) + { + var spill = new System.Collections.Generic.HashSet(System.Collections.Generic.ReferenceEqualityComparer.Instance); + spill.Add(source); + return spill; + } private static NodeDto Manual(Node source) => new NodeDto { Value = source.Value, Child = source.Child == null ? null : Manual(source.Child) }; } "); - var measure = Compile(result).GetType("Probe")!.GetMethod("Measure")!; + var probe = Compile(result).GetType("Probe")!; + var measure = probe.GetMethod("Measure")!; + var spillBytes = (long)probe.GetMethod("SpillStorageBytes")!.Invoke(null, null)!; var values = (long[])measure.Invoke(null, new object[] { true })!; var manual = (long[])measure.Invoke(null, new object[] { false })!; Console.WriteLine("ThrowOnCycle bytes/op at depth 1/2: " + string.Join(", ", values)); - // The first named edge already used a literal; it must not gain a path-node allocation. - for (var round = 0; round < 3; round++) Assert.AreEqual(manual[round * 2 + 1] - manual[round * 2], values[round * 2 + 1] - values[round * 2]); + // The first named edge is a literal; the only extra overhead at depth 2 is one populated spill set. + for (var round = 0; round < 3; round++) Assert.AreEqual(spillBytes, + (values[round * 2 + 1] - manual[round * 2 + 1]) - (values[round * 2] - manual[round * 2]), + "The second simultaneous reference must allocate exactly the deferred spill storage, without a path node."); var wide = Generate(ChainFixture(true).Replace("NodeDto Map(Node source)", "NodeDto[] Map(Node[] source)") + @" public static class Probe @@ -117,7 +135,7 @@ private static NodeDto[] Manual(Node[] source) var bytes = (long)wideMeasure.Invoke(null, new object[] { true })!; var manualBytes = (long)wideMeasure.Invoke(null, new object[] { false })!; Console.WriteLine("ThrowOnCycle bytes/op for 256 sibling depth-2 chains: " + bytes); - Assert.AreEqual(values[0] - manual[0], bytes - manualBytes, "Shallow siblings may share one tracker, but must not allocate a path object for every first edge."); + Assert.AreEqual(values[1] - manual[1], bytes - manualBytes, "Depth-2 siblings must reuse one tracker and one spill set, without a path object or a set per sibling."); } [TestMethod] @@ -238,11 +256,13 @@ public static void Break(Node value, bool converter) var error = Assert.ThrowsExactly(() => helper.Invoke(null, new[] { broken, tracker, EmptyPath(), "Map" })); if (converterFailure) Assert.AreSame(mapper.GetField("Failure")!.GetValue(null), error.InnerException); else Assert.IsInstanceOfType(error.InnerException); - var active = trackerType.GetField("_active", BindingFlags.NonPublic | BindingFlags.Instance)!.GetValue(tracker)!; - Assert.AreEqual(0, active.GetType().GetProperty("Count")!.GetValue(active), "Every entered reference must unwind after failure."); + var state = trackerType.GetField("_state", BindingFlags.NonPublic | BindingFlags.Instance)!; + var active = (System.Collections.Generic.HashSet)state.GetValue(tracker)!; + Assert.AreEqual(0, active.Count, "Every entered reference must unwind after failure."); var fresh = probe.GetMethod("Create")!.Invoke(null, null)!; Assert.IsNotNull(helper.Invoke(null, new[] { fresh, tracker, EmptyPath(), "Map" })); - Assert.AreEqual(0, active.GetType().GetProperty("Count")!.GetValue(active)); + Assert.AreSame(active, state.GetValue(tracker), "Failure recovery must reuse existing spill storage without demotion."); + Assert.AreEqual(0, active.Count); } [TestMethod]