Skip to content

fix: treat unexpected cookie values as transient network failures in cookie handler tests#25419

Merged
rolfbjarne merged 2 commits into
mainfrom
copilot/fix-flaky-test-nsurlsessionephemeralhandlercookies
May 13, 2026
Merged

fix: treat unexpected cookie values as transient network failures in cookie handler tests#25419
rolfbjarne merged 2 commits into
mainfrom
copilot/fix-flaky-test-nsurlsessionephemeralhandlercookies

Conversation

Copilot AI commented May 13, 2026

Copy link
Copy Markdown
Contributor

TestNSUrlSessionHandlerCookies and TestNSUrlSessionEphemeralHandlerCookies were flaky across unrelated PRs because the CI-ignore guard only checked whether Set-Cookie headers were present, not whether they contained the expected value. When httpbin.org transiently returns a different cookie (e.g. from a CDN/load balancer), managedCookieResult is true but the value assertion fails — escaping the guard entirely.

Changes

  • tests/monotouch-test/System.Net.Http/MessageHandlers.cs — In TestNSUrlSessionHandlerCookiesImpl, pre-compute managedHasExpectedCookie / nativeHasExpectedCookie after the async block and fold them into the IgnoreInCI condition alongside the existing presence checks:
// Before: only guarded on header presence
if (!completed || !managedCookieResult || !nativeCookieResult)
    TestRuntime.IgnoreInCI ("Transient network failure - ignore in CI");

// After: also guard on expected cookie value
var managedHasExpectedCookie = managedCookies?.Any (v => v.StartsWith ("cookie=chocolate-chip;", StringComparison.Ordinal)) == true;
var nativeHasExpectedCookie  = nativeCookies?.Any  (v => v.StartsWith ("cookie=chocolate-chip;", StringComparison.Ordinal)) == true;

if (!completed || !managedCookieResult || !nativeCookieResult || !managedHasExpectedCookie || !nativeHasExpectedCookie)
    TestRuntime.IgnoreInCI ("Transient network failure - ignore in CI");

The downstream assertions reuse the pre-computed booleans, keeping local runs semantically identical.

…lerCookiesImpl

The guard condition in TestNSUrlSessionHandlerCookiesImpl only checked whether
Set-Cookie headers were present, not whether they contained the expected cookie
value. When httpbin.org transiently returns unexpected cookies, managedCookieResult
is true but the value assertion fails.

Pre-compute the expected-cookie-value booleans and include them in the
IgnoreInCI check so transient server responses are skipped in CI.

Ref: #25412

Agent-Logs-Url: https://github.com/dotnet/macios/sessions/ec1eca8a-e6a8-476d-a6c9-afad95e1fec0

Co-authored-by: rolfbjarne <249268+rolfbjarne@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix flaky test MonoTests.System.Net.Http.MessageHandlerTest.TestNSUrlSessionEphemeralHandlerCookies fix: treat unexpected cookie values as transient network failures in cookie handler tests May 13, 2026
Copilot AI requested a review from rolfbjarne May 13, 2026 09:46
@rolfbjarne rolfbjarne marked this pull request as ready for review May 13, 2026 09:50
Copilot AI review requested due to automatic review settings May 13, 2026 09:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces CI flakiness in the NSUrlSessionHandler cookie tests by treating “unexpected Set-Cookie values” as transient network failures (in addition to the existing guard that only checked header presence). This aligns with observed CI failures where Set-Cookie headers exist but don’t contain the expected cookie value.

Changes:

  • Precompute managedHasExpectedCookie / nativeHasExpectedCookie based on Set-Cookie header contents.
  • Extend the existing IgnoreInCI guard to also ignore when the expected cookie value isn’t present.
  • Reuse the computed booleans in the subsequent assertions to keep behavior consistent.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

Copy link
Copy Markdown
Collaborator

✅ [PR Build #f2f881e] Build passed (Detect API changes) ✅

Pipeline on Agent
Hash: f2f881eeb24023c3e808947bf8ca1f0868af5537 [PR build]

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

Copy link
Copy Markdown
Collaborator

✅ [PR Build #f2f881e] Build passed (Build packages) ✅

Pipeline on Agent
Hash: f2f881eeb24023c3e808947bf8ca1f0868af5537 [PR build]

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

Copy link
Copy Markdown
Collaborator

✅ API diff for current PR / commit

NET (empty diffs)

✅ API diff vs stable

NET (empty diffs)

ℹ️ Generator diff

Generator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes)

Pipeline on Agent
Hash: f2f881eeb24023c3e808947bf8ca1f0868af5537 [PR build]

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

Copy link
Copy Markdown
Collaborator

✅ [PR Build #f2f881e] Build passed (Build macOS tests) ✅

Pipeline on Agent
Hash: f2f881eeb24023c3e808947bf8ca1f0868af5537 [PR build]

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

Copy link
Copy Markdown
Collaborator

🚀 [CI Build #f2f881e] Test results 🚀

Test results

✅ All tests passed on VSTS: test results.

🎉 All 175 tests passed 🎉

Tests counts

✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (iOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (MacCatalyst): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (macOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (Multiple platforms): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (tvOS): All 1 tests passed. Html Report (VSDrops) Download
✅ framework: All 2 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 4 tests passed. Html Report (VSDrops) Download
✅ generator: All 5 tests passed. Html Report (VSDrops) Download
✅ interdependent-binding-projects: All 4 tests passed. Html Report (VSDrops) Download
✅ introspection: All 6 tests passed. Html Report (VSDrops) Download
✅ linker: All 44 tests passed. Html Report (VSDrops) Download
✅ monotouch (iOS): All 16 tests passed. [attempt 3] Html Report (VSDrops) Download
✅ monotouch (MacCatalyst): All 18 tests passed. [attempt 2] Html Report (VSDrops) Download
✅ monotouch (macOS): All 18 tests passed. Html Report (VSDrops) Download
✅ monotouch (tvOS): All 16 tests passed. Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. (⚠️ Html Report Publish failed ⚠️) Download
✅ sharpie: All 1 tests passed. Html Report (VSDrops) Download
✅ windows: All 3 tests passed. [attempt 3] Html Report (VSDrops) Download
✅ xcframework: All 4 tests passed. Html Report (VSDrops) Download
✅ xtro: All 1 tests passed. Html Report (VSDrops) Download

macOS tests

✅ Tests on macOS Monterey (12): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Ventura (13): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Sonoma (14): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Sequoia (15): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Tahoe (26): All 5 tests passed. Html Report (VSDrops) Download

Linux Build Verification

Linux build succeeded

Pipeline on Agent
Hash: f2f881eeb24023c3e808947bf8ca1f0868af5537 [PR build]

@rolfbjarne rolfbjarne merged commit d230e9b into main May 13, 2026
48 of 49 checks passed
@rolfbjarne rolfbjarne deleted the copilot/fix-flaky-test-nsurlsessionephemeralhandlercookies branch May 13, 2026 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI Postmortem] Flaky: TestNSUrlSessionEphemeralHandlerCookies on iOS/MacCatalyst

5 participants