Skip to content

{Test SDK} Fix foreground busy loop during playback polling - #34300

Merged
Ethan Yang (necusjz) merged 3 commits into
Azure:devfrom
kestry:fix-testsdk-playback-polling-yield
Nov 2, 2026
Merged

Ethan Yang (necusjz) merged 3 commits into
Azure:devfrom
kestry:fix-testsdk-playback-polling-yield

Conversation

@kestry

@kestry Jean Kim (kestry) commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

🤖 PR Validation — ⚠️ Review suggested

Breaking Changes Tests Suppressions
️✔️ None ️✔️ 130/130 ⚠️ 1 added

This PR adds warning or check suppressions. Fix the underlying warning where possible and justify each remaining suppression in a code comment or the PR description.

1 added suppression
File Line Kind Code
src/azure-cli-testsdk/azure/cli/testsdk/patches.py 94 pylint def _yield_to_polling_thread(*args, **kwargs): # pylint: disable=unused-argument

Related command

Shared azure-cli-testsdk scenario playback for commands that wait for background pollers; not specific to one command module.

Description

Problem:
Currently, playback replaces both ARM service polling delays and the CLI foreground completion-check delay with no-ops. However, the foreground loop can consequently spin while a background poller is trying to make progress. This caused high CPU utilization and led playback to run near as long as the live recording. (54min live, 41min playback).

Local testing showed 9.9 million polls in ~57 seconds.

Solution:
Keep the ARM service delay as a no-op, but replace the foreground playback delay with threading.Event.wait(0.001). This requests a short real wait and yields to the background poller even when playback has patched time.sleep to a no-op. Existing test cleanup restores both delay methods.

Local test results:

Measurement Before After
Local poller.done() checks 9,902,094 20
Test-body duration ~57 seconds 6.24 seconds

Playback down from 41min to 40sec:
image

Additional info:
Added regression coverage for both replay-hook orders, the no-op service delay versus the requested 1 ms foreground wait, cleanup, playback-only registration, completion, and exception propagation. Documented the behavior in the test SDK README.

This change is limited to four test SDK files. It does not change production/core polling, live execution, recordings, package versions, or Edge Action-specific behavior.

Testing Guide

Run from the repository root using the existing development environment. AZURE_TEST_RUN_LIVE must be unset; setting it to a nonempty string such as false is not equivalent.

Remove-Item Env:AZURE_TEST_RUN_LIVE -ErrorAction SilentlyContinue
$env:PYTEST_DISABLE_PLUGIN_AUTOLOAD='1'
& 'C:\wcodebase\repos\azure-cli-extensions\.venv\Scripts\python.exe' -m pytest 'src\azure-cli-testsdk\azure\cli\testsdk\scenario_tests\tests' 'src\azure-cli-testsdk\azure\cli\testsdk\tests' -q --disable-warnings

Result: passed, 21 tests and 4 subtests, with 4 warnings, in 1.82 s.

& 'C:\wcodebase\repos\azure-cli-extensions\.venv\Scripts\python.exe' -m flake8 --exclude='' 'src\azure-cli-testsdk\azure\cli\testsdk\patches.py' 'src\azure-cli-testsdk\azure\cli\testsdk\tests'

Result: passed.

git diff --check
git diff --cached --check

Result: both passed before committing.

The local pre-commit hook reported that its Python environment was not activated and exited successfully without running its checks; the explicit pytest and flake8 commands above were run separately and passed. No live operations or full Edge Action eight-scenario playback run were performed for this publication.

History Notes

N/A. Test-only change; the {Test SDK} title intentionally excludes a customer-facing history entry.


This checklist is used to make sure that common guidelines for a pull request are followed.

Yield to background pollers using a short threading event wait during playback while continuing to skip ARM service delays. Cover replay hook order, cleanup, playback-only registration, completion, and error propagation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 29, 2026 04:26
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the playback polling-delay change.

No open issues 🎉

What changed in this PR

This PR prevents busy looping in shared test SDK playback by giving background pollers time to progress. Production polling and live execution remain unchanged.

Changes:

  • Replace the foreground playback delay with threading.Event.wait(0.001), while keeping ARM service delays disabled.
  • Add regression tests for patch order, delay behavior, cleanup, playback-only registration, completion, and exception propagation.
  • Document playback polling behavior.
File Description
src/​azure-cli-testsdk/​README.rst Document playback delays and cleanup behavior.
src/​azure-cli-testsdk/​azure/​cli/​testsdk/​tests/​test_patches.py Add regression coverage for polling delay patches.
src/​azure-cli-testsdk/​azure/​cli/​testsdk/​tests/​__init__.py Initialize the test package.
src/​azure-cli-testsdk/​azure/​cli/​testsdk/​patches.py Add a short foreground wait independent of patched time.sleep.

Review effort: Balanced

@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

Use the same subsection heading style as HISTORY.rst so the combined package description renders without inconsistent-title errors.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

@necusjz

Copy link
Copy Markdown
Member

better to move the tests under azure/cli/testsdk/scenario_tests/tests.

Move existing tests without changing their contents and remove the redundant test package initializer.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

@necusjz
Ethan Yang (necusjz) merged commit 7376cde into Azure:dev Nov 2, 2026
52 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants