Skip to content

Update Resilience tools to GA SDK - #3968

Merged
Vinay Gera (g2vinay) merged 17 commits into
microsoft:mainfrom
adishiritwick:adishi/GA
Oct 20, 2026
Merged

Vinay Gera (g2vinay) merged 17 commits into
microsoft:mainfrom
adishiritwick:adishi/GA

Conversation

@adishiritwick

@adishiritwick adishiritwick commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Updating MCP tools to GA SDK of Resilience.

Expected tool Prompts Minimum score Rank #1 and score >= 0.6
resiliency_drill_create 5 0.605350 5/5
resiliency_drill_update 5 0.624731 5/5
resiliency_drill_validate-for-execution 4 0.676847 4/4
resiliency_drill_resource_add-or-update 2 0.643143 2/2
resiliency_drill_run_failover 4 0.644671 4/4
resiliency_drill_run_reprotect 3 0.693543 3/3
resiliency_usageplan_create 3 0.653704 3/3

Per-prompt evidence

Scores are shown to the evaluator's six-decimal precision.

Expected tool Prompt Rank Score Result
resiliency_drill_create Create a zonal resilience drill named <drill_name> in service group <service_group> using subscription <subscription>, region <region>, resource group <resource_group>, automated built-in roles, and recovery plan <recovery_plan_name> 1 0.741130 Pass
resiliency_drill_create Create a regional resilience drill named <drill_name> in service group <service_group> using subscription <subscription>, region <region>, and manual RBAC setup 1 0.676797 Pass
resiliency_drill_create Create a resilience drill for service group <service_group> 1 0.659705 Pass
resiliency_drill_create Create a zonal resilience drill in service group <service_group> with Health Model monitoring for <health_model_id> using a system-assigned identity 1 0.696061 Pass
resiliency_drill_create Configure a new resilience drill with goal-assignment identity and Availability and Latency SLI monitoring 1 0.605350 Pass
resiliency_drill_update Update resilience drill <drill_name> in service group <service_group> to use manual RBAC setup 1 0.653459 Pass
resiliency_drill_update Associate recovery plan <recovery_plan_name> with resilience drill <drill_name> in service group <service_group> 1 0.664798 Pass
resiliency_drill_update Change the Health Model monitored by resilience drill <drill_name> in service group <service_group> to <health_model_id> without changing its identity 1 0.624731 Pass
resiliency_drill_update Clear SLI monitoring on resilience drill <drill_name> in service group <service_group> while preserving its other configuration 1 0.639201 Pass
resiliency_drill_update Move the supporting resources of resilience drill <drill_name> in service group <service_group> to subscription <subscription> and region <region> 1 0.673745 Pass
resiliency_drill_validate-for-execution Validate resilience drill <drill_name> in service group <service_group> for execution from source location <source_location> 1 0.681224 Pass
resiliency_drill_validate-for-execution Preflight resilience drill <drill_name> in service group <service_group> to confirm it is ready to run from source locations <source_locations> 1 0.677986 Pass
resiliency_drill_validate-for-execution Validate resilience drill <drill_name> in service group <service_group> for Reprotect without specifying source locations 1 0.676847 Pass
resiliency_drill_validate-for-execution Preflight resilience drill <drill_name> in service group <service_group> using default validation from source location <source_location> 1 0.734419 Pass
resiliency_drill_resource_add-or-update Add resource <resource_id> to resilience drill <drill_name> in service group <service_group> with a fault duration of <fault_duration_minutes> minutes 1 0.643143 Pass
resiliency_drill_resource_add-or-update Update or exclude the resources of resilience drill <drill_name> in service group <service_group> 1 0.745129 Pass
resiliency_drill_run_failover Start failover for drill run <drill_run_name> of drill <drill_name> in service group <service_group>, using source location <source_location> 1 0.751500 Pass
resiliency_drill_run_failover Start failover for drill run <drill_run_name> of resilience drill <drill_name> in service group <service_group> using service defaults with no failover payload 1 0.770722 Pass
resiliency_drill_run_failover Fail over drill run <drill_run_name> from physical zone <source_location> and pause for manual input instead of automatic failover 1 0.644671 Pass
resiliency_drill_run_failover Fail over selected resources <resource_ids> in drill run <drill_run_name> from physical zones <source_locations>, and automatically continue after fault injection 1 0.706581 Pass
resiliency_drill_run_reprotect Reprotect failed-over resources in drill run <drill_run_name> for drill <drill_name> in service group <service_group> 1 0.699817 Pass
resiliency_drill_run_reprotect Reprotect only recovery resources <resource_ids> in drill run <drill_run_name> of resilience drill <drill_name> in service group <service_group> 1 0.770808 Pass
resiliency_drill_run_reprotect Start reprotection for all qualified resources in drill run <drill_run_name> without specifying a resource selection 1 0.693543 Pass
resiliency_usageplan_create Create a resilience usage plan <usage_plan_name> with plan type Standard in resource group <resource_group_name> 1 0.756060 Pass
resiliency_usageplan_create Set up a Standard resilience usage plan named <usage_plan_name> in resource group <resource_group_name> 1 0.717216 Pass
resiliency_usageplan_create Update resilience usage plan <usage_plan_name> in resource group <resource_group_name> to use the Standard plan type 1 0.653704 Pass

Pre-merge Checklist

  • Required for All PRs
    • Read contribution guidelines
    • PR title clearly describes the change
    • Commit history is clean with descriptive messages (cleanup guide)
    • Added comprehensive tests for new/modified functionality
    • Created a changelog entry if the change falls among the following: new feature, bug fix, UI/UX update, breaking change, or updated dependencies. Follow the changelog entry guide
  • For MCP tool changes:
    • One tool per PR: This PR adds or modifies only one MCP tool for faster review cycles
    • Updated servers/Azure.Mcp.Server/README.md and/or servers/Fabric.Mcp.Server/README.md documentation
    • Validate README.md changes running the script ./eng/scripts/Process-PackageReadMe.ps1. See Package README
    • For new or modified tool descriptions (excluding Fabric MCP tools), ran ToolDescriptionEvaluator and obtained a score of 0.4 or more and a top 3 ranking for all related test prompts
    • For tools with new names, including new tools or renamed tools, update consolidated-tools.json
    • For renamed tools, follow the Tool Rename Checklist and tag the PR with the breaking-change label
    • For new tools associated with Azure services or publicly available tools/APIs/products, add URL to documentation in the PR description
  • Extra steps for Azure MCP Server tool changes:
    • Updated command list in servers/Azure.Mcp.Server/docs/azmcp-commands.md
    • Ran ./eng/scripts/Update-AzCommandsMetadata.ps1 to update tool metadata in azmcp-commands.md (required for CI)
    • Updated test prompts in servers/Azure.Mcp.Server/docs/e2eTestPrompts.md
    • 👉 For Community (non-Microsoft team member) PRs:
      • Security review: Reviewed code for security vulnerabilities, malicious code, or suspicious activities before running tests (crypto mining, spam, data exfiltration, etc.)
      • Manual tests run: added comment /azp run mcp - pullrequest - live to run Live Test Pipeline

dynamicdhx and others added 3 commits October 15, 2026 13:59
Migrate the Resiliency SDK integration and existing Drills API surfaces to GA. Add request validation, isolated fixtures, recording scenarios, HTTP contract tests, and documentation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
There may be pipelines that require an authorized user to comment /azp run to run.

@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution adishiritwick! We will review the pull request and get back to you soon.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Automated provisioning lacks required test outputs, and isolated fixtures lack compatible teardown.

2 open findings
What changed in this PR

Migrates Azure Resilience Management tools to the GA SDK and API, adding drill configuration capabilities and updating tests and documentation.

Changes:

  • Adds monitoring configuration, operation selection, optional failover inputs, and selected-resource reprotection.
  • Updates goal responses and restricts usage plans to Standard.
  • Adds contract tests, isolated fixtures, recording sanitizers, and usage guidance.
File Description
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​test-resources.bicep Updates API and removes goal-template outputs.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​test-resources-post.ps1 Updates goals and workflow readiness setup.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​test-resources-drills.bicep Adds isolated drill infrastructure.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​test-resources-drills-post.ps1 Configures isolated drill fixtures.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​remove-test-resources-pre.ps1 Updates GA cleanup calls.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​Azure.Mcp.Tools.ResilienceManagement.Tests/​UsagePlans/​UsagePlanCreateCommandTests.cs Tests Standard-only plan creation.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​Azure.Mcp.Tools.ResilienceManagement.Tests/​Services/​ResilienceManagementServiceTests.cs Tests GA goal mappings and SDK types.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​Azure.Mcp.Tools.ResilienceManagement.Tests/​Services/​DrillsGaContractTests.cs Adds drill request and response contract tests.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​Azure.Mcp.Tools.ResilienceManagement.Tests/​ResilienceManagementRecordedTestsBase.cs Adds shared recording sanitizers.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​Azure.Mcp.Tools.ResilienceManagement.Tests/​Recovery/​Plans/​RecoveryPlanValidateForOperationCommandTests.cs Updates SDK operation types.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​Azure.Mcp.Tools.ResilienceManagement.Tests/​Drills/​Runs/​DrillRunReprotectCommandTests.cs Tests selected-resource reprotection.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​Azure.Mcp.Tools.ResilienceManagement.Tests/​Drills/​Runs/​DrillRunFailoverCommandTests.cs Tests optional failover inputs.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​Azure.Mcp.Tools.ResilienceManagement.Tests/​Drills/​Resources/​DrillResourceGetCommandTests.cs Tests response-validation error propagation.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​Azure.Mcp.Tools.ResilienceManagement.Tests/​Drills/​Resources/​DrillAddOrUpdateResourcesCommandTests.cs Tests drill-resource identifier validation.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​Azure.Mcp.Tools.ResilienceManagement.Tests/​Drills/​DrillValidateForExecutionCommandTests.cs Tests operation and source combinations.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​Azure.Mcp.Tools.ResilienceManagement.Tests/​Drills/​DrillUpdateCommandTests.cs Tests monitoring updates and clearing.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​Azure.Mcp.Tools.ResilienceManagement.Tests/​Drills/​DrillCreateCommandTests.cs Tests GA configuration forwarding.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​Azure.Mcp.Tools.ResilienceManagement.Tests/​Drills/​DrillConfigurationTests.cs Tests configuration validation and limits.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​Azure.Mcp.Tools.ResilienceManagement.Tests/​DrillCreateRecordedTests.cs Adds recorded creation and cleanup coverage.
tools/​Azure.Mcp.Tools.ResilienceManagement/​tests/​Azure.Mcp.Tools.ResilienceManagement.Tests/​assets.json Updates recording asset tag.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Services/​ResilienceManagementService.cs Implements GA mappings and drill requests.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Services/​IResilienceManagementService.cs Updates service contracts.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Options/​UsagePlans/​UsagePlanCreateOption.cs Documents Standard-only support.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Options/​Drills/​Runs/​DrillRunReprotectOption.cs Adds resource selection.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Options/​Drills/​Runs/​DrillRunFailoverOption.cs Makes failover inputs optional.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Options/​Drills/​Resources/​DrillAddOrUpdateResourcesOption.cs Clarifies drill-resource IDs.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Options/​Drills/​DrillValidateForExecutionOption.cs Adds operation selector.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Options/​Drills/​DrillUpdateOption.cs Adds monitoring update options.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Options/​Drills/​DrillCreateOption.cs Adds GA configuration options.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Models/​UsagePlanKind.cs Removes Basic plan type.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Models/​GoalResourceInfo.cs Adds zonal resiliency details.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Models/​GoalAssignmentInfo.cs Adds zonal resiliency requirement.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Models/​DrillExecutionOperation.cs Defines execution operations.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Commands/​Recovery/​Plans/​RecoveryPlanValidateForOperationCommand.cs Uses renamed SDK operation type.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Commands/​Drills/​Runs/​DrillRunReprotectCommand.cs Validates and forwards resource selection.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Commands/​Drills/​Runs/​DrillRunFailoverCommand.cs Supports service-default failover requests.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Commands/​Drills/​Resources/​DrillAddOrUpdateResourcesCommand.cs Validates drill-resource ownership.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Commands/​Drills/​DrillValidateForExecutionCommand.cs Validates operation-specific inputs.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Commands/​Drills/​DrillUpdateCommand.cs Handles GA configuration updates.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Commands/​Drills/​DrillCreateCommand.cs Handles GA creation configuration.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Commands/​Drills/​DrillConfigurationPatch.cs Preserves sparse PATCH fields.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Commands/​Drills/​DrillConfiguration.cs Parses and validates configuration JSON.
tools/​Azure.Mcp.Tools.ResilienceManagement/​src/​Commands/​Drills/​DrillActionValidation.cs Shares recovery-resource validation.
tools/​Azure.Mcp.Tools.ResilienceManagement/​docs/​sdk-ga-upgrade-usage-recovery-plan.md Documents SDK migration.
servers/​Azure.Mcp.Server/​docs/​e2eTestPrompts.md Adds GA evaluation prompts.
servers/​Azure.Mcp.Server/​docs/​azmcp-commands.md Documents updated command inputs.
servers/​Azure.Mcp.Server/​changelog-entries/​resiliency-drills-ga-sdk.yaml Records features, fixes, and breaking changes.
Directory.Packages.props Upgrades Resilience Management SDK.
.github/​skills/​resilience-management-operations/​references/​workflows.md Updates operational workflows.
.github/​skills/​resilience-management-operations/​references/​tools.md Updates tool capabilities and parameters.
.github/​skills/​resilience-management-operations/​references/​payloads.md Documents GA configuration payloads.
.github/​skills/​resilience-management-operations/​references/​development.md Documents fixture and validation requirements.

🧠 Review effort: Balanced


💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tools/Azure.Mcp.Tools.ResilienceManagement/tests/test-resources-drills-post.ps1 Outdated
adishiritwick and others added 6 commits October 16, 2026 16:28
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@adishiritwick adishiritwick changed the title Adishi/ga Update Resilience tools to GA SDK Oct 18, 2026
Comment thread .github/skills/resilience-management-operations/references/development.md Outdated
adishiritwick and others added 2 commits October 20, 2026 18:29
…ols with GA SDK

- Drills fixture brings the recovery plan back to Ready (exclude new service-group members, finalize, checkReadiness) before and after the full failover/reprotect cycle; RO rejects failover unless the plan is Ready.
- Remove --recoveryplan from drill create/update: recoveryPlanId is read-only in the GA SDK and never serialized; the service links the service group's plan.
- Return the drill managed identity in drill get/create/update output.
- Always emit hasCompleted for drill resync, validation, add-or-update, and mark-complete results.
- Update docs, e2e prompts, skill reference, changelog, and tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@g2vinay
Vinay Gera (g2vinay) merged commit 412e187 into microsoft:main Oct 20, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants