test(uipath-governance): add 7 tasks + lifecycle re-tag (58% → ~72%)

[t-hsia]

Summary

• Adds 7 new test tasks for uipath-governance targeting the highest-priority coverage gaps from the /test-coverage report
• Adds the required lifecycle:* tag to all 12 existing governance tasks (previously missing on every task — tag taxonomy violation)
• Estimated component coverage improvement: 58% → ~72% (crossing the ≥70% green threshold)

New tests

Test	Tier	Mode	Gap Closed
classify_ambiguous_smoke	smoke	diagnose	Disambiguation gate — Rules #1-#3, all top-level anti-patterns
no_trigger_platform_ops_smoke	smoke	diagnose	Anti-pattern #4 sibling redirect (negative skill-trigger test)
deployed_policy_s2s_smoke	smoke	diagnose	deployed-policy get --user-id / --tenant-only S2S modes
evaluate_smoke	smoke	diagnose	access-policy evaluate PDP dry-run (taught but untested)
product_get_smoke	smoke	diagnose	aops-policy product get (only list was covered)
deployment_group_smoke	smoke	diagnose	deployment group list/get (only user/tenant covered)
template_get_smoke	smoke	diagnose	aops-policy template get for update flow (only bootstrap covered)

Lifecycle re-tag (12 existing tasks)

Lifecycle Tag	Applied To
lifecycle:discover	All read-only smokes (list, get, catalog, discovery, deployed-policy, identity-lookup)
lifecycle:generate	template_bootstrap_smoke
lifecycle:setup	All mutation smokes + all 3 e2e lifecycle tests

Test plan

• CLI verb check passes on all 7 new files
• YAML schema validation passes
• Smoke tests pass (≥95% rate)
• No env_packages with @uipath/cli
• All 19 tasks have required lifecycle:* tag

🤖 Generated with Claude Code

[github-actions]

Claude finished @t-hsia's task in 3m 9s —— View job

---

PR Review: test(uipath-governance) — 7 new tasks + lifecycle re-tag

• Read review criteria and project conventions
• Analyze full diff (27 files, +950/−12)
• Read all 27 changed files in full
• Check YAML structure, tags, and success criteria
• Check for anti-patterns (env_packages, cross-skill refs, secrets)
• Post review

---

Summary

Adds 7 new governance smoke tasks covering disambiguation, negative-trigger, S2S deployed-policy, deployment groups, product get, evaluate, and template get. Also adds the required lifecycle:* tag to all 12 existing governance tasks. Separately, the branch includes 8 new uipath-admin tasks (APMS, identity, OMS, PAT) from earlier commits that aren't mentioned in the PR title/description.

---

Change-by-Change Review

1. Governance lifecycle re-tags (12 existing files)

Severity: OK

All 12 existing governance tasks correctly receive lifecycle:*:

• Read-only smokes → lifecycle:discover ✓
• template_bootstrap_smoke → lifecycle:generate ✓
• deployment_configure_smoke → lifecycle:setup ✓
• All 3 e2e lifecycle tests → lifecycle:setup ✓

Tag placement is consistent (appended after mode:*). No other changes.

2. classify_ambiguous_smoke.yaml (new)

Severity: OK

Well-designed negative-guard test for the governance disambiguation gate. The three command_not_executed criteria with escalating specificity (AOps create → Access create → any create/update) provide good coverage. The explicit "known limitation" note about idle-agent passes is appreciated and appropriate — the failure mode being tested (silently creating wrong-branch policy) is the right one.

3. no_trigger_platform_ops_smoke.yaml (new)

Severity: OK

Clean negative-trigger test using skill_triggered with expected: "no" and a command_not_executed guard. Good marketplace precision test for anti-pattern #4.

4. evaluate_smoke.yaml (new)

Severity: OK

Covers the previously untested access-policy evaluate PDP dry-run verb. Clean structure — single command_executed with --output json baked into the pattern, plus file_exists. Weights appropriate (3.0 for core command, 1.0 for output file).

5. deployed_policy_s2s_smoke.yaml (new)

Severity: OK

Tests both --user-id and --tenant-only flags for S2S mode. Reasonable weight split (3.0 for --user-id, 2.5 for --tenant-only).

6. deployment_group_smoke.yaml (new)

Severity: Low

Description says "list, get, delete" (deployment_group_smoke.yaml:3) but only list and get are tested — no command_executed criterion for delete and the prompt doesn't ask for it. Minor inconsistency.

Fix this →

7. product_get_smoke.yaml (new)

Severity: OK

Covers the untested product get verb. Good pattern: list first, then get for a specific product.

8. template_get_smoke.yaml (new)

Severity: OK

Tests template get with --output-dir, the update-flow prerequisite. Clean and focused.

9. Uipath-admin tasks (8 new files — out of stated scope)

Severity: Medium (scoping)

The PR title and description only mention governance, but the diff includes 8 new uipath-admin tasks plus 3 fix commits for them. Per repo conventions, changes should be scoped to the skill being modified — mixing two skills' coverage expansions in one PR makes review harder.

The admin tasks themselves are well-structured:

• apms_bypass_rule_lifecycle_e2e.yaml — full bypass-rule CRUD. OK.
• apms_enforcement_enable_e2e.yaml — Rule 31 safety workflow. OK.
• apms_ip_range_edit_e2e.yaml — IP range CRUD. OK.
• audit_scope_ambiguity_smoke.yaml — scope disambiguation gate. OK.
• identity_user_lifecycle_e2e.yaml — user CRUD lifecycle. OK.
• oms_tenant_create_poll_e2e.yaml — create + async poll. OK.
• oms_tenant_service_remove_e2e.yaml — add/remove/verify services. OK.
• pat_regenerate_smoke.yaml — PAT regenerate with --expiration. OK.

Tag concern (Low): identity_user_lifecycle_e2e.yaml:11 and apms_ip_range_edit_e2e.yaml:10 use lifecycle:edit, which isn't in the documented tag taxonomy (README §Tag Taxonomy lists only discover, generate, setup). This follows existing precedent (~10 tasks in the repo already use it), but the tag should be formally proposed and added to the taxonomy.

---

What's Missing

• Passing-run claims: The test plan checklist mentions smoke pass rate but no coder-eval run results are attached. The lint workflow flags missing claims as High.
• lifecycle:edit taxonomy entry: Two admin tasks use lifecycle:edit without a corresponding entry in tests/README.md §Tag Taxonomy. Should be added if it's an accepted value.
• PR scope split: Admin and governance coverage expansions would be cleaner as separate PRs (or at minimum the PR description should document both).

---

Area Ratings

Area	Status	Notes
Frontmatter	OK	All tasks have valid YAML, required tags present
E2E Tests	OK	Good mix of smoke and e2e across both skills
Task Structure	OK	Minimal prompts, behavior-only criteria, appropriate weights
Tag Taxonomy	Low issue	lifecycle:edit undocumented; follows precedent
Repo Hygiene	Medium issue	Mixed skill scoping (admin + governance in one PR)

---

Issues for Manual Review

• CLI verb accuracy: Cannot verify that uip gov aops-policy deployment group list/get, uip gov aops-policy product get, uip gov access-policy evaluate, and uip gov aops-policy template get --output-dir are real CLI verbs with those exact flags. The skill teaches them, but they should be validated against the current CLI.
• expected_skill field semantics: The skill_triggered criterion uses both skill_name and expected_skill in classify_ambiguous_smoke.yaml:38-39 and no_trigger_platform_ops_smoke.yaml:30-31. This matches the activation.yaml pattern, but the README example only documents skill_name + expected. Worth confirming the framework handles both fields correctly.
• Negative-only test reliability: classify_ambiguous_smoke and audit_scope_ambiguity_smoke pass if the agent does nothing. Both descriptions acknowledge this limitation explicitly, but pass rates above 95% should be interpreted with this in mind.

---

Conclusion

The 7 new governance tasks and 12 lifecycle re-tags are solid. They follow established patterns, have well-structured criteria, and close meaningful coverage gaps. The one governance nit is the description/test mismatch in deployment_group_smoke.yaml.

The main concern is PR scoping — the admin tasks are a separate logical change that should be documented in the PR description at minimum. The lifecycle:edit tag should be added to the taxonomy in a follow-up.

Overall: approve with nits.

[github-actions]

Claude finished @t-hsia's task in 5m 38s —— View job

---

Coder-eval task lint (advisory)

27 task YAMLs changed; verdict counts: 0 Critical, 0 High, 0 Medium, 3 Low, 24 OK.

Rubric: .claude/commands/lint-task.md. This check is advisory and never blocks merge.

Evidence of passing run

❌ High — PR body does not claim the changed tasks have been run and passed. The test plan checklist contains unchecked items (- [ ] Smoke tests pass (≥95% rate)) but no assertion of a passing run. Please edit the PR description to add a line like:

Ran skill-gov-classify-ambiguous-smoke (and all other changed tasks) locally and they passed.

Per-task lint

tests/tasks/uipath-governance/classify_ambiguous_smoke.yaml — verdict: Low (theme-captured; see Theme 2)

tests/tasks/uipath-governance/no_trigger_platform_ops_smoke.yaml — verdict: Low (theme-captured; see Theme 2)

tests/tasks/uipath-governance/access-policy/evaluate_smoke.yaml — verdict: OK

tests/tasks/uipath-governance/access-policy/access-policy_lifecycle_e2e.yaml — verdict: OK

tests/tasks/uipath-governance/access-policy/get_policy_smoke.yaml — verdict: OK

tests/tasks/uipath-governance/access-policy/identity_lookup_smoke.yaml — verdict: OK

tests/tasks/uipath-governance/access-policy/list_policies_smoke.yaml — verdict: OK

tests/tasks/uipath-governance/aops-policy/deployed_policy_s2s_smoke.yaml — verdict: OK

tests/tasks/uipath-governance/aops-policy/deployment_group_smoke.yaml — verdict: OK

tests/tasks/uipath-governance/aops-policy/product_get_smoke.yaml — verdict: OK

tests/tasks/uipath-governance/aops-policy/template_get_smoke.yaml — verdict: OK

tests/tasks/uipath-governance/aops-policy/aops-policy_deployment_lifecycle_e2e.yaml — verdict: OK

tests/tasks/uipath-governance/aops-policy/aops-policy_lifecycle_e2e.yaml — verdict: OK

tests/tasks/uipath-governance/aops-policy/deployed_policy_smoke.yaml — verdict: OK

tests/tasks/uipath-governance/aops-policy/deployment_configure_smoke.yaml — verdict: OK

tests/tasks/uipath-governance/aops-policy/deployment_discovery_smoke.yaml — verdict: OK

tests/tasks/uipath-governance/aops-policy/discover_catalog_smoke.yaml — verdict: OK

tests/tasks/uipath-governance/aops-policy/list_policies_smoke.yaml — verdict: OK

tests/tasks/uipath-governance/aops-policy/template_bootstrap_smoke.yaml — verdict: OK

tests/tasks/uipath-admin/apms_bypass_rule_lifecycle_e2e.yaml — verdict: OK

tests/tasks/uipath-admin/apms_enforcement_enable_e2e.yaml — verdict: OK

tests/tasks/uipath-admin/apms_ip_range_edit_e2e.yaml — verdict: OK

tests/tasks/uipath-admin/audit_scope_ambiguity_smoke.yaml — verdict: Low (theme-captured; see Theme 2)

tests/tasks/uipath-admin/identity_user_lifecycle_e2e.yaml — verdict: OK

tests/tasks/uipath-admin/oms_tenant_create_poll_e2e.yaml — verdict: OK

tests/tasks/uipath-admin/oms_tenant_service_remove_e2e.yaml — verdict: OK

tests/tasks/uipath-admin/pat_regenerate_smoke.yaml — verdict: OK

Within-PR duplicates

No duplicate clusters detected. The three disambiguation tests (classify_ambiguous_smoke, no_trigger_platform_ops_smoke, audit_scope_ambiguity_smoke) share a structural pattern (purely negative guards) but test different skills and different rules — not interchangeable.

Themes

• Theme 1 (Low) — file_exists: output.txt sans content check: 11 governance smoke tasks include file_exists: output.txt (weight 1.0) without content validation. Since primary validators are command_executed patterns, this adds minimal marginal signal. Consider replacing with file_contains asserting a substring from expected CLI stderr (e.g. "error" or "unauthorized") to confirm the output file captured real command output. All member tasks are OK after theme-aware downgrade.

• Theme 2 (Medium) — Purely negative disambiguation tests: 3 tasks (classify_ambiguous_smoke :7-12, no_trigger_platform_ops_smoke :1-7, audit_scope_ambiguity_smoke :8-15) rely exclusively on command_not_executed / skill_triggered expected: no guards. An idle agent that does nothing satisfies all criteria. All three descriptions explicitly acknowledge this trade-off (e.g. classify_ambiguous_smoke lines 10-12: "Known limitation: an idle agent that does nothing also passes the negative guards."). The failure mode under test — agent silently creating a wrong-branch policy or running audit commands without scope clarification — is correctly caught; the idle-agent false-positive is a documented, accepted limitation of negative-property testing. Suggested fix (if desired): add a low-weight command_executed criterion matching a Read or Grep tool call for the SKILL.md, to verify the agent at least engaged with the skill before halting.

CLI verb reachability

All command_pattern verbs in the 27 changed tasks were manually cross-checked against assets/uip-catalog-snapshot.json (v1.197.0). No unreachable or retired verbs found. (scripts/check-cli-verbs.py was not run due to permission constraints — manual verification is equivalent for this PR.)

Conclusion

⚠ 3 task(s) have issues, max severity Low (after theme-aware downgrade from Medium). Evidence of passing run is missing (High). Advisory only — not blocking merge.

---