feat(governance): policy backend client, YAML compiler, loader#121
Conversation
There was a problem hiding this comment.
Pull request overview
Adds the “native” governance policy ingestion path: fetch policy YAML + enforcement mode from the backend, compile YAML into an in-memory PolicyIndex, and cache/prefetch it at runtime startup.
Changes:
- Introduces a governance backend client + policy API client for one-shot policy fetches (fail-open).
- Adds a YAML →
PolicyIndexcompiler that tolerates partial/malformed packs by skipping invalid rules/checks. - Implements a cached loader with optional background prefetch plus extensive unit tests covering fetch/parse/load behavior.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_yaml_to_index.py | Comprehensive tests for YAML → PolicyIndex compilation across supported check types and edge cases. |
| tests/test_policy_api_client.py | Tests for policy fetch skip paths, HTTP failure handling, and JSON body parsing. |
| tests/test_policy_agent_type.py | Tests agent-type selector behavior and URL query parameter composition. |
| tests/test_loader.py | Tests loader caching, prefetch coordination, enforcement mode application, and empty-index diagnostics. |
| src/uipath/runtime/governance/native/policy_api_client.py | Implements policy URL building, one-shot GET, and backend response parsing into PolicyResponse. |
| src/uipath/runtime/governance/native/loader.py | Adds cached loader + background prefetch coordination and enforcement-mode application. |
| src/uipath/runtime/governance/native/backend_client.py | Shared backend URL/header composition, org/tenant resolution, agent-type selector, and safe-call helper. |
| src/uipath/runtime/governance/native/_yaml_to_index.py | YAML compiler from packs/rules/checks into native governance models. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
418fd8f to
14bd3cc
Compare
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… worker failure, default explicit conditions to AND, policy_chars label, importorskip wrapper in agent-type test Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- backend_client/policy_api_client/loader read org/tenant (+ job context) from the environment via runtime-local ENV_* constants instead of importing UiPathConfig. Adds ENV_TRACE_ID. Diagnostic/log messages no longer reference uipath-platform. - _yaml_to_index: convert the parsed logic string to the Logic enum (Check.logic is now typed Logic). - test_loader: assert on env-var names; import reset helper from tests._helpers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
14bd3cc to
acfa5b5
Compare
…their definition site) loader.py imported ENV_ORGANIZATION_ID/ENV_TENANT_ID/resolve_organization_id/ resolve_tenant_id from policy_api_client, which only re-imports them from backend_client — tripping mypy's no_implicit_reexport (4 attr-defined errors). Import them directly from backend_client where they're defined. No runtime change; clears mypy across the stack. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
This PR is crossing the library boundaries.
I raised the same concern on the first PR in this stack: uipath-runtime should stay at the runtime abstraction layer and should not own platform-specific concerns like auth, tenant/org resolution, access tokens, base URL handling, HTTP transport, headers, retries/timeouts or UiPath platform endpoint details.
In this PR, /runtime/policy transport and platform request context are implemented inside uipath-runtime. that duplicates logic already owned by uipath-platform and creates another place where platform auth/config behavior has to be maintained.
uipath-runtime should not own the /runtime/policy transport at all. runtime should only consume a policy source abstraction or already-resolved policy content.
My proposed approach:
- define a small protocol/model in
uipath-core, e.g.GovernancePolicyProvider/GovernancePolicyResponse uipath-runtimedepends only on that protocol and turnspoliciesintoPolicyIndex- uipath-platform` implements the provider using existing platform auth/config/HTTP services (check how the other services are implemented)
- the top-level
uipathCLI wires the platform provider into runtime construction
As a general note:
uipath-core defines contracts
uipath-platform talks to UiPath platform
uipath-runtime handles runtime policy parsing/evaluation
uipath (CLI) composes them
…ovider
Replaces the direct backend HTTP fetch with a GovernancePolicyProvider
indirection so the runtime no longer owns transport, auth, or wire
format. Adds the GovernanceRuntime wrapper and the architecture doc.
- src/uipath/runtime/governance/runtime.py: new GovernanceRuntime(delegate,
policy_provider). Extracts delegate._agent_definition.is_conversational
(depth-capped chain walk), registers the provider, kicks off prefetch.
Passthrough at execute/stream/get_schema/dispose — policy loading only,
no enforcement yet (evaluator slice lands separately).
- src/uipath/runtime/governance/native/loader.py: provider-only loader.
set_policy_provider, set_agent_conversational, prefetch_policy_index,
get_policy_index, clear_policy_cache. Cached PolicyIndex; fail-open on
every failure path (raise / empty / malformed / zero rules / timeout).
- src/uipath/runtime/governance/native/_yaml_to_index.py: drop hardcoded
default clause-id messages ("A.7.4" / "A.8.4" / "A.10.4"); messages
now come from YAML, defaulting to "".
- src/uipath/runtime/governance/config.py: docstrings reworded for the
provider-supplied enforcement mode (no endpoint references).
- Removed src/uipath/runtime/governance/native/policy_api_client.py and
src/uipath/runtime/governance/native/backend_client.py — direct HTTP
fetcher and its shared helpers. Selector + timeout moved into loader.py.
- pyproject.toml: bump uipath-core to ==0.5.21.
- tests: new tests/test_governance_runtime.py (extraction, fail-open,
selector-overwrite regression, prefetch integration), rewritten
tests/test_loader.py for the provider contract, shared StubPolicyProvider
in tests/_helpers.py.
- docs/governance-architecture.md: provider-only design with explicit
'policy loading only, no enforcement yet' staging caveat, module map,
lifecycle diagram, failure-mode table.
ruff / mypy clean, 197 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
55510f9 to
e708f4f
Compare
…sational Addresses radu's review on PR #121 — collapses three architectural boundary concerns into the loader/runtime layers. 1. PolicyLoader is now instance-scoped, not module-globals. Each GovernanceRuntime constructs its own loader carrying its own provider, cache, prefetch state, and conversational selector. uipath eval can spin up multiple runtimes in parallel without them clobbering each other's policy state. 2. is_governance_enabled() reads removed from the runtime layer. The decision "should governance attach?" belongs to the wiring layer (uipath CLI) — it chooses whether to construct GovernanceRuntime at all. Inside the loader the contract is purely "provider present → load policies; provider missing → empty PolicyIndex". The feature flag itself stays in uipath-core. 3. _extract_is_conversational and its delegate-walking deleted. GovernanceRuntime now takes is_conversational explicitly as a keyword arg; the wiring layer (which knows the agent type) passes it in. Runtime no longer reaches into _delegate._agent_definition private attrs. Plus two correctness fixes called out in the readiness re-check: - clear_cache() vs in-flight prefetch worker race: worker now checks _prefetch_event is event before publishing self._policy_index so an orphaned worker can't clobber the just-cleared cache. - _load_from_provider takes the narrowed provider as a parameter instead of asserting self._provider is not None — the bandit B101 "assert stripped under -O" finding is now gone. Tests rewritten around PolicyLoader instances; cross-instance isolation pinned; orphan-worker race regression test added; conftest autouse reset fixture removed (no module state to clean). 187 pass, ruff/mypy/bandit clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses radu's follow-up on PR #121 (discussion r3465934815): the enforcement mode was still process-level scoped via config._state, defeating the point of the loader instance-scoping when uipath eval runs parallel runtimes with mixed-mode policies. - PolicyLoader now owns _enforcement_mode and exposes it via the enforcement_mode property (defaults to AUDIT when no provider response has supplied a mode) - _load_from_provider writes the instance field instead of calling the global set_enforcement_mode - config.py deleted entirely: _state / _EnforcementModeState / get_enforcement_mode / set_enforcement_mode are gone. No production consumers outside the loader; canonical EnforcementMode lives in uipath.core.governance Tests: - _helpers.reset_enforcement_mode dropped (no global to reset) - test_enforcement_mode_default rewritten around PolicyLoader.enforcement_mode; new test_two_loaders_carry_independent_enforcement_modes pins the cross-instance isolation invariant - test_governance_runtime / test_loader drop the reset fixture and the get/set imports; mode-persistence test exercises two consecutive loads on a single loader 188 passed, ruff/mypy/bandit clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
7c86294
into
feat/governance-foundation
Addresses radu's recurring concerns (PR #121 patterns) applied to the audit pipeline; resolves the post-rebase ImportError that traces.py would have hit after PR #121 deleted governance/config.py. Architectural - AuditManager is no longer a process-singleton. _audit_manager / get_audit_manager / reset_audit_manager / _configure_default_sinks are deleted. Each GovernanceRuntime constructs its own manager. uipath eval parallel runtimes no longer share a worker thread or sink list. - Constructor auto-registers the platform-mandated `traces` sink. Tests pass `register_default_sinks=False` for bare-manager fixtures. Mode on event - emit_rule_evaluation / emit_hook_summary / emit_session_start / emit_session_end now require enforcement_mode: EnforcementMode. - traces.py drops `from uipath.runtime.governance.config import get_enforcement_mode` (that module is gone post-rebase) and instead reads mode from event.data via _resolve_mode(). - Inlined _mode_to_spec → mode.value.upper(); MODE_AUDIT/MODE_ENFORCE constants removed. Production-readiness fixes - Bounded atexit: replaced per-instance atexit.register(self.method) with a process-level handler walking weakref.WeakSet(AuditManager). N managers → 1 atexit slot, no strong ref pinning disposed managers. - Fork-rebuild safety: _ensure_alive_after_fork double-checks _pid under _sinks_lock so two threads in a fresh-fork child can't both rebuild queue/worker concurrently. - Removed dead `if TYPE_CHECKING: pass`. Tests - Deleted test_audit_manager_singleton.py (singleton it pinned no longer exists). - test_audit_register_sink uses register_default_sinks=False so assertions about registered sinks see only what the test put there. - test_traces_severity carries mode on the event; new test_two_events_carry_independent_modes pins cross-runtime isolation. - New test_audit_manager_lifecycle: 6 tests covering single atexit registration, weakref GC, no-double-close, fork-rebuild lock (8-thread barrier race), same-PID fast path. 211 passed, ruff/mypy/bandit clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…from wiring Addresses radu's recurring PR #121 patterns applied to the guardrail compensation slice. Resolves the post-PR-#121 ImportError in the test file (it referenced the deleted ``uipath.runtime.governance.config`` / ``tests._helpers.reset_enforcement_mode``). Architectural — match the AuditManager / PolicyLoader shape - New GuardrailCompensator class. Each GovernanceRuntime instance gets one — owns its own ThreadPoolExecutor, BoundedSemaphore, and provider. uipath eval parallel runtimes no longer share workers, queue slots, or saturation state. - Module globals _pool / _inflight / _INFLIGHT_CAP / @atexit.register decorator removed. Process cleanup via a weakref.WeakSet of live compensators + one process-level atexit hook (same pattern PR #122 introduced for AuditManager): N runtimes → 1 atexit slot, no strong ref pinning disposed compensators. - close() is an instance method, idempotent, logs at debug on failure. - The free submit_compensation function is gone — callers use compensator.submit(...). Boundary — env reads move to the wiring layer - _resolve_trace_id signature changed to (supplied, fallback). It no longer reads UIPATH_TRACE_ID. The runtime layer is now env-free for this code path. - GovernanceRuntime accepts a trace_id: str | None constructor arg and exposes it via the .trace_id property. The wiring layer (uipath CLI) reads UIPATH_TRACE_ID and passes the value in; the evaluator slice forwards it into GuardrailCompensator(provider, trace_id=...). - GuardrailCompensator accepts trace_id at construction; it becomes the authoritative source. Per-submit trace_id is a per-call fallback. Polish - Replaced bare except Exception: pass in _resolve_trace_id with a logger.debug (bandit B110 cleared on this file). - Removed ENV_TRACE_ID constant + the os import that backed it. Tests - Full rewrite of test_guardrail_compensation to drop deleted imports (config, reset_enforcement_mode), use GuardrailCompensator(provider), and mirror AuditManager's lifecycle test set (one atexit registration, weakref GC, idempotent close, cross-instance isolation, semaphore release on provider error). - New test_resolve_trace_id_does_not_read_env pins the boundary rule: even with UIPATH_TRACE_ID set, the runtime layer ignores it. - New test_compensator_trace_id_overrides_caller_supplied_value pins the construction-supplied value winning over per-submit. - New test_governance_runtime_stashes_trace_id + test_governance_runtime_default_trace_id_is_none cover the new GovernanceRuntime kwarg + property. 238 passed, ruff/mypy clean; bandit clean on the touched files (one pre-existing B101 in _yaml_to_index.py is unchanged and out of scope). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…orts Closes radu's recurring boundary objection for the evaluator slice and makes the post-rebase stack actually import. The evaluator was the last place where everything PR #121-#123 instance-scoped collapsed back to process globals. Architectural - GovernanceEvaluator gains constructor injection: GovernanceEvaluator(policy_index, *, enforcement_mode=AUDIT, audit_manager=None, compensator=None) - Drop get_audit_manager() / get_enforcement_mode() / submit_compensation free-function lookups. The evaluator now consults zero process-globals on the hot path. - mode property is read-only (drop the setter); no two-writer race between the loader and evaluator. - audit_manager=None and compensator=None short-circuit cleanly so tests + minimal wirings work without injecting every dep. - Drop unused is_enforce_mode() public method (dead code; no caller in src/ or tests/). Post-rebase plumbing - _dispatch_compensation uses self._compensator.submit(...) instead of the deleted free function; reads r.validator (Pydantic attribute) instead of the old r["validator"] TypedDict access. - _emit_audit passes policy_id (PR #122 trace-contract field, was rule_id) and enforcement_mode=mode enum (PR #122 required arg). - Import EnforcementMode from uipath.core.governance (governance.config deleted in PR #121); import AuditManager from _audit.base (audit/ is _audit/ post-PR-#122). native/__init__.py - Drop the four module-level loader-function re-exports (get_policy_index / load_policy_index / prefetch_policy_index / reset_policy_index) — all deleted in PR #121's PolicyLoader refactor. - Export PolicyLoader instead. Tests - test_evaluator: full rewrite. Drop deleted-import paths (tests._helpers.reset_enforcement_mode, governance.config). Replace the global-manager fixture with a per-test AuditManager that uses register_default_sinks=False + a capturing sink. Every GovernanceEvaluator() call routes through a _build_evaluator helper with explicit mode + manager. New test_no_audit_manager_short_circuits replaces the previous test that mocked the global to raise. - test_evaluator_operators: drop the autouse mode-isolating fixture (no globals to isolate); DISABLED-mode test passes enforcement_mode=EnforcementMode.DISABLED via constructor. - test_guardrail_compensation: rebase-conflict resolution dropped the stale incoming-side imports (Action/LifecycleHook, backend_client, unguarded GovernanceEvaluator) since none of them are referenced in the rest of the file. 357 passed, 1 skipped (pre-existing wrapper skip). Ruff clean. Mypy clean (11 source files). Bandit shows only the pre-existing B101 in _yaml_to_index.py (out of scope). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Stacked PR 2/7 — part of splitting
feat/governance-coreinto reviewable slices. Base:feat/governance-foundation. One logical slice (branch is cumulative so CI is green). Merge in order #1 → #7 and delete each branch on merge so the next PR auto-retargets ontofeat/agentic-governance.feat/governance-corekept untouched as backup.