direct: Fix spurious recreate of schemas and volumes with mixed-case names#5531
Open
denik wants to merge 7 commits into
Open
direct: Fix spurious recreate of schemas and volumes with mixed-case names#5531denik wants to merge 7 commits into
denik wants to merge 7 commits into
Conversation
The direct engine saves local config to state, so when the UC API normalizes an identifier (lowercases catalog/schema/volume names) or strips a trailing slash from a storage URL, the next plan compares the original local value against the normalized remote value and triggers a spurious recreate/update. On every second deploy with no config changes, schemas and volumes were recreated/renamed. Previously this was suppressed with per-resource OverrideChangeDesc methods. Replace those with two declarative resources.yml keys, normalize_case and normalize_slash, parsed at runtime like the other lifecycle rules and applied in addPerFieldActions before recreate/update classification. Adding a new UC resource is now a few YAML lines instead of a Go method. - config.go: NormalizeCase / NormalizeSlash on ResourceLifecycleConfig - bundle_plan.go: shouldSkipNormalized wired into the skip-ladder - schema.go, volume.go: delete OverrideChangeDesc; declare fields in resources.yml - testserver: lowercase schema/volume identifier names to mimic UC - schema_uppercase_name invariant config is schema-only (the migrate invariant deploys via Terraform first, and the TF provider rejects an uppercase volume schema_name); volume coverage lives in resources/volumes/uppercase-name Co-authored-by: Isaac
Co-authored-by: Isaac
volume_uppercase_name covers the uppercase no-drift case on direct via the no_drift invariant; excluded from migrate (Terraform deploys first and rejects an uppercase volume schema_name). The dedicated schemas/uppercase-name test is redundant with the no_drift invariant, so remove it. Co-authored-by: Isaac
denik
changed the title
Co-authored-by: Isaac
pietern
approved these changes
Jun 10, 2026
|
|
||
| # migrate deploys via Terraform first, and the TF provider rejects an uppercase | ||
| # volume schema_name ("inconsistent final plan"). Covered by no_drift on direct. | ||
| EnvMatrixExclude.no_volume_uppercase = ["INPUT_CONFIG=volume_uppercase_name.yml.tmpl"] |
Contributor
There was a problem hiding this comment.
Interesting; this error doesn't happen for schemas?
shouldSkipNormalized returned a hardcoded reason and ignored the per-field
reason from resources.yml, unlike every other rule. Route the config reason
through (like recreate_on_changes etc.) and drop the now-unused
Reason{UC,URL}Normalization constants.
Record the relevant plan changes (field action + reason) in the
volumes/uppercase-name test so the skip classification is visible.
Co-authored-by: Isaac
Collaborator
|
Commit: 0169c0c
22 interesting tests: 15 SKIP, 7 KNOWN
Top 28 slowest tests (at least 2 minutes):
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The direct engine saves local config to state, so when Unity Catalog lowercases a schema/volume identifier, the next deploy sees the original mixed-case value drift against the normalized remote value and recreates/renames the resource — on every second deploy with no config changes. This PR fixes that.
The fix is expressed declaratively via a new
resources.ymlkey,normalize_case. The same mechanism also absorbs the existing trailing-slashstorage_locationsuppression (#5145) as a second key,normalize_slash— a refactor of behavior that already lived in per-resourceOverrideChangeDescmethods, not a new behavior.