Skip to content

direct: ignore UC-managed schema property defaults#5195

Open
janniklasrose wants to merge 8 commits into
mainfrom
janniklasrose/schema-backend-defaults
Open

direct: ignore UC-managed schema property defaults#5195
janniklasrose wants to merge 8 commits into
mainfrom
janniklasrose/schema-backend-defaults

Conversation

@janniklasrose

@janniklasrose janniklasrose commented May 6, 2026

Copy link
Copy Markdown
Contributor

Changes

  • Add backend_defaults for two fields properties['...']
  • Handle the nil-vs-map case from structdiff where a remote-only map change is emitted at the parent path: only skip the parent if every remote entry matches a configured child rule. Synthesize child paths with structpath.NewBracketString to match what structdiff produces (rather than NewStringKey, which would diverge for identifier-like keys).
  • Acceptance test with test server setting the properties based on schema name

Why

UC auto-populates system-managed property keys out of band (e.g. setting unity.catalog.managed.delta.defaults.delta.enableRowTracking). This causes the planner to output Update when there is nothing to do

Tests

  • Existing & new Acceptance tests
  • ⚠️ our current cloud test accounts do not exhibit this behaviour, only saw it in my own staging workspace

This pull request and its description were written by Isaac.

UC auto-populates system-managed property keys (e.g.
`unity.catalog.managed.delta.defaults.delta.enableRowTracking`) on
schema creation. Without a backend_defaults rule, the planner sees the
remote map as drift, emits Update, and DoUpdate sends an empty payload
which UC rejects with "UpdateSchema Nothing to update".

The rule only applies when both saved and new are nil, so user-set
properties still drive real drift detection.

Also mirror the UC behavior in the fake testserver so the no-drift
invariant is exercised locally; added acceptance/.../schemas/drift/
managed_properties covering the reproducer.

Co-authored-by: Isaac
@github-actions

github-actions Bot commented May 6, 2026

Copy link
Copy Markdown
Contributor

Approval status: pending

/acceptance/bundle/ - needs approval

5 files changed
Suggested: @denik
Also eligible: @pietern, @andrewnester, @shreyas-goenka, @anton-107, @lennartkats-db

/bundle/ - needs approval

Files: bundle/direct/bundle_plan.go, bundle/direct/bundle_plan_test.go, bundle/direct/dresources/resources.yml
Suggested: @denik
Also eligible: @pietern, @andrewnester, @shreyas-goenka, @anton-107, @lennartkats-db

General files (require maintainer)

Files: libs/testserver/schemas.go
Based on git history:

  • @denik -- recent work in bundle/direct/, bundle/direct/dresources/, libs/testserver/

Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db) can approve all areas.
See OWNERS for ownership rules.

The fake populated UC-managed properties on every property-less schema. The direct engine ignores them as backend defaults, but terraform treats them as drift, which broke unrelated schema acceptance tests across both engines. Gate the injection on a dedicated schema name so only the backend-default drift test opts in, and drop recreate's now-incidental property line.

Co-authored-by: Isaac
// to opt into UC's managed-property simulation. Scoping the injection to this name
// keeps unrelated schema tests free of the property, which terraform would otherwise
// report as drift on redeploy.
const schemaNameManagedDefaults = "schema_managed_defaults"

@janniklasrose janniklasrose Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not great to use a magic name here, but this is similar to the backend which doesn't do this based on any API parameter either

Comment thread bundle/direct/bundle_plan.go Outdated
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Commit: 66711bf

Run: 27275945278

Env 🟨​KNOWN 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
🟨​ aws linux 7 15 261 949 8:10
🟨​ aws windows 7 15 263 947 13:49
💚​ aws-ucws linux 7 15 357 863 7:02
💚​ aws-ucws windows 7 15 359 861 14:37
💚​ azure linux 1 17 264 947 6:50
💚​ azure windows 1 17 266 945 11:03
💚​ azure-ucws linux 1 17 362 859 7:49
💚​ azure-ucws windows 1 17 364 857 14:08
💚​ gcp linux 1 17 260 950 7:23
💚​ gcp windows 1 17 262 948 13:30
22 interesting tests: 15 SKIP, 7 KNOWN
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/replace_existing 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_projects/update_display_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/grants/select 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
Top 28 slowest tests (at least 2 minutes):
duration env testname
7:26 aws-ucws windows TestAccept
6:50 gcp windows TestAccept
6:14 azure-ucws windows TestAccept
6:08 azure windows TestAccept
4:30 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:28 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:23 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:23 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:06 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:29 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:20 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:20 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:15 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:07 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:00 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:59 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:55 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:53 gcp linux TestAccept
2:51 azure linux TestAccept
2:50 aws-ucws linux TestAccept
2:47 azure-ucws linux TestAccept
2:41 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:38 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:38 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:37 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:34 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:28 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:22 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

@janniklasrose janniklasrose marked this pull request as ready for review June 10, 2026 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants