Skip to content

Reload user passwords from a Secret without restarting ClickHouse - #2100

Open
alex-zaitsev wants to merge 8 commits into
0.28.0from
hot-reload-user-passwords
Open

alex-zaitsev wants to merge 8 commits into
0.28.0from
hot-reload-user-passwords

Conversation

@alex-zaitsev

@alex-zaitsev alex-zaitsev commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Add valueFrom.hotReload: true for ClickHouse user password, password_sha256_hex, and password_double_sha1_hex fields to support password rotation from Kubernetes Secrets without restarting Pods.
  • Publish hot-reload users in chop-generated-hot-reload-users.xml inside the CHI-owned Secret chi-<chi>-users. Other users remain in the existing users ConfigMap. Both files are projected into users.d without subPath, allowing ClickHouse to reload updated credentials.
  • Watch referenced source Secrets and reconcile the generated Secret when their values change. Update it only when its contents differ; missing or invalid credentials abort reconciliation and preserve the last valid configuration.
  • Preserve existing from_env behavior when hotReload is omitted. Enabling hot reload requires a one-time Pod rollout; subsequent password rotations do not.
  • Reject hotReload on unsupported fields, including settings, profiles, quotas, files, and Keeper configuration. Automatic Secret-change detection supports references declared directly in the CHI; template-inherited references require manual reconciliation.

Resolves #2092.

Test plan

  • Go unit tests: credential rendering, hash validation, missing Secrets, default-user handling, Secret-change detection, reconciliation, cleanup, and restart behavior.
  • E2E test_010011_6: verify password rotation on both replicas, rejection of previous passwords, unchanged passwords for users without hotReload, and no Pod restarts.
  • CI checks pass.

@alex-zaitsev
alex-zaitsev marked this pull request as ready for review October 12, 2026 09:21
@alex-zaitsev alex-zaitsev added the planned for review This feature is planned for review label Oct 12, 2026
alex-zaitsev and others added 8 commits October 15, 2026 16:40
valueFrom.hotReload publishes those users in a CHI-owned Secret mounted beside chop-generated-users.xml, so a later password change is picked up from users.d instead of a Pod recreate.

Co-authored-by: Cursor <cursoragent@cursor.com>
The plan is stored on chi-storage-{chi} and is gone once the namespace is deleted.

Co-authored-by: Cursor <cursoragent@cursor.com>
@alex-zaitsev
alex-zaitsev force-pushed the hot-reload-user-passwords branch from e15e67c to 88ff4ca Compare October 15, 2026 13:41

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

planned for review This feature is planned for review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant