Skip to content

add build_and_push and update_service tools - #198

Open
Rui Gao (hippogr) wants to merge 10 commits into
devfrom
ruigao/merge-image-service-tools
Open

Rui Gao (hippogr) wants to merge 10 commits into
devfrom
ruigao/merge-image-service-tools

Conversation

@hippogr

Copy link
Copy Markdown
Contributor

add build_and_push and update_service tools

Copilot AI balanced review requested due to automatic review settings October 7, 2026 23:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Service discovery misses supported workloads, and Cilium pushes can falsely report completion.

4 open findings
What changed in this PR

Adds maintenance tooling for image build/push workflows and ordered PAI service restarts.

Changes:

  • Adds resumable image discovery, build, and push automation.
  • Adds ordered service restart automation.
  • Documents both tools.
File Description
tools/​build_and_push_images.sh Builds and pushes maintained images with progress tracking.
tools/​updatePaiService.py Discovers and restarts running PAI services.
tools/​README.md Documents tool usage and behavior.

🧠 Review effort: Balanced


Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

Comment thread tools/build_and_push_images.sh
Comment thread tools/updatePaiService.py Outdated
Comment thread tools/updatePaiService.py Outdated
Comment thread tools/updatePaiService.py Outdated
Rui Gao and others added 2 commits October 13, 2026 23:42
Track command results and private-session execution ownership for resumable service maintenance. Delegate Cilium publication with image preflight, grouped progress and manifest updates. Include focused regressions and usage documentation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Concurrent batch executions are unsafe, and mismatched Cilium Agent and Operator versions can be published.

2 open findings
4 resolved since last review

🧠 Review effort: Balanced

Comment thread tools/build_and_push_images.sh
Comment thread src/cilium/build/push.sh Outdated
Hold a nonblocking workflow lock across progress reset, builds and pushes. Reject mismatched Agent and Operator versions before publication. Add focused regression coverage and document both safeguards.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The Cilium manifest update can overwrite a concurrent edit made after validation.

1 open finding
2 resolved since last review

🧠 Review effort: Balanced

Comment thread src/cilium/build/push.sh Outdated
Ask for explicit approval before updating Cilium deployment references after publication. Record declined updates as published, fail on unavailable confirmation input, and keep batch resume consistent. Remove batch and publisher locks in favor of user-managed concurrency; update regressions and README.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Resume correctness, YAML traversal, and batch execution issues could cause inconsistent deployments or misleading success.

1 open finding
Previously missed (4)

In code that hasn't changed since last review

Medium severity Retain node objects to prevent recycled IDs causing false aliases

src/​cilium/​build/​push.sh:79

The alias detector stores only integer object IDs. As compose_all advances through the production manifest's many YAML documents, nodes from older documents can be collected and their IDs reused, causing a valid later node to be reported as an alias. Retain the node objects themselves so their identities cannot be recycled during traversal.

Medium severity Reuse registry authentication across per-image pushes

tools/​build_and_push_images.sh:347

Each non-Cilium image starts a new pai_build.py push, whose push_center() unconditionally performs Azure account/ACR checks and an ACR login before pushing. With dozens of tasks this repeats the registry authentication workflow once per image, adding substantial latency and throttling risk. Authenticate once for the batch or provide a reusable/no-login per-image push path while retaining progress checkpoints.

Medium severity Reject --from values beyond the discovered task count

tools/​build_and_push_images.sh:447

--from accepts any positive integer, but values greater than the discovered task count make every task hit the skip branch and the command exits successfully without building or pushing anything. Validate the requested step against ${#TASKS[@]} after discovery so a typo cannot be reported as a successful workflow.

Medium severity Persist the Cilium manifest snapshot across resumed runs

tools/​updatePaiService.py:450

The Cilium manifest copy lives only in this invocation's temporary directory. If delete succeeds and apply fails, a resumed run skips the delete but recreates this copy from whatever manifest is currently in the worktree, so it may apply a different resource set than the one it deleted. Persist and reuse a run-scoped manifest snapshot until both Cilium steps complete.

🧠 Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An out-of-range resume step silently succeeds without performing work, and captured command diagnostics are discarded.

2 open findings
1 resolved since last review

🧠 Review effort: Balanced

Comment thread tools/build_and_push_images.sh
Comment thread tools/updatePaiService.py Outdated
Reject out-of-range resume steps before resetting progress or executing image commands. Include captured failure diagnostics while withholding raw kubeconfig stdout. Add regression coverage and update usage documentation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Destructive cluster operations, registry publication, and crash recovery require final human validation.

0 open findings

2 resolved since last review

🧠 Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Cilium Agent and Operator commit-qualified versions are incorrectly collapsed into potentially conflicting tags.

1 open finding

🧠 Review effort: Balanced

Comment thread src/cilium/build/push.sh
Scope YAML node identity tracking to each document and cover the multi-document deployment manifest in publication regression tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Clarify intentional source-hash stripping for all three images and cover republishing different image IDs under the same version-date tags.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Disruptive cluster operations and a confirmed cleanup race require correction and human validation of recovery behavior.

1 open finding
1 resolved since last review

🧠 Review effort: Balanced

Comment thread tools/updatePaiService.py Outdated
Treat ProcessLookupError as a vanished procfs identity, add regression coverage, and focus the maintenance README on script usage and operational cautions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Cluster-targeting and shared deployment-file isolation issues can cause cross-cluster mutations.

2 open findings
1 resolved since last review

🧠 Review effort: Balanced

Comment thread tools/updatePaiService.py
Comment thread tools/updatePaiService.py Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Destructive cluster operations and crash-recovery behavior require final human validation.

0 open findings

2 resolved since last review

🧠 Review effort: Balanced

Remove the kubeconfig snapshot and environment override, call paictl without -c, and document caller responsibility for configuration consistency. Update recovery coverage for inherited configuration behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Whole-manifest Cilium deletion and crash-recovery process supervision require human review and integration validation.

0 open findings

🧠 Review effort: Balanced

@hippogr
Rui Gao (hippogr) requested a balanced review from Copilot October 15, 2026 04:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Destructive Cilium operations and crash-recovery supervision require human validation of real-cluster failure and recovery behavior.

0 open findings

🧠 Review effort: Balanced

Remove prometheus/prometheus from the batch image exclusion list and update the documented exclusions. Keep all other exclusions unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

This branch was successfully deployed

1 active deployment
auto-test — 2b7a9936 Deployed Oct 19, 2026 by hippogr via Build and Deploy #488
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants