feat(external): pull-based subscription for shared artifact repos consumed by many pipelines

[joshua-temple]

Problem

cascade's external-artifact coordination assumes an owned topology. A producer repo declares notify: and pushes into a single consuming pipeline's external-update.yaml using a shared dispatch token, which fans in cleanly for many producers to one pipeline.

It does not cover the inverse. When an org standardizes on a single shared artifact repo that many independently owned pipelines consume, cascade has no surface for it. notify: targets exactly one primary and offers no fan-out, and having the central repo push to N consumers would require it to hold dispatch tokens into every consumer, which the hardening guidance explicitly warns against ("never replicate one long-lived token across many repositories").

The result today: a shared central artifact repo consumed by N pipelines is unmodeled, and the only push-based workaround concentrates cross-repo write tokens in the one place that should stay least-privileged.

Proposal

Add a pull-based subscription mode so a consuming pipeline resolves the latest published version of a central artifact repo on its own cadence. No token is held by the center, and the center carries no per-consumer configuration. Its generated workflows stay constant no matter how many pipelines consume it.

Leading design: extend the primary's existing external: entry with a pull mode, since the primary already names the external artifact repo there. external: is already a list, so a consumer subscribed to multiple central repos is simply multiple entries, each self-contained (its own version source and cadence, tracked under its own state key). This is additive and backward compatible, and keeps the trust boundary legible: pull mode present means no token crosses that boundary.

Cadence (when the consumer picks up a new version)

• Resolve latest on the consumer's own next pipeline run (default). The consumer queries the center at run time and pins to the resolved version. No new trigger infrastructure, no cron, no cross-repo token.
• Scheduled poll (opt-in). A generated workflow in the consumer's own repo that picks up new central versions without waiting for a local commit.
• Manual dispatch, retained as a near-free workflow_dispatch trigger on the poll workflow rather than a separately engineered mode.

Version source (what the consumer resolves), decided

A bounded field, separate from the existing ref (which pins the workflow version, not the artifact version):

• Blank or default: latest tag, defined as the highest semver tag matching a version pattern, not the most-recently-created tag, so a backport does not silently win.
• Override: latest commit on the producer's trunk, for consumers that track the edge or producers that do not tag. This mode assumes the producer publishes an artifact per commit; the generator should enforce or document that, otherwise a consumer pins a commit with no built artifact.

Cadence and version source are orthogonal axes: three cadences plus two sources, not a six-way matrix.

Out of scope

• The owned push path (notify:) is unchanged.
• No consumer registry or fan-out logic held by the center.

Future consideration

Forced or expedited propagation, such as pushing a security-critical artifact version to consumers ahead of their own pull cadence, is out of scope here but a plausible future capability. General push fan-out is rejected on the trust model, since the center must not hold write tokens into consumers. The clean form of a security force-update keeps it out of the central repo as well, for example a security-flagged release consumers react to more aggressively, or an org-scoped dispatcher (a security team's app, not the artifact repo) that triggers each consumer's manual-dispatch hook. The manual-dispatch trigger this issue already generates is the natural seam for it.

Acceptance

• Manifest surface and JSON schema for both the pull mode and the version-source field, additive and backward compatible (old manifests still parse).
• Generator emits the resolve-latest step and the opt-in scheduled poll workflow into the consumer's own repo only. Nothing is emitted into the central repo.
• A consumer subscribed to multiple centrals gets one scheduled-poll workflow that fans over every poll-mode entry, not one workflow per subscribed repo. Pull holds no write or dispatch token anywhere; resolving from a private central repo uses a read credential (one default secret, per-entry overridable).
• Unit tests on generated output for both cadences and both version sources, plus schema back-compat.
• An e2e/ scenario covering a consumer subscribed to two central artifact repos, one on tag with next-run and one on commit with scheduled-poll, proving per-entry independence and that a single poll workflow fans over all poll-mode entries.
• Fleet coverage: a central artifact example repo plus at least one consumer example repo that subscribes to it, wired into the fleet-e2e fan-out.
• Docs updated:
  • Root README.md gains a mermaid diagram showing the federated pull topology (one central artifact repo, N consumers pulling on their own cadence, no token held by the center), alongside the existing owned push fan-in.
  • architecture.md external-artifact section and diagram updated to present both push (owned) and pull (federated) directions.
  • configuration.md documents the new field, the three cadences, and the two version sources.
  • A hardening note on why pull needs no central token.
  • cli-reference.md and the testing/coverage docs updated if flags or the e2e scenario list change.

Resolved: placement

Pull mode, version source, and cadence are per-entry fields on the primary's existing external: list (already []ExternalRepoConfig). A consumer subscribed to N central repos is N external: entries, each self-contained. A separate top-level block is rejected because it would re-invent the list external: already provides. No open design questions remain; the rest is implementation shape settled during TDD.

[joshua-temple]

Decision on the version-source open question: fold in a bounded resolver field now.

• Default (blank): latest tag, defined as the highest semver tag matching a version pattern, not the most-recently-created tag, so a backport does not silently win.
• Override: latest commit on the producer trunk, for consumers tracking the edge or producers that do not tag. This mode assumes the producer publishes an artifact per commit; the generator should enforce or document that, otherwise a consumer pins a commit with no built artifact.

This is a new field, kept distinct from the existing ref, which pins the workflow version rather than the artifact version.