Skip to content

Rust #325: maintenance loop, takeover of credentials, update validation and 2.11.3, CI adapter cache - #797

Merged
rmcdaniel merged 8 commits into
mainfrom
rust-325-maintenance
Oct 27, 2026
Merged

rmcdaniel merged 8 commits into
mainfrom
rust-325-maintenance

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Part of #325. Pre-RC native Rust (tier C).

Independent commits, batched to share one CI cycle:

1. Native runs the repair pass from its own maintenance loop (a2b07b07d)

The published image's scheduler loops every 10 s through the repair pass, schedule evaluation, timeout enforcement, payload cleanup and history pruning. Native already fired timers and schedules every 250 ms, but ran the repair pass only when someone called POST /api/system/repair/pass. So an idle run past its execution or run deadline never got the task that closes it.

The ticker can now run the repair pass with PHP's loop throttle: at most once per 5 s, shared with requested passes that respect it.

  • Opt-in: the loop runs only with DW_RUST_MAINTENANCE_LOOP=1, which the native image will set. A PHP server container never repairs on its own; PHP's separate scheduler container does.
  • Why not on by default: on this PR's first full run, the loop changed corrected-marker, deadline-health and repair-status results that compare native against a bare PHP server.
  • DW_V2_MATCHING_ROLE_QUEUE_WAKE=false (or 0) still turns it off, as it does PHP's.
  • Test: the_maintenance_loop_gives_an_overdue_idle_run_its_deadline_task.
  • With the loop off by default the execution suite passes (558). With it on, it passed too, and native passed the 11 shared fixtures most exposed to background repair.
  • Checkpoint: docs/rust-server-port-checkpoints/2026-10-26-maintenance-repair.md.

2. Takeover keeps PHP runtime credentials (fe2090bc8)

Native serves runtime credentials from PHP's own runtime_credentials table and looks tokens up by the same SHA-256 digest PHP stores (hash('sha256', $token)). So the rows need no conversion, and PHP-issued worker tokens keep working after cutover. Takeover no longer refuses them.

  • Shared PHP rows are checked on SQLite, PostgreSQL, MySQL and MariaDB: the live token authenticates with its subject, roles, tenant and claims, and the revoked one stays refused.

Update validation tasks also carry over (717445543). Native validates updates in the same workflow_update_validation_tasks rows and keeps no state of its own. A pending task whose caller was waiting on PHP is then what a PHP restart leaves: a validator may still claim it, and a caller's retry with the same request reads its recorded outcome. takeover_keeps_php_update_validation_tasks covers this. Takeover still refuses message streams and external payloads.

3. Takeover of published Server 2.11.3 (84c5085e4)

Server 2.11.3 (Core 2.14.2) was published today, and native refused every 2.11.3 database as unqualified.

  • Each recorded layout now names its parent, so 2.11.3 descends from both 2.11.1 variants.
  • Deltas were captured from real published-image databases on four engines along five upgrade paths: fresh; 2.11.1→; 2.8.0→; 2.9.7→; 2.9.7→2.11.1→. All gave the same delta from their parent.
  • The recipe:
    • copies PHP's recorded session holders (PHP and native use the same SHA-256 of the holder token);
    • counts and drops full failure messages (native reads the labelled preview, as PHP's older readers do; history keeps the full message);
    • counts and drops completed dispatches;
    • drops the idempotency digest and the stream reclaim deadline;
    • refuses with upgrade_source_service_call_dispatches_pending while a dispatch is scheduled or claimed, since native cannot resume one.
  • MySQL and MariaDB run it as resumable journal step 10.
  • Tests: SQLite 57/57 upgrade tests; PostgreSQL 7, MySQL 8 and MariaDB 8 takeover qualification tests, including a 2.11.3 takeover killed before step 10 and resumed. The native binary took over all five real published SQLite databases.
  • CI: a php2113 cohort takes over a published 2.11.3 source end to end on SQLite, PostgreSQL and MySQL. The cohort expects 51 PHP tables for 2.11.3, which adds the dispatch outbox, and 50 otherwise.
  • Checkpoint: docs/rust-server-port-checkpoints/2026-10-26-php-2113-takeover.md.

4. CI adapter cache: added, then reverted in this PR (53fd36beb, reverted by 037a97d10)

The organization's central action policy requires every cache key in a pull-request workflow to include ${{ github.event_name }}. So a cache that only pushes to main save could never be read by pull request or merge group runs, the runs it was meant to speed up. The Docker workflow isolation policy also refused the inline key step. The cache is out of this PR until the design is settled with the policy owners. The PR's net workflow change is only the php2113 cohort.

Not in this PR

The parity reference is still published 2.11.1. Core 2.14's behavior changes are native work for the reference move to 2.11.3:

  • full failure-message storage on write;
  • service-call scheduling through the dispatch outbox;
  • the new idempotency scope.

🤖 Generated with Claude Code

rmcdaniel and others added 7 commits October 26, 2026 23:03
The published image's scheduler loop runs workflow:v2:repair-pass every
few seconds unless DW_V2_MATCHING_ROLE_QUEUE_WAKE is false or 0, and the
parity reference runs it every second. Native's ticker fired timers and
schedules and enforced activity and cooperative deadlines, but ran the
repair pass only on request. So an idle run past its execution or run
deadline was never given the task that closes it, and overdue ready tasks
and due service calls waited for someone to ask.

The ticker now also runs the repair pass with PHP's loop throttle (at
most once per 5 seconds, shared with requested passes that respect it).
DW_V2_MATCHING_ROLE_QUEUE_WAKE=false or 0 turns it off, as it does PHP's.
Runtime::set_maintenance_repair does the same for one runtime.

A new test waits for the loop alone to give an idle run past its deadline
its workflow task. The two tests that count one requested pass, or keep
expired leases until they retry, turn the loop off. The execution suite
passes with the loop on (558), and native passes the shared fixtures most
exposed to background repair (repair-pass, the lease, cancellation and
deadline fixtures, worker sessions, service calls).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Native serves runtime credentials from PHP's own table and checks the
same SHA-256 token digests, so the rows need no conversion and
PHP-issued worker tokens keep working after cutover. Takeover no longer
refuses them. Shared PHP rows are checked on SQLite, PostgreSQL, MySQL
and MariaDB: the live token authenticates, the revoked one stays refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Server 2.11.3 (Core 2.14.2) was published today, and native refused every
2.11.3 database as unqualified. Each recorded layout now names its parent,
so 2.11.3 descends from both 2.11.1 variants. Deltas were captured from
real published-image databases on SQLite, PostgreSQL, MySQL and MariaDB
along five upgrade paths; all gave the same delta from their parent.

The recipe copies PHP's recorded session holders, counts and drops full
failure messages, completed dispatches, the idempotency digest and the
stream reclaim deadline, and refuses while a dispatch is still pending
(native cannot resume it). MySQL runs it as resumable journal step 10.
A php2113 CI cohort takes over a published 2.11.3 source end to end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every Shared Server fixtures matrix job waits for published-sdk-adapters,
which spent about four of its 6.4 minutes building the same pinned,
--locked adapter dependencies each run (PR run 40414928426).

The job now restores the public Cargo registry data (registry, git) and
the build tree, never the rest of CARGO_HOME. The exact key names the
Rust image, the runner OS and architecture, and a digest of every tracked
adapter input: the five adapter crates, their build scripts, pip wrapper
and SDK profiles. The key step fails unless all five crates and scripts
are among the inputs, so the key is never empty. There are no restore
keys. Only successful pushes to main save, so no pull request or merge
group writes what a later run reads.

Every run still builds, stamps and hashes its own adapters, and the final
adapter directories and their revision checks are unchanged. A unit test
pins this boundary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Native validates updates in PHP's own workflow_update_validation_tasks
rows and keeps no state of its own, so takeover can leave them as they
are instead of refusing the source. A pending task whose caller was
waiting on PHP is then what a PHP restart leaves: a validator may still
claim it, and a caller's retry with the same request reads its recorded
outcome.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The organization's central action policy requires every cache key in a
pull-request workflow to include github.event_name, so a cache saved only
by pushes to main could never be read by pull request or merge group runs,
which are the runs it was meant to speed up. The Docker workflow isolation
policy also refuses the inline key step. Take the cache out of this PR and
settle the design with the policy owners first.

This reverts commit 53fd36b.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR #797's first full run showed that a default-on repair loop changes what
native does beside a PHP server container. A PHP server never repairs on
its own; PHP's separate scheduler container does. With the loop on, native
repaired a task the corrected-marker cohort expects untouched, repaired
what the deadline-health audit expects left alone, and raced
repair_status's staged candidates. The binary now takes the scheduler's
role only with DW_RUST_MAINTENANCE_LOOP=1, which the native image will
set. DW_V2_MATCHING_ROLE_QUEUE_WAKE=false still turns it off.

The php2113 upgrade cohort expected PHP's 50 tables. PHP 2.11.3 has 51,
because the service-call dispatch outbox is new, so the expected count
follows the layout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@rmcdaniel
rmcdaniel added this pull request to the merge queue Oct 27, 2026
Merged via the queue into main with commit 8b84670 Oct 27, 2026
57 checks passed
@rmcdaniel

Copy link
Copy Markdown
Member Author

Merged through the merge queue at 2026-10-27T01:57:08Z, 23 minutes after enqueue (01:34:25Z, pinned to head d0108c708846da5961d0575e3a30f19200994276).

  • Merge commit: 8b84670c3b59a54b989a31cf298abd3a52c766f2
  • Tree: 7f63789e8a62417be2001b00e6509b5192d7476a, identical to the head's tree
  • Parents: b4bdf82663af909cbc78a8787b0d9a1bac07356f (main) and d0108c708846da5961d0575e3a30f19200994276 (head)

Follow-ups:

rmcdaniel added a commit that referenced this pull request Oct 27, 2026
The opt-in for native's maintenance loop (#797) was missing from the
config audit's native-only names, so setting it logged "Unknown DW_
variables set in environment". The native image smoke test showed it.
The shipped-environment test now sets it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel
rmcdaniel deleted the rust-325-maintenance branch October 27, 2026 02:05
rmcdaniel added a commit that referenced this pull request Oct 27, 2026
#797 merged; #799, this PR and the native image are next. External payload
takeover stays refused: PHP keeps references in about 30 locations in
their URI form, so it is its own project, and the release-candidate scope
question now includes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant