Skip to content

Add declarative resilience primitives to DSL: retry/backoff/on_error, timeout, sub-second sleep, and set-style fan-out#166

Open
Copilot wants to merge 2 commits into
mainfrom
copilot/add-declarative-retry-backoff
Open

Add declarative resilience primitives to DSL: retry/backoff/on_error, timeout, sub-second sleep, and set-style fan-out#166
Copilot wants to merge 2 commits into
mainfrom
copilot/add-declarative-retry-backoff

Conversation

Copy link
Copy Markdown

Copilot AI commented May 23, 2026

The DSL lacked first-class resilience and fan-out constructs, forcing users to hand-build retry/backoff/error flows with loop/if/sleep and manual coordination. This change adds declarative primitives for retry, timeout, error handling, sub-second timers, and array-based parallel composition.

  • DSL resilience primitives

    • Added df.retry(...) with policy, max attempts, exponential backoff controls, jitter, and optional on_error handler.
    • Added df.catch(fut, handler) and df.on_error(fut, handler) as explicit failure-branch combinators.
    • Added df.timeout(fut, seconds) for first-class deadline wrapping.
  • Sub-second durable sleep

    • Added overloaded df.sleep(double precision) and updated df.sleep(bigint) encoding to milliseconds.
    • Runtime sleep execution now supports both legacy integer-second payloads and new millisecond config for backward compatibility.
  • Set-style fan-out

    • Added df.parallel(text[]) and df.join_all(text[]) to compose N-way joins without pairwise join/join3 chaining.
  • Runtime orchestration behavior

    • Implemented retry execution semantics in loop handling:
      • policy-based retryability checks,
      • capped exponential backoff with deterministic jitter,
      • terminal fallback branch execution via on_error_node.
    • Implemented timeout wrapper handling in race execution to return deterministic timeout failures.
  • Graph/config handling + grants/docs

    • Extended config child traversal/transformation to include on_error_node so nested handler nodes are validated and materialized correctly.
    • Updated function grant allowlists to include the new DSL signatures.
    • Updated user guide DSL reference for the new primitives.
SELECT df.start(
  df.retry(
    df.timeout(
      df.sql('SELECT call_external_api()'),
      0.5
    ),
    policy            => 'on_429',
    max_attempts      => 10,
    initial_backoff_ms=> 200,
    max_backoff_ms    => 30000,
    backoff_multiplier=> 2.0,
    jitter            => 0.2,
    on_error          => df.sql('SELECT log_terminal_failure()')
  )
);

Copilot AI changed the title [WIP] Add declarative retry and backoff support to DSL Add declarative resilience primitives to DSL: retry/backoff/on_error, timeout, sub-second sleep, and set-style fan-out May 23, 2026
Copilot AI requested a review from pinodeca May 23, 2026 12:51
@pinodeca pinodeca marked this pull request as ready for review May 23, 2026 23:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Declarative retry / backoff / on_error / sub-second sleep missing from DSL (resilience gap)

2 participants