Skip to content

Support Taskflow-like syntax in the Go SDK #66937

Closed

Description

@jason810496

Background

Today a Go task function takes ctx, an sdk.Client / sdk.VariableClient,
and a *slog.Logger, and pulls XCom data explicitly:

// go-sdk/example/bundle/main.go
func transform(ctx context.Context, client sdk.VariableClient, log *slog.Logger) error {
    val, err := client.GetVariable(ctx, "my_variable")
    ...
}

Java has just landed Taskflow-style task signatures under #66332, where
the upstream task's return value is declaratively injected into the
downstream task's parameter list rather than pulled via the client. We
want the same shape in Go so authors can write:

func extract(ctx context.Context) (SomeType, error) { ... }
func transform(ctx context.Context, upstream SomeType) error { ... }

What needs to happen

  1. Decide how to declare task-to-parameter wiring in Registry.AddTask
    (does depends []string carry positional binding, or do we use typed
    task handles returned by AddTask?).
  2. Use reflection in bundlev1.NewTaskFunction to recognise an extra
    typed parameter and treat it as an XCom pull from the named upstream.
  3. Serialise the binding into the Dag JSON in coordinator-mode so the
    supervisor knows which XCom to feed in.
  4. Update example/bundle/main.go to a Taskflow-style example.

Acceptance criteria

  • A Go task can declare an upstream-typed parameter and receive the
    upstream's return value at execution time without calling
    client.GetXCom explicitly.
  • Serialisation matches what the Python supervisor expects for a
    Taskflow-wired Dag (validated against test_dags.yaml).
  • The existing client-pull pattern remains supported.

Context

Activity

  1. self-assigned this
    on May 14, 2026
  2. added
    priority:mediumBug that should be fixed before next release but would not block a release
    go-sdkLabel to track work items for golang task sdk
    on May 14, 2026
  3. moved this to Ready in Go-SDKon May 14, 2026
  4. moved this from Ready to No status in Go-SDKon May 25, 2026
  5. added
    priority:lowBug with a simple workaround that would not block a release
    and removed
    priority:mediumBug that should be fixed before next release but would not block a release
    on Jun 3, 2026
  6. moved this to Ready in Go-SDKon Jun 9, 2026
  7. added this to the Airflow 3.3.0 milestone on Jun 9, 2026
  8. added
    priority:highHigh priority bug that should be patched quickly but does not require immediate new release
    and removed
    priority:lowBug with a simple workaround that would not block a release
    on Jun 9, 2026
  9. moved this from Ready to Backlog in Go-SDKon Jun 11, 2026
  10. uranusjr commented on Jun 15, 2026

    @uranusjr
    Member

    Removed from 3.3 milestone since (non-Python) SDKs have their own release cycles.

  11. jason810496 commented on Sep 17, 2026

    @jason810496
    MemberAuthor

    Settled down the design in #72043.

  12. moved this from Backlog to Done in Go-SDKon Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

go-sdkLabel to track work items for golang task sdkkind:featureFeature Requestspriority:highHigh priority bug that should be patched quickly but does not require immediate new release

Type

No type

Projects

  • Status
    Done

Relationships

None yet

Development

No branches or pull requests

Issue actions