Skip to content

Cross-SDK DagSerialization compatibility CI for the Go SDK #66942

Closed

Description

@jason810496

Background

ADR 0003 explicitly promises:

The Go SDK gains a serde package that performs this encoding from
bundlev1.Bundle / bundlev1.Task, validated against
validation/serialization/test_dags.yaml (the same fixture set the
Java SDK uses), so the Go and Java outputs are byte-identical for
shared inputs.

The Go serde package exists
(go-sdk/pkg/execution/serde.go, serde_test.go) but the cross-SDK
parity hook does not - there is no prek/CI step that runs the
Go serializer against the shared test_dags.yaml and diffs the output
against Python's SerializedDAG.serialize_dag like the Java SDK does
via compare.py.

What needs to happen

  1. Add a Go-side serializer driver that consumes
    validation/serialization/test_dags.yaml and emits a JSON file
    per Dag fixture (parallel to the Java
    SerializationCompatibilityTest).
  2. Wire it into scripts/ci/prek/ so the existing compare.py runs
    against serialized_go.json alongside serialized_java.json.
  3. Fail the check on any field-level divergence.

Acceptance criteria

  • A regression in SerializedDAG.serialize_dag shape that Go doesn't
    follow fails CI before merge.
  • The Go and Java SDKs share the fixture file (no Go-only copy).

Context


Drafted-by: Claude Code (Opus 4.7) (no human review before posting)

Activity

  1. self-assigned this
    on May 14, 2026
  2. added
    priority:mediumBug that should be fixed before next release but would not block a release
    kind:taskA task that needs to be completed as part of a larger issue
    go-sdkLabel to track work items for golang task sdk
    on May 14, 2026
  3. added
    area:CIAirflow's tests and continious integration
    on May 14, 2026
  4. added
    priority:lowBug with a simple workaround that would not block a release
    and removed
    priority:mediumBug that should be fixed before next release but would not block a release
    on Jun 3, 2026
  5. added this to the Airflow 3.4.0 milestone on Jun 9, 2026
  6. jason810496 commented on Oct 20, 2026

    @jason810496
    MemberAuthor

    Already covered in the serialization compatibility prek hook.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:CIAirflow's tests and continious integrationarea:dev-toolsgo-sdkLabel to track work items for golang task sdkkind:featureFeature Requestskind:taskA task that needs to be completed as part of a larger issuepriority:lowBug with a simple workaround that would not block a release

Type

No type

Projects

  • Status
    Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions