Repository navigation
Cross-SDK DagSerialization compatibility CI for the Go SDK #66942
Copy link
Copy link
Closed
Labels
area:CIAirflow's tests and continious integrationAirflow's tests and continious integrationarea:dev-toolsgo-sdkLabel to track work items for golang task sdkLabel to track work items for golang task sdkkind:featureFeature RequestsFeature Requestskind:taskA task that needs to be completed as part of a larger issueA task that needs to be completed as part of a larger issuepriority:lowBug with a simple workaround that would not block a releaseBug with a simple workaround that would not block a release
Milestone
Description
Background
ADR 0003 explicitly promises:
The Go SDK gains a
serdepackage that performs this encoding from
bundlev1.Bundle/bundlev1.Task, validated against
validation/serialization/test_dags.yaml(the same fixture set the
Java SDK uses), so the Go and Java outputs are byte-identical for
shared inputs.
The Go serde package exists
(go-sdk/pkg/execution/serde.go, serde_test.go) but the cross-SDK
parity hook does not - there is no prek/CI step that runs the
Go serializer against the shared test_dags.yaml and diffs the output
against Python's SerializedDAG.serialize_dag like the Java SDK does
via compare.py.
What needs to happen
- Add a Go-side serializer driver that consumes
validation/serialization/test_dags.yamland emits a JSON file
per Dag fixture (parallel to the Java
SerializationCompatibilityTest). - Wire it into
scripts/ci/prek/so the existingcompare.pyruns
againstserialized_go.jsonalongsideserialized_java.json. - Fail the check on any field-level divergence.
Acceptance criteria
- A regression in
SerializedDAG.serialize_dagshape that Go doesn't
follow fails CI before merge. - The Go and Java SDKs share the fixture file (no Go-only copy).
Context
- ADR:
go-sdk/adr/0003-coordinator-protocol-msgpack-ipc.md. - Java parallel: Add CI, E2E Tests, and Pre-commit Hooks for Java SDK #65959 introduced
check_java_serialization_compatibility.py; we extend the same
pattern. - Coordinator-distribution prek hooks tracking issue: Introduce common prek hooks for the new coordinator distribution #66515.
Drafted-by: Claude Code (Opus 4.7) (no human review before posting)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
area:CIAirflow's tests and continious integrationAirflow's tests and continious integrationarea:dev-toolsgo-sdkLabel to track work items for golang task sdkLabel to track work items for golang task sdkkind:featureFeature RequestsFeature Requestskind:taskA task that needs to be completed as part of a larger issueA task that needs to be completed as part of a larger issuepriority:lowBug with a simple workaround that would not block a releaseBug with a simple workaround that would not block a release
Type
Projects
- StatusShow more project fieldsDone
Activity
jason810496 commentedon Oct 20, 2026
Already covered in the serialization compatibility prek hook.