Skip to content

fix: Read value-format metadata when the value contains the key fields - #436

Merged
ferenc-csaky merged 1 commit into
mainfrom
fix/kafka-safe-value-format-metadata
Oct 28, 2026
Merged

ferenc-csaky merged 1 commit into
mainfrom
fix/kafka-safe-value-format-metadata

Conversation

@mateczagany

Copy link
Copy Markdown
Collaborator

What does this change?

With key.fields set and 'value.fields-include' = 'ALL', kafka-safe returned NULL for value-format metadata columns, such as debezium-json's value.source.timestamp. The metadata range started at keyProjection.length + valueProjection.length, which counts the key fields twice. It now starts
at the physical field count. Upstream has the same bug (FLINK-41255), and a TODO marks switching to its helper once that's released.

Area

  • Runner CLI and Docker image (flink-sql-runner)
  • Connectors (connectors)
  • Formats (formats)
  • Function libraries (stdlib)
  • Environment utilities (env-utils)
  • Tests and samples (testing)
  • Build / CI / dependencies
  • Docs (README.md, stdlib-docs)

How did you test it?

Added a case to SafeKafkaDynamicSourceTest that fails without the fix (+I(42,Ada,null) instead of +I(42,Ada,m)). mvn test -pl connectors/kafka-safe-connector passes.

Checklist

  • The PR title starts with a type such as feat:, fix: or docs:, optionally with a scope (e.g. fix(flexible-json):)
  • mvn clean install passes locally and the code is formatted with mvn spotless:apply
  • New or changed behavior is covered by tests (given_when_then naming, AssertJ assertions)
  • New dependency and plugin versions are defined as properties in the root pom.xml
  • Commits are signed off and signed (git commit -s -S)
  • README.md and stdlib-docs are updated if user-facing behavior or functions changed

🤖 Generated with Claude Code

Signed-off-by: Mate Czagany <mate@datasqrl.com>
@ferenc-csaky ferenc-csaky added the bug Something isn't working label Oct 28, 2026
@ferenc-csaky ferenc-csaky added this to the 0.12.3 milestone Oct 28, 2026
@ferenc-csaky
ferenc-csaky merged commit 270d1c7 into main Oct 28, 2026
16 checks passed
@ferenc-csaky
ferenc-csaky deleted the fix/kafka-safe-value-format-metadata branch October 28, 2026 16:06
ferenc-csaky pushed a commit that referenced this pull request Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants