Skip to content

Preserve float kinds in DAG-JSON encoding - #655

Open
sueun-dev wants to merge 1 commit into
ipld:masterfrom
sueun-dev:fix/dagjson-float-kind-20261004
Open

sueun-dev wants to merge 1 commit into
ipld:masterfrom
sueun-dev:fix/dagjson-float-kind-20261004

Conversation

@sueun-dev

Copy link
Copy Markdown

dagjson.Encode writes a whole-valued float such as 32.0 as 32. A decode then produces an Int node, and a LinkSystem store/load round trip into a schema Float field fails with AssignInt on a Float node. Related to the case @sssion reported in #403.

This wraps refmt's encoder in EncodeOptions.Encode, adding .0 only when a Float token's numeric write would otherwise look like an integer. Integer-looking floats outside int64 use scientific notation, avoiding the decoder's overflowing integer prefix (for example 1e20 becomes 1.0e+20).

Integer/string tokens and existing decimal/exponent float output keep their formatting. Generic JSON and the deprecated Marshal with a caller-supplied token sink are unchanged.

This changes encoded bytes when a float previously looked like an integer, so callers deriving CIDs from those bytes should expect different CIDs.

Regressions exercise both encoding entrypoints, nested mixed kinds, signed zero and float extremes, plus typed LinkSystem.Store/Load with a byte buffer and a fractional-value control.

Local checks on Linux arm64:

  • Full root suite on Go 1.25.7 and 1.26.8: 31 tested packages passed, 13 tests skipped per run.
  • Go 1.25.7 root shuffled coverage and race with the CI skipgenbehavtests tag: passed, 37 tests skipped per run.
  • Go 1.25.7 formatting, all four modules' vet/staticcheck v0.6.1, and tidy/verify: passed; all module files unchanged.
  • Deterministic private probe on both Go versions: 14,743 finite-float kind-and-bit cases, nine integer and three nonfinite controls; zero failures.

Windows/macOS and remote CI have not run here.

AI assistance was used for the patch and regression tests.

@rvagg

rvagg commented Oct 5, 2026

Copy link
Copy Markdown
Member

I was going to say that we don't do this on purpose cause some impls (mainly JS) can't discriminate .. but apparently I updated the spec to say that implementations should do this .. so this is reasonable. https://ipld.io/specs/codecs/dag-json/spec/#numbers
But this approach is too much of a hack. If you can't work around refmt's behaviour then you're going to go and need to update refmt itself to allow this. But it probably needs to be either an opt-in over there, we have a config object to pass in so maybe somethign that goes in with that.
It would probably also be appropriate to update codec-fixtures for this to start building some cases for floats with no fractional component.
@vmx may have opinions here and may want to address the Rust side.
Also .. it is actually possible to do this on the JS side these days since we control the entire pipeline and don't use bare JSON.stringify. I'm not sure how much I care though. What's this for anyway @sueun-dev? What's the use-case driving this concern?

@vmx

vmx commented Oct 5, 2026

Copy link
Copy Markdown
Member

Originally DAG-JSON was only supposed to be for debugging and it shouldn't end up in CIDS as it's to easy to arrive at different CIDS (thoughm that's just my opinion). I know that there are systems that use DAG-JSON and produce CIDs off it.

I've spent a lot of time recently on floats and how to encode them in JSON. I care about that in regards to the AT Protocol, which is also IPLD based in the wider sense. They use a different JSON encoding, but floats have the same problems.

I'd be in favour in preserving the fractions in all libraries where this is possible. As this could then also help folks in the ATProto ecosystem. https://github.com/vmx/json-floats-native-types has an extensive list of how programming languages/libraries behave and what workarounds are available.

@sueun-dev

Copy link
Copy Markdown
Author

A practical case is ipfs dag get followed by ipfs dag put, with DAG-CBOR used for storage. I checked this locally with Kubo 0.43.1: storing {"Long":32.0} as DAG-CBOR and reading it back as DAG-JSON gives {"Long":32}. Putting that output back stores an integer and produces a different DAG-CBOR CID. 32.1 and integer 32 round-trip unchanged. A direct DAG-CBOR round trip preserved the original CID.

For a schema-backed example, @sssion reported this in #403: Location.Long is a Float; 32.1 works, but 32.0 fails on load. The reporter found that DAG-CBOR worked. Preserving the float in the JSON representation would let that data pass through JSON without changing its kind.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants