Repository navigation
GH-54534: [C++] Fix truncation of sub-minute UTC offsets when formatting and parsing zoned timestamps - #54684
Open
ShivanshhhG wants to merge 1 commit into
Open
ShivanshhhG wants to merge 1 commit into
ShivanshhhG wants to merge 1 commit into
Conversation
…ormatting and parsing zoned timestamps When formatting zoned timestamps whose UTC offset includes sub-minute seconds (e.g. historical IANA zones like Africa/Monrovia prior to 1972 at UTC-00:44:30 or Europe/Paris prior to 1911), the vendored date.h library truncated the offset to whole minutes via duration_cast<minutes>, dropping the remaining seconds and emitting an inaccurate offset (e.g. -0044 instead of -004430). When parsed back, the timestamp was shifted by up to 59 seconds. This patch: 1. Updates date.h %z and %Ez formatters to format seconds when the UTC offset contains non-zero seconds (±hhmmss or ±hh:mm:ss). 2. Updates ParseTimestampISO8601 in value_parsing.h to recognize and parse [+-]HHMMSS and [+-]HH:MM:SS zone offsets. 3. Recognizes %Ez as providing a zone offset in StrptimeTimestampParser and GetZone. 4. Adds regression unit tests in value_parsing_test.cc and scalar_cast_test.cc. Closes apache#54534. Signed-off-by: SHIVANSH-ux-ys <singaser78@gmail.com>
|
|
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
When formatting zoned timestamps whose UTC offset includes sub-minute seconds (e.g. historical IANA zones such as
Africa/Monroviaprior to 1972 at UTC-00:44:30 orEurope/Parisprior to 1911 at UTC+00:09:21), the vendoreddate.hlibrary truncated the offset to whole minutes viaduration_cast<minutes>, dropping the remaining seconds and emitting an inaccurate offset (e.g.-0044instead of-004430). When parsed back (viaCastor CSV reader), the timestamp instant was shifted by up to 59 seconds.What changes are included in this PR?
cpp/src/arrow/vendored/datetime/date.h: Into_streamfor%zand%Ez, check whether*offset_sechas non-zero seconds (s != 0). If present, format the seconds component (±hhmmssfor%z,±hh:mm:ssfor%Ez). If whole minutes, preserve the standard±hhmm/±hh:mmrepresentation.cpp/src/arrow/util/value_parsing.h: Adddetail::ParseHHMMSSto parse 6-digit offsets and updateParseTimestampISO8601to recognize and parse[+-]HHMMSSand[+-]HH:MM:SSzone offsets.cpp/src/arrow/util/value_parsing.cc&cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc: Recognize%Ezas providing a zone offset inStrptimeTimestampParserandGetZone.cpp/src/arrow/util/value_parsing_test.cc: Add unit tests for sub-minute offsets (+004430,+00:44:30,-004430,-00:44:30) and invalid offset validations (+00:00:60,+000060,+240000,+24:00:00).cpp/src/arrow/compute/kernels/scalar_cast_test.cc: Add regression tests casting zoned timestamps inAfrica/Monrovia(-00:44:30) andEurope/Paris(+00:09:21) to string arrays.Are these changes tested?
Yes, verified with standalone compilation and unit tests covering parsing, formatting, validation bounds, and cast kernels.
Are there any user-facing changes?
Formatting zoned timestamps with historical sub-minute UTC offsets now accurately emits the seconds component (
±HH:MM:SSor±HHMMSS) rather than truncating them to minutes, preserving instant fidelity across round-trips.