Skip to content

GH-54534: [C++] Fix truncation of sub-minute UTC offsets when formatting and parsing zoned timestamps - #54684

Open
ShivanshhhG wants to merge 1 commit into
apache:mainfrom
ShivanshhhG:fix-54534-subminute-offsets
Open

ShivanshhhG wants to merge 1 commit into
apache:mainfrom
ShivanshhhG:fix-54534-subminute-offsets

Conversation

@ShivanshhhG

Copy link
Copy Markdown

Rationale for this change

When formatting zoned timestamps whose UTC offset includes sub-minute seconds (e.g. historical IANA zones such as Africa/Monrovia prior to 1972 at UTC-00:44:30 or Europe/Paris prior to 1911 at UTC+00:09:21), the vendored date.h library truncated the offset to whole minutes via duration_cast<minutes>, dropping the remaining seconds and emitting an inaccurate offset (e.g. -0044 instead of -004430). When parsed back (via Cast or CSV reader), the timestamp instant was shifted by up to 59 seconds.

What changes are included in this PR?

  1. cpp/src/arrow/vendored/datetime/date.h: In to_stream for %z and %Ez, check whether *offset_sec has non-zero seconds (s != 0). If present, format the seconds component (±hhmmss for %z, ±hh:mm:ss for %Ez). If whole minutes, preserve the standard ±hhmm / ±hh:mm representation.
  2. cpp/src/arrow/util/value_parsing.h: Add detail::ParseHHMMSS to parse 6-digit offsets and update ParseTimestampISO8601 to recognize and parse [+-]HHMMSS and [+-]HH:MM:SS zone offsets.
  3. cpp/src/arrow/util/value_parsing.cc & cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc: Recognize %Ez as providing a zone offset in StrptimeTimestampParser and GetZone.
  4. cpp/src/arrow/util/value_parsing_test.cc: Add unit tests for sub-minute offsets (+004430, +00:44:30, -004430, -00:44:30) and invalid offset validations (+00:00:60, +000060, +240000, +24:00:00).
  5. cpp/src/arrow/compute/kernels/scalar_cast_test.cc: Add regression tests casting zoned timestamps in Africa/Monrovia (-00:44:30) and Europe/Paris (+00:09:21) to string arrays.

Are these changes tested?

Yes, verified with standalone compilation and unit tests covering parsing, formatting, validation bounds, and cast kernels.

Are there any user-facing changes?

Formatting zoned timestamps with historical sub-minute UTC offsets now accurately emits the seconds component (±HH:MM:SS or ±HHMMSS) rather than truncating them to minutes, preserving instant fidelity across round-trips.

…ormatting and parsing zoned timestamps

When formatting zoned timestamps whose UTC offset includes sub-minute seconds (e.g. historical IANA zones like Africa/Monrovia prior to 1972 at UTC-00:44:30 or Europe/Paris prior to 1911), the vendored date.h library truncated the offset to whole minutes via duration_cast<minutes>, dropping the remaining seconds and emitting an inaccurate offset (e.g. -0044 instead of -004430). When parsed back, the timestamp was shifted by up to 59 seconds.

This patch:
1. Updates date.h %z and %Ez formatters to format seconds when the UTC offset contains non-zero seconds (±hhmmss or ±hh:mm:ss).
2. Updates ParseTimestampISO8601 in value_parsing.h to recognize and parse [+-]HHMMSS and [+-]HH:MM:SS zone offsets.
3. Recognizes %Ez as providing a zone offset in StrptimeTimestampParser and GetZone.
4. Adds regression unit tests in value_parsing_test.cc and scalar_cast_test.cc.

Closes apache#54534.

Signed-off-by: SHIVANSH-ux-ys <singaser78@gmail.com>
@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #54534 has been automatically assigned in GitHub to PR creator.

@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #54534 has no components, please add labels for components.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant