Repository navigation
Fix Fold.takeEndBySeq for negative and large elements - #3267
Merged
Merged
Conversation
harendra-kumar
commented
Oct 21, 2026
harendra-kumar
commented
Member
- When a short pattern is packed into a Word, a negative element set the upper bits, so with Int8 the pattern [1,-1] matched [-1,-1]. Each element is now masked to its own size.
- The Karp-Rabin hash used for long patterns called fromEnum, which fails on Word64 values above maxBound :: Int. It now hashes the element bytes, as splitSepByOneOf does.
harendra-kumar
force-pushed
the
splitops-fixes
branch
from
October 21, 2026 12:03
3cbc0a6 to
43590c3
Compare
When the pattern fits in a machine word, Fold.takeEndBySeq packs the elements into a Word using fromEnum. A negative fromEnum sets all the upper bits, so for Int8 the pattern [1,-1] matched the input [-1,-1]. addToWord now masks each element to its size. The Karp-Rabin hash used for longer patterns in Fold.takeEndBySeq and Fold.takeEndBySeq_ called fromEnum, which fails on Word64 values above maxBound :: Int. Use the byte hash from Streamly.Internal.Data.SeqMatch, as splitSepByOneOf does. In takeEndBySeq use RingArray.replace_ as the replaced element is not needed. On GHC 9.14.1 the step of foldMany (takeEndBySeq ...) is now specialised for every fold state, the state is no longer boxed for each element. In the Data.Fold benchmarks, "takeEndBySeq suffix crlf" allocates 56x less and is 2.8x faster, the 26 element pattern allocates 86x less and is 2.4x faster.
When the pattern fits in a machine word, the functions that emit the buffered elements rebuild them from the packed word without sign extension, toEnum 255 :: Int8 fails on the input [-1,3,4] with the pattern [1,2]. Add a Known Issues note to the Haddock docs of Fold.takeEndBySeq_ and of the Stream takeEndBySeq_, splitSepBySeq_, splitEndBySeq_, dropMatches, replaceMatches and dropSuffix.
harendra-kumar
force-pushed
the
splitops-fixes
branch
from
October 21, 2026 14:00
43590c3 to
1e6a1c0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.