Repository navigation
Commit 7ce5324
committed
GH-55399: [C++][Parquet] Use CountIf16 for level counting in reader and writer
### Rationale for this change
Follow-up to the `CountIf16` helper: use it for the level counting loops
in the Parquet column reader/writer and in the file scanner, replacing
open-coded scalar loops and `std::count` calls.
Converted sites:
- `TypedColumnReaderImpl::ReadLevelsInCurrentPage`: count non-null values.
- `TypedRecordReader::SkipRecordsInBufferNonRepeated`: count values to skip.
- `TypedRecordReader::DelimitRecords`: count records (rep level == 0) and
values seen (def level == max).
- `TypedRecordReader::ReadDenseForOptionalInBuffer`: count values to read.
- `TypedColumnWriterImpl::WriteLevels`: count values to write (both
overloads) and records written (rep level == 0).
- `TypedColumnWriterImpl::MaybeCalculateValidityBits`: count values to
write and spaced values to write.
- `ScanFileContents`: count rows (rep level == 0).
Sites are left as is where counting is not the bottleneck: the
per-element bitmap loops in `DefRepLevelsToListInfo` also build validity
bitmaps and write list offsets for every level, `page_index.cc` counts
pages rather than levels, and the non-nullable validation loop in
`WriteArrow` needs to `break` early on the first mismatch.
`UpdateLevelHistogram`'s `max_level == 1` fast path is also left as is: its
levels are known to be 0 or 1, so it sums them with `std::accumulate` in the
`int16_t` space, which is a single 16-bit add per vector, while counting the
1s costs a compare and a subtract.
### What changes are included in this PR?
Use `CountIf16` for the level counting call sites above.
### Are these changes tested?
Covered by the existing Parquet reader, writer and Arrow reader/writer
tests.
### Are there any user-facing changes?
No.1 parent fcb9cb5 commit 7ce5324
4 files changed
Lines changed: 52 additions & 51 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
| 50 | + | |
50 | 51 | | |
51 | 52 | | |
52 | 53 | | |
| |||
1546 | 1547 | | |
1547 | 1548 | | |
1548 | 1549 | | |
1549 | | - | |
1550 | | - | |
| 1550 | + | |
| 1551 | + | |
| 1552 | + | |
| 1553 | + | |
1551 | 1554 | | |
1552 | 1555 | | |
1553 | 1556 | | |
| |||
1961 | 1964 | | |
1962 | 1965 | | |
1963 | 1966 | | |
1964 | | - | |
1965 | | - | |
1966 | | - | |
| 1967 | + | |
| 1968 | + | |
| 1969 | + | |
| 1970 | + | |
1967 | 1971 | | |
1968 | 1972 | | |
1969 | 1973 | | |
| |||
2128 | 2132 | | |
2129 | 2133 | | |
2130 | 2134 | | |
2131 | | - | |
2132 | | - | |
2133 | | - | |
| 2135 | + | |
| 2136 | + | |
2134 | 2137 | | |
2135 | 2138 | | |
2136 | 2139 | | |
| |||
2144 | 2147 | | |
2145 | 2148 | | |
2146 | 2149 | | |
2147 | | - | |
2148 | | - | |
| 2150 | + | |
| 2151 | + | |
| 2152 | + | |
| 2153 | + | |
2149 | 2154 | | |
2150 | 2155 | | |
2151 | 2156 | | |
| |||
2234 | 2239 | | |
2235 | 2240 | | |
2236 | 2241 | | |
2237 | | - | |
2238 | | - | |
| 2242 | + | |
| 2243 | + | |
| 2244 | + | |
| 2245 | + | |
2239 | 2246 | | |
2240 | 2247 | | |
2241 | 2248 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
| 58 | + | |
58 | 59 | | |
59 | 60 | | |
60 | 61 | | |
| |||
1698 | 1699 | | |
1699 | 1700 | | |
1700 | 1701 | | |
1701 | | - | |
1702 | | - | |
1703 | | - | |
1704 | | - | |
1705 | | - | |
| 1702 | + | |
| 1703 | + | |
| 1704 | + | |
| 1705 | + | |
1706 | 1706 | | |
1707 | 1707 | | |
1708 | 1708 | | |
| |||
1714 | 1714 | | |
1715 | 1715 | | |
1716 | 1716 | | |
1717 | | - | |
1718 | | - | |
1719 | | - | |
1720 | | - | |
1721 | | - | |
1722 | | - | |
| 1717 | + | |
| 1718 | + | |
| 1719 | + | |
| 1720 | + | |
1723 | 1721 | | |
1724 | 1722 | | |
1725 | 1723 | | |
| |||
1749 | 1747 | | |
1750 | 1748 | | |
1751 | 1749 | | |
1752 | | - | |
1753 | | - | |
1754 | | - | |
1755 | | - | |
1756 | | - | |
| 1750 | + | |
| 1751 | + | |
| 1752 | + | |
| 1753 | + | |
| 1754 | + | |
| 1755 | + | |
| 1756 | + | |
| 1757 | + | |
| 1758 | + | |
| 1759 | + | |
1757 | 1760 | | |
1758 | 1761 | | |
1759 | 1762 | | |
| |||
1809 | 1812 | | |
1810 | 1813 | | |
1811 | 1814 | | |
1812 | | - | |
1813 | | - | |
1814 | | - | |
1815 | | - | |
1816 | | - | |
1817 | | - | |
| 1815 | + | |
| 1816 | + | |
| 1817 | + | |
| 1818 | + | |
1818 | 1819 | | |
1819 | 1820 | | |
1820 | 1821 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
| 47 | + | |
47 | 48 | | |
48 | 49 | | |
49 | 50 | | |
| |||
977 | 978 | | |
978 | 979 | | |
979 | 980 | | |
980 | | - | |
981 | | - | |
982 | | - | |
983 | | - | |
984 | | - | |
| 981 | + | |
| 982 | + | |
| 983 | + | |
985 | 984 | | |
986 | 985 | | |
987 | 986 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
23 | 22 | | |
24 | 23 | | |
25 | 24 | | |
26 | 25 | | |
27 | 26 | | |
28 | 27 | | |
29 | 28 | | |
| 29 | + | |
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| |||
160 | 160 | | |
161 | 161 | | |
162 | 162 | | |
163 | | - | |
164 | | - | |
165 | | - | |
166 | | - | |
167 | | - | |
168 | | - | |
169 | | - | |
170 | | - | |
171 | | - | |
172 | | - | |
173 | | - | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
174 | 168 | | |
175 | 169 | | |
176 | 170 | | |
| |||
0 commit comments