[robobun]

Merge after oven-sh/bun#47042 (it pins oven-sh/mimalloc#62), or a WebKit bump fails to link. Tests: oven-sh/bun#47013.

Problem

• AutomaticThread and Atomics.wait call mi_on_thread_idle() once, then block (AutomaticThread.cpp:277, WaiterListManager.cpp:152). One call leaves the free blocks of 96 KiB and up that the thread just used.
• After 6 wasm compiles with the modules collected at once, 8 threads keep 28 to 41 MiB for 10 s.

Fix

• WTF::IdleFreeMemoryRelease holds mimalloc's rule for both callers. While the allocator holds back memory in the thread's own pages, the thread releases again 100 ms later, at most 4 times.
• Correct: a notify ends each 100 ms wait, and each wait counts toward the thread's timeout.
• Verified on a release bun: later bursts end at 2.7 to 4.5 MiB, Atomics.wait at 2.6 to 3.3 (19 to 21 before).

Background

• mimalloc's idle sweep leaves those blocks when their page was used within the last two sweeps.
• Considered mi_on_thread_idle_pending() as the key. It is also true for other threads' pages: a busy JS thread took 2564 faults per round, not 69.

Downsides

• Memory freed into a thread's pages after its last release stays until it exits: 17 to 28 MiB for modules collected 250 ms after the compile (30 to 42 before).
• A wake-up during the second release of Atomics.wait waits for it: 45 of 96 timeouts came over 2 ms late after large frees (4 of 96 before).
• Idle compiler threads hold mimalloc's 4 MiB floor for free large blocks until they exit.

Notes

The order.

1. theap: mi_on_thread_idle_pending_own, for a thread that comes back only for its own large pages mimalloc#62 adds mi_on_thread_idle_pending_own.
2. deps: mimalloc to oven-sh/mimalloc#62 (mi_on_thread_idle_pending_own) bun#47042 moves bun's mimalloc pin to it, alone.
3. This PR merges.
4. Bump WebKit (oven-sh/WebKit#911 preview): an idle JSC thread releases again while mimalloc holds its large blocks back bun#47013 moves bun's WebKit pin and adds the tests.

This repository's CI cannot see a missing export: the jsc shell and testFFI link ExternalMimallocShims.cpp, which answers false there, so they keep one release per idle period. A bun that links this WebKit against a mimalloc without the export fails on an undefined mi_on_thread_idle_pending_own in libbmalloc.a.

#585 changes the two waitUntil calls of the same loop in waitSyncImpl to the monotonic clock. This PR does not change those calls. The hunks are next to each other, so the PR that merges second needs a rebase of that hunk. Both changes stay as they are.

What one call leaves. mimalloc moves a process-wide sweep epoch at most once per 100 ms, and only a sweep moves it. An idle sweep leaves the free blocks of a large page that was allocated from in the current epoch or in the one before. A thread that never made a release has its large pages freed at once, so the first burst of compiles in a process returns most of its memory with one call (0 to 12 MiB stay). Each later burst does not. mi_on_thread_idle_pending() tells the caller that a sweep left such blocks, and mimalloc.h asks a caller that blocks for good to call again 100 ms later, 3 more times at the most. bun's own idle threads do that since oven-sh/bun#42668. These two callers did not: the single call was written before mimalloc got that rule.

Why the own-pages answer. A sweep also passes the abandoned pages of the heap (a large page is abandoned when it is full), which are the pages that other threads allocate from. mi_on_thread_idle_pending() is true for those as well. With that key the GC helper threads came back after every collection (84 second releases after 97 first ones), every such sweep moved the epoch, and a JS thread that takes 10 MiB of 256 KiB buffers again every 300 ms without parking found them given back: 2564 minor faults per round where it takes 69. mi_on_thread_idle_pending_own() is true only for pages the calling thread owns.

What still follows up on the any-page answer. mimalloc's mi_on_thread_idle_start()/_end() pair (bun's thread pool and event loops park with it) and bun's Windows event loop. This PR does not change them. The self-review measured what that leaves: with one thread-pool task or one Worker timer in every round, the JS thread of the buffer loop still takes 2113 to 2561 faults per round (64 without).

The two callers.

• AutomaticThread::start: while a release is due and more than 100 ms of the timeout are left, the thread waits 100 ms, leaves on a notify, and releases with the lock dropped and m_isReleasingFreeMemory set, as before. poll() still runs once per notify. A thread with m_timeout of 100 ms or less does nothing new. The budget is reset only before work().
• WaiterListManager::waitSyncImpl: the release runs with the list lock dropped, as before. The next one is due 100 ms after the call returned. The loop tests isOnList() and termination under the lock again after each release.

Which threads follow up. Wasm compiler threads after a compile, and a thread in Atomics.wait that freed large blocks right before it. GC helper, collector and warm-up threads do not: 0 of 109 idle releases in 14 rounds of the buffer loop, and 0 of 1510 in the self-review's census. JIT worklist threads are the one other class that does: in that census they were told "held back" in 72 of 287 first releases and made 41 follow-ups. They return 7.5 to 11 MiB after very large compiles, and 0 to 3 MiB (inside the spread) in prettier, an SSR server, webpack and a TypeScript transpile loop. No test covers that arm.

Measurements. Release builds of bun for Linux x64, 12 CPUs, load average under 2. "This PR" is bun c643ea8c2a with WebKit 982277c9dd3e (bun's pin) plus this commit, as WebKit's CI built it, and mimalloc 639adf50. "Before" is the same bun with WebKit 982277c9dd3e. A line marked (old pin) was measured on 680ef47cec2c plus this commit and was not repeated.

• Lowest RSS growth within 1 s after each later burst of 6 compiles, 8 compiler threads, 5 runs of 3 later bursts: 2.7 to 4.5 MiB (27.9 to 41.2 MiB before, for 10 s). About 3 MiB of what stays is under mimalloc's floor for free large blocks.
• bun's test (compile-rss.test.ts: the first burst and two later bursts have to end under 10 MiB): 350 of 350 runs pass on 12, 2 and 1 CPUs and beside 12 and 24 busy loops (0 of 40 before). A later burst is under 10 MiB 219 ms after the collection (p90 233 ms).
• The loop of WebAssembly.compile retains ~10 MB per wasm compiler thread; RSS grows with core count and is never released bun#41438 (30 compiles, each result dropped, and after every 5 a collection and a 200 ms sleep): ends at +11 to +16 MB (+25 to +40 MB before). A 200 ms sleep samples the second release half way, so the samples on the way vary: +11 to +25 MB (+14 to +45 before). With a 300 ms sleep: +9 to +11 MB (+39 to +43 MB before).
• Atomics.wait after 46 MiB of 96 to 480 KiB blocks were freed on that thread: 2.6 to 3.3 MiB stay, reached 258 to 309 ms into the wait (18.8 to 20.8 MiB for the whole wait before).
• A burst of 6 compiles 1 s after the one before: 16.8k to 18.4k minor faults (12.4k to 14.2k before).
• A thread notified within 100 ms (old pin): 24 instructions from m_isWaiting = true to the next poll, 2 of them alignment nops (23 before), 4 conditional branches (4), 2 calls (2), no allocator call.
• Follow-up releases under 32 concurrent 256 KiB request bodies, 20 rounds (old pin): 170 and 185 idle sweeps (174 and 178 before), 0 follow-ups.
• JS thread that takes 10 MiB of 256 KiB buffers again every 300 ms (old pin): median 69 minor faults per round in 3 of 3 runs (69 before, 2564 with the any-page key). With 100 MiB per round: 5189 (5189 before, 25605 with the any-page key). A round in which a JIT compile ends takes 1612 to 1617 faults (68 to 81 before): the JIT thread comes back for its own pages, and its sweeps move the epoch. With a compile in every round the two builds do not differ (median 79 and 78 to 79, and 1 to 3 rounds of 8 over 1500 faults in both).
• Wake-ups in the 1.5 s after a burst (old pin): 22 to 25 for 11 wasm compiler threads (11 before), 1.17 to 2.06 ms CPU (0.80 to 1.01).
• Size (old pin): .text +64 bytes, .data and .bss unchanged.

A server under load. Express 4.18 with React 18 server rendering, ab -c 32, 5 runs of 6000 requests on each build, the builds in turn. The spread between runs is larger than the difference between the builds.

build	requests/s	p99	minor faults per request	RSS at the end
before	183 to 199 (median 186)	257 to 439 ms (median 328)	3.4	144 MB
this PR	176 to 208 (median 198)	241 to 475 ms (median 269)	3.6	147 MB

Costs in Atomics.wait. Every wait of more than 200 ms can get a second release. The callers are Atomics.wait and wasm memory.atomic.wait.

• A wake-up waits for a release that is under way. A timeout that falls right after the start of the second release came more than 2 ms late in 45 of 96 waits (4 of 96 before). The waits followed 22 to 132 MiB of large frees in six patterns, 16 waits each. The median lateness per pattern is 0.2 to 3.1 ms (0.14 to 0.18 ms before), and the longest is 22 ms. After the first release a timeout is 0.11 to 0.14 ms late at the median on both builds. AutomaticThread passes over a thread that is in a release. Atomics.wait has no other thread to wake.
• A worker that takes 10 MiB of 256 KiB buffers again after each 250 ms wait takes 1984 faults per round (1024 before). With 44 MiB of 96 to 480 KiB buffers: 7809 to 8928 at the median (6712 to 6744 before). The cost is bounded by what the second release returns, which is not the whole working set.

The floor. mimalloc leaves up to 4 MiB of free large blocks alone, process-wide, for the pages that were used last (purge_holes_large_floor). After a burst those are pages of the compiler threads, and a thread gives its share back only at its next sweep or when it exits. Probe: a JS thread takes 3 MiB of 256 KiB buffers every 500 ms and parks in between, and a burst of 6 compiles runs once. With this change it faults the buffers in again (768 faults) on each of the 19 uses until the compiler threads exit, in 6 of 6 runs. Before, it did so in 4 of 6. On the previous pin the counts were 13 of 13 and 5 of 13. The scavenger gives the share of a parked thread back after purge_holes_large_floor_epochs. Nothing does that for a thread that calls mi_on_thread_idle().

Late frees, and what was weighed for them. Memory that another thread frees into an idle thread's pages after that thread's last release stays until the thread works again or exits. With the modules held for 250 ms after their compile, 17 to 28 MiB stay (30 to 42 MiB before). Three mechanisms would return it. None is in this PR.

• A shorter idle timeout for wasm compiler threads (built, 1 s, behind an environment variable): the late frees are back about 1.2 s after the collection. The threads do not reach a timeout at the 200 ms sleep of WebAssembly.compile retains ~10 MB per wasm compiler thread; RSS grows with core count and is never released bun#41438. Every compile that comes more than 1 s after the one before starts all threads again: about 28 ms of CPU more per compile with 31 threads.
• The thread hands its heap to the allocator when it goes idle, as a thread that exits does (built, mi_thread_done() after the last release): late frees stay at about 0 MiB and the floor is free after one round. test/js/node/worker_threads/worker_heap_snapshot_gc.test.ts in bun crashes in 6 of 6 runs with it (0 of 6 without).
• A release granted when a wasm module is destroyed (not built, the self-review emulated it under gdb): with the budget granted again by hand, 20.4 to 28.2 MiB fall to 1.2 to 4.5 MiB within 250 ms in 6 of 6 rounds. It needs a wake-up in AutomaticThreadCondition that neither clears m_isWaiting nor restarts the timeout, and a call from the module's destructor. That is a change of its own.

Not removed.

• A notify without work restarts the full timeout and makes no new release.
• A young abandoned large page is not followed up by anyone. The next sweep of any thread that finds it old enough takes its free blocks.
• The 100 ms interval is a copy of mimalloc's default purge_holes_min_interval, now in one place (IdleFreeMemoryRelease::intervalInMilliseconds).
• Not measured without the change: macOS, Windows, aarch64. bun's CI ran the tests of Bump WebKit (oven-sh/WebKit#911 preview): an idle JSC thread releases again while mimalloc holds its large blocks back bun#47013 with the change on 9 lanes (numbers in that PR).

Tests. The tests are in oven-sh/bun#47013, because the effect needs bun's mimalloc: test/js/bun/wasm/compile-rss.test.ts (the first burst and later bursts), test/js/web/atomics.test.ts (retention in Atomics.wait, and a notify that arrives during the second release), test/js/bun/jsc/heapStats-mimalloc.test.ts (GC helper threads leave the buffers of a busy thread). Mutations of this change, each built: without idleRelease.reset() the wasm test fails 10 of 10 runs. Without the isOnList() test after a release the second notify test fails 6 of 6 runs. With mi_on_thread_idle_pending() as the key the helper test fails 16 of 16 runs. The three mutants are builds of the previous pin.

The branches. This branch is based on 40b30d27fb and merges into main (bd40ac037e) without a conflict. robobun/1a8bcc9f/idle-thread-releases-again-on-982277c9 is bun's pin 982277c9dd3e plus this commit (61e155bcefa3). Its CI run published the prerelease that oven-sh/bun#47013 pins until this merges.

[github-actions]

Preview build of 3572343: autobuild-preview-pr-911-3572343f