Skip to content

Vulkan optimization, scene-reading transparency and a TAA overhaul - #217

Merged
CanReader merged 37 commits into
stagingfrom
vulkan-optimization
Oct 25, 2026
Merged

CanReader merged 37 commits into
stagingfrom
vulkan-optimization

Conversation

@CanReader

Copy link
Copy Markdown
Owner

What this changes

Brings the vulkan-optimization branch (34 commits) into staging: the Vulkan performance pass, scene-reading transparent draws on Vulkan and OpenGL, and a TAA overhaul with RCAS sharpening.

Why

  • Performance: no upload stalls, a persisted pipeline cache, shadow and lighting pass savings, fewer shadow push constants, and the validation layer loads only in Debug or with --validate.
  • Correctness: fixes for leaks, semaphores, shadow matrix pairing (Vulkan and OpenGL), and TAA/forward pass ordering.
  • Scene-reading transparent formats (VertexLayoutDesc::transparentReadsScene), which SleakCraft's water uses.
  • TAA: reprojection in double precision. In float32, inverse(VP) * VP drifted from identity far from the world origin and smeared a still camera. Also YCoCg variance clipping, a reactive mask written by transparent draws, and RCAS sharpening after TAA (Application::SetSharpenStrength, GraphicsConfig::sharpenStrength).

SleakCraft's matching changes are in CanReader/SleakCraft#87.

How it was verified

Built and run with SleakCraft on Linux (Debug, Vulkan validation layer on, no validation errors). Deterministic -capture frames of a saved world, still and with --autofly: error against an 8-frame supersampled reference went from 3.7% to 1.1%. The OpenGL backend ran 900 frames with no shader errors.

The forward pass discarded depth that TAA and SSR sample afterwards, and
neither the forward nor the TAA render pass declared an outgoing
dependency, so the post passes could read hdrScene, depth and the TAA
result before those writes landed. Store forward depth and add
0->EXTERNAL dependencies to both passes.
The map was only transitioned to a readable layout on creation and after
a resolution change, so the lighting pass sampled undefined depth until
the first shadow pass ran. Clear it to far depth through the shadow
render pass instead.
The previous view-projection starts zeroed, so frame 0 reprojected
through a zero matrix and could write NaN into the history that the
next frames then blend from.
--gpuprofile brackets every recorded pass with timestamp queries, logs
avg/max ms per pass every 2 s and adds GPU_<pass>_ms columns to --bench
CSVs. -capture <file> -capture-frame <n> runs on a fixed 1/60 s step
with user input dropped, then writes frame n without UI as a PPM and
exits, so renderer changes can be checked pixel for pixel.
The layer was enabled on every run whenever it was installed, Release
included, so every Vulkan call paid validation overhead and --validate
did nothing. It now loads in Debug builds or with --validate, and
--syncval also turns on synchronization validation.
Every voxel caster shares one transform, so the shadow pass pushed the
same 128 bytes for each draw. Push once per shadow pass while the cached
product is unchanged; all shadow pipelines share one layout, so the
values persist across pipeline binds.
Destroying any buffer that had ever been batch uploaded forced a
synchronous submit and fence wait while a batch was open, stalling the
CPU on the previous frame thousands of times per minute while streaming.
Deferred deletion already covers that lifetime, so the flush is gone and
an unsubmitted batch is discarded at shutdown. Staging now comes from a
resident VMA pool so longer-lived staging no longer churns 64 MB blocks.
Constant buffers move from uncached BAR memory to cached host memory
because the CPU reads them on every bind. A frame whose acquire fails no
longer submits its upload batch or advances the deletion clock.
Every pipeline was compiled without a VkPipelineCache. Create one at
startup from cache/vulkan_pipelines.bin when its header matches this
device and driver, pass it to every pipeline and to ImGui, and save it
on shutdown through a temp file and a rename, skipping the write when
nothing changed. With the driver shader cache off, pipeline creation at
startup drops from about 450 ms to 3 ms once the cache is warm.
Custom-format variants were compiled mid-frame on first use, including
the forward variant the deferred path never binds, and every format
switch re-ran the builder under the registry lock. Look variants up per
handle, build the ones the active path needs when a format is
registered or the GBuffer is rebuilt, and keep lazy creation as a
fallback. Startup no longer compiles the forward and skinned GBuffer
pipelines twice, and the shared pipeline layout is created once instead
of leaking on every rebuild.
The device enabled every supported feature, robustBufferAccess
included, which bounds-checks buffer and vertex fetches. Enable sampler
anisotropy and full 32-bit indices, and robustBufferAccess only when
--validate is passed.
The forward, skybox and debug line pipelines described the bone ID and
weight attributes their shaders never read, which raised eight
validation warnings at startup. Declare locations 0 to 4 only.
Rebuilding swapchain resources recompiled the forward, skinned, debug
line and skybox shaders without freeing the old modules, and an MSAA
change also leaked the skybox and ImGui descriptor pools. Reuse them
instead. Texture descriptor sets were never freed, so texture churn
exhausted their pool; a destroyed texture now hands its sets and GPU
objects to a queue that frees them once the last submission that could
use them has finished.
RecreateSwapChain resized the image fence table but not the acquire and
render-finished semaphores, so a swapchain with more images indexed
past their end. Grow both lists to the new image count and clear the
fence table, also after an MSAA change.
A suboptimal present rebuilt the swapchain and every size-dependent
resource behind a device wait, even when the surface size had not
changed. Check the surface extent first. Out-of-date presents and
capture mode behave as before, and resize events that keep the same
extent are skipped too.
TAA copied its resolved image back into the HDR scene every frame, a full
screen RGBA16F copy with four layout transitions. SSR, bloom and the
composite now sample the TAA target written this frame and fall back to the
HDR scene when TAA is off. The capture is bit-identical.
The composite sampled the SSR and bloom images on every pixel even when both
were disabled and bound to black fallbacks. A push constant now tells it
whether SSR ran, and bloom is only sampled with a non-zero strength. The
output is unchanged.
The 3x3 closest-depth search did ten texel fetches; four depth gathers cover
the same texels, visited in the original order with the same strict compare.
The current color and its 3x3 clamp neighborhood are read with texelFetch,
and the history taps use an explicit LOD after the non-uniform early outs.
The capture is bit-identical.
… pass

The lighting pass reads depth and the GBuffer targets with texelFetch at
the fragment's pixel, takes the shadow PCF taps at an explicit LOD, and
skips the 16-tap PCF where the back-face smoothstep zeroes the shadow term
anyway. The capture is bit-identical.
The SSAO term was only used in the IBL branch, and IBL is never enabled on
Vulkan, so SSAO had no visible effect. The hemisphere ambient now takes the
baked AO times SSAO remapped to [0.5, 1], the same way the GL lighting pass
applies it. With SSAO off the bound fallback is white and the output is
unchanged.
Vulkan binds 1x1 stub IBL maps and never bakes them, so the lighting pass
always takes the hemisphere ambient path. Requesting IBL was silently
ignored; it now logs a one-time warning instead.
When vkAcquireNextImageKHR failed, BeginRender returned before
ImGui::NewFrame while the game still issued UI calls, which tripped the
WithinFrameScope assert in Debug. Those frames now start a UI frame and
EndRender closes it without drawing. Normal frames are unchanged.
Each of the 16 PCF taps evaluated sqrt, sin and cos of its own Vogel angle.
The unrotated disk is now a constant table and the per-pixel rotation is a
single sin/cos pair applied to every tap. The capture is bit-identical.
The shadow pass is recorded in BeginRender with the light matrix staged
during the previous update, but the light UBO was then filled with the
newly computed matrix, so lighting and water sampled the map one snapped
frustum step off while the camera moved. The UBO's LightVP now carries
the matrix of the last recorded shadow pass.
LightManager::GetShadowLightVP returns the view-projection last staged on
the renderer, which the next BeginRender commits for its shadow pass, so
games can cull shadow casters against the light's box.
Each BindBoneBuffer copied into the same per-frame bone UBO at offset 0,
so every skinned draw of a frame, including the skinned shadow casters
recorded earlier in BeginRender, rendered with the last palette written.
Set 1 is now a dynamic uniform buffer backed by a per-frame ring of 16 KB
slots that grows in 32-slot chunks with their own buffer and set; each
bind claims a slot and binds it by dynamic offset, and pass starts bind
slot 0, which holds an identity palette.
The shadow map is rendered in BeginRender with the light matrix staged
during the previous update, but the shadow UBO then received the newer
matrix, so shadows slid by one snapped frustum step while moving. Keep
the matrix used for the last shadow pass and upload that one instead,
matching the Vulkan fix.
Shadow depth bias, push constant size, bone palette size, the capture
default frame, the staging probe size and the FNV hash constants are now
named, and the pipeline cache header is read through the offsets of
VkPipelineCacheHeaderVersionOne instead of hand-counted byte offsets.
VertexLayoutDesc gains transparentReadsScene. A format that sets it gets a
snapshot of the scene color, depth, sky cubemap and a SceneReadUBO camera
block in its transparent pass, so water and glass can refract and reflect.
The first scene-reading transparent draw of a frame ends the forward pass,
copies the HDR scene and depth into snapshot images and resumes the pass,
so frames without such draws pay nothing. Set 3 gains the snapshot, sky
cubemap and SceneReadUBO bindings, and the UBO carries the TAA jitter so
the forward draw lines up with the jittered GBuffer depth.
The first scene-reading transparent draw of a frame copies the default
framebuffer into a texture and binds it with the GBuffer depth, the sky
cubemap and a SceneReadUBO. Rendering stays in the default framebuffer, so
no offscreen HDR target is introduced.
@CanReader
CanReader force-pushed the vulkan-optimization branch from b9ca883 to 2b2de74 Compare October 25, 2026 15:25
@CanReader
CanReader force-pushed the vulkan-optimization branch from 2b2de74 to 92180f8 Compare October 25, 2026 15:26
@CanReader
CanReader merged commit 690f50e into staging Oct 25, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant