fix(preview): 单时钟播放引擎 + 实时拖拽预览(渲染管线按上游单表面重写,#142)#144
Open
appergb wants to merge 14 commits into
Open
Conversation
added 7 commits
June 24, 2026 17:30
Rewrite the timeline preview/playback pipeline to upstream's single-surface model (VideoEngine.swift / PreviewView.swift): the engine owns playback, the view only renders. Removes the dual-surface / dual-clock stopgap that caused the reported pause-twitch, non-live scrub, and stutter. - previewEngine.ts: app-level single clock + shared element registry. One rAF authority; runs only while playing or scrubbing; auto-pauses at end. Replaces the playbackClock refcount + usePlaybackTicker + the in-component rAF. - Surface state machine = browser equivalent of upstream exact/interactiveScrub: PLAY and SCRUB use the cheap live <video>/<audio> stack (scrub live-seeks to the frame); SETTLED uses the high-fidelity Rust GPU composite. Drops the 140ms trailing debounce — the expensive composite now fires once, on settle. - Wire the long-unused isScrubbing through the ruler scrub (TimelineContainer, scrub branch only — editing gestures untouched) and the Preview scrub bar. - TimelinePlaybackLayer becomes a pure renderer (registers elements, no clock). - advancePlayhead extracted as a pure, unit-tested helper. Scope: full transform/crop/text compositing DURING playback still awaits the streaming engine (#53); this lands the faithful single-clock structure and fixes the three reported playback bugs. Rust composite_frame unchanged. Bump version 0.1.0 -> 0.1.1. tsc clean; 56 web tests pass (+4 engine tests).
…142) Two preview regressions: - Pause jumped back to the start. The composite targeted currentFrame, which the engine freezes at the play-start frame during playback (it only advances activeFrame), so the settled composite fetched frame 0. Target activeFrame — the live playhead — so the composite is the frame you paused on. useTimelineFrame now also returns which frame its url was rendered for; the composite is shown only once it has decoded the CURRENT frame, and the <video> backdrop (frozen on the pause frame) holds the picture until then — so pausing is correct instantly with no stale-frame flash. - Scrub-bar progress fill rendered as a tall cream bar down the preview's left edge: its absolute fill/handle escaped the (unpositioned) track. Add position:relative to contain them. Bump 0.1.1 -> 0.1.2. tsc clean; 56 tests pass.
Codex-authored changes addressing the upstream-consistency audit, brought onto the working branch. Compiles (tsc clean) and all 56 web tests pass. - Clipboard: copy/cut/paste clips (⌘C/⌘X/⌘V) + clipboardStore + empty toast. - Clip right-click context menu (ClipContextMenu) + editActions wiring. - Snap: stronger targets / includePlayhead / tolerance tweaks (snap.ts). - Clip helpers + types: new fields and edit helpers (clip.ts, types.ts). - Timeline interactions: drag/hit-test/canvas refinements (TimelineContainer, hitTest, timelineCanvas, clipRenderer incl. link-offset badge). - uiStore: transient toast; media-folder setter. HoverButton cursor: pointer. - Preview: position:relative on the stage (contains the scrub-bar fill, audit #10). Authored by Codex; reviewed for compile + tests only. Per-item correctness vs upstream is tracked in the audit issues filed alongside.
The transport play/pause button retained keyboard focus after a mouse click. On WebKit (Tauri WebView) a focused <button>'s Space activation fires on keyup, which the window keydown handler's preventDefault does not cancel — so Space toggled play TWICE (focused-button onClick + global shortcut), leaving isPlaying net-unchanged (the button never flipped to the paused state) and churning the engine (spurious playhead jump). onMouseDown preventDefault stops the button taking focus on click; Space is then handled solely by the global shortcut. onClick is unaffected; Tab focus (accessibility) is preserved.
added 7 commits
June 24, 2026 18:55
ROOT CAUSE (thanks to the Codex audit): the paused-state GPU composite <img> (position:absolute; inset:0) had no pointerEvents:none, so it intercepted every mouse event over the preview area — the play/pause button "couldn't be pressed" and looked stuck (the icon WAS updating; the overlay just blocked the click and covered it). The state machine was never wrong (verified: togglePlay toggles isPlaying cleanly). - Composite <img>: pointerEvents:none. - <TimelinePlayback> surface: pointerEvents:none (same class of display-only layer). - Revert the earlier focus-prevention guess on HoverButton (wrong hypothesis; togglePlay was confirmed single-call/correct). The togglePlay rewind-from-end (uiStore) is intended replay behaviour and only fires at the timeline end; the perceived "jump to start" was a downstream effect of not being able to pause mid-timeline, resolved by the click fix.
Pause twitch + late-icon (#149, two-agent root-cause): the <video> backdrop freezes on its real-time decode frame F_video, but the engine's activeFrame lags by a tick, so the settled GPU composite was fetched for an EARLIER frame and, when it painted over the frozen video, the picture twitched back. The icon felt late because that twitch render was the one the user noticed. Fix: on the pause transition, snap activeFrame to the visual element's frozen frame (frameForSourceTime of its currentTime), so composeFrame == F_video and the composite paints the same picture — no jump, pause reads instant. Also adds docs/EDITING-ENGINE-PLAN.md: the editing-engine implementation map (ops layer is 1:1 with upstream; gaps are in the wiring layer), the linked-audio behaviour (upstream design — A1/A2 for video+audio is correct; "no-audio video" is a stale-cache or upstream-1:1-vs-deviation question for the user), and the A→B→C close-out plan over issues #145/#146/#147/#86/#87/#98. No probe.rs change: its channels-missing default is intentional (audio-only files report no channels in the test mock; flipping would drop real audio).
Owner
Author
现状交接(诚实说明,勿误判为「渲染已修好」)本 PR 已包含:单时钟播放引擎重写、状态机选面 + 实时拖拽、删 140ms 防抖、剪辑引擎工作整合(Codex,审计项)、播放层 但渲染/播放引擎在双表面模型上仍未根治 —— 用户实测 0.1.5 仍有:中途卡死、渲染无效、暂停抽搐。根因是「双表面 + 双时钟」与上游单表面 AVPlayer 本质不兼容,详见 #151(P0,含架构分析)。 建议: 本 PR 的增量修复可作为脚手架参考合入(剪辑接线/单时钟/实时拖拽都是净改进),但渲染保真 + 不卡死 + 不抽搐必须靠 #53 流式引擎重写,不要再在双表面上叠补丁。详见 docs/EDITING-ENGINE-PLAN.md。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景:整套播放/预览架构对照上游重写
用户报告时间轴预览三大问题:暂停抽搐、拖动不实时(只在松手才出画面)、卡顿。对照上游 palmier-pro(
VideoEngine.swift/CompositionBuilder.swift)逐行分析后确认根因是架构层面的"双表面 + 双时钟"拼接,而非局部 bug。旧实现(双表面 + 双时钟)
<video>/<audio>栈,音频当 master 时钟,只应用 opacity + 层级。composite_frame→ wgpu → PNG → base64 →<img>,每帧 spawn 一个 ffmpeg 进程,外面套 140ms 尾随防抖。usePlaybackTicker+TimelinePlaybackLayer内部 rAF)靠mediaClock引用计数仲裁。三个症状各自归因:暂停抽搐 = 松手前残留的 rAF tick 把未来帧写进播放头;拖动不实时 = 140ms 防抖;卡顿 = 请求/响应式逐帧合成(ffmpeg spawn + PNG 编码)。
本 PR 做了什么:对齐上游单表面模型
上游是一个
AVPlayer+ 一个表面,play/pause/seek/scrub 全走同一表面,只有.exact/.interactiveScrub两种 seek 模式。本 PR 把这套忠实映射到浏览器:<video>/<audio>栈实时播放<video>栈实时 seek 到帧.interactiveScrub松容差.exact帧精确改动清单
previewEngine.ts—— app 级单一时钟引擎 + 共享元素注册表(对齐上游 VideoEngine 是 app 级、PreviewView 只渲染)。一个 rAF 权威时钟,只在播放||拖拽时运行,到尾自动暂停。替代playbackClock引用计数 +usePlaybackTicker+ 组件内 rAF。Preview.tsx—— 状态机选面;删掉 140ms 防抖;静止时才取一次合成帧(取已提交的currentFrame)。TimelinePlaybackLayer.tsx—— 降级为纯渲染器(挂载/注册元素、按状态控制可见性,不再持时钟)。isScrubbing(此前定义了但全程没人用):标尺(TimelineContainer仅 scrub 三处,move/trim/marquee/razor 等剪辑手势完全未碰)+ Preview 进度条 pointerdown/up。timelinePlayback.ts—— 抽出纯函数advancePlayhead(可单测)。playbackClock.ts+usePlaybackTicker.ts。三个症状如何被消除
if(!isPlaying)return读最新 state),松手前残留 tick 不再写未来帧。<video>seek(正是 single-media 预览那条已验证流畅的便宜路径),昂贵合成只在松手后取一次。净 +133 / −313 行(删掉双时钟 + 防抖 cruft,合并进一个干净引擎)。
Scope 边界(诚实说明)
composite_frame未改(静止时单次取帧足够)。测试
tsc -b= 0 错误;vitest run= 56 passed(含新增 4 个advancePlayhead引擎单测)。Test plan