feat(export): linear audio mixdown for full-timeline export (refs Phase5/#112)#117
Merged
Conversation
…se5/#112) Extend the #112 video export spine with a linear audio mixdown, replacing the dead `push_audio` side-channel with a wired decode → mix → mux path. - opentake-media: new pure `encode::mix` module — lay each clip's mono f32 PCM at its frame-derived sample offset, apply a per-sample gain envelope, sum overlapping clips, hard-limit to [-1,1]. `mono_f32_to_s16le` for the mux wire format. Fully unit-tested, no ffmpeg/domain deps. - opentake-media encoder: `finish` now mux's a supplied mixed buffer via a second ffmpeg pass (`-c:v copy` + AAC/LPCM `-shortest`), restoring the video-only file if the mux fails. `mux_args` is pure and unit-tested. - export.rs orchestrator: decode every audio/video clip's source window with the reused `extract_pcm` (mix rate, mono f32), build the `volume_at` gain envelope, skip muted tracks and audio-less sources, mix and push to the encoder. No-audio timelines still produce the same video-only output. - export_integration.rs: ffmpeg+GPU-gated test builds a sine-audio fixture, exports, and asserts ffprobe reads back an AAC audio stream + cleaned temps; video-only test now asserts no audio stream leaks in. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
appergb
approved these changes
Jun 23, 2026
appergb
left a comment
Owner
There was a problem hiding this comment.
审核通过 ✅(自动审核流程 · 主控亲审,子 Agent 因 API 529 过载)
- #112 导出 spine 的线性音频混音扩展,真接通原 dead
push_audio。 - mix.rs 纯混音正确:帧→样本偏移铺放、逐样本 volume_at 增益包络、叠加、硬限幅 [-1,1]、mono f32→s16le;12 单测覆盖空/单/偏移/重叠/溢出双轨/增益/长度校验/输出长度/编码与 clamp。
- encode finish() mux:第二趟 ffmpeg(
-c:v copy+ AAC/LPCM +-shortest);mux 失败回退视频-only 文件(不丢导出)、临时件 best-effort 清理;mux_args/sibling_temp 纯函数有测试。 - export.rs 编排:解码音/视频 clip 源窗(extract_pcm,mono f32 @48k)→增益包络→混音→push;跳过静音轨/无音源/NoTrack→静音;视频路径未改(-18 仅文档/import),无音频仍产出与 #112 相同视频-only 文件。
- 集成测试:加 440Hz 正弦 fixture 断言 AAC 轨 + 视频-only 断言不漏音轨 + 临时件清理。不碰 #91 区,CI 双绿。
follow-up(非阻塞):project_clip_audio ~60 行略超 50 规约;本切片仅线性混音(无重采样曲线/pan/立体声/动态),按设计后续。
appergb
added a commit
that referenced
this pull request
Jun 23, 2026
docs: 导出线性音频混音(#117)归档 + Phase 5 进度
H-Chris233
pushed a commit
to H-Chris233/OpenTake
that referenced
this pull request
Jun 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Extends the #112 full-timeline video export spine with a linear audio mixdown, turning the dead
push_audioside-channel into a wired decode → mix → mux path. A timeline with audio now exports an mp4 with a real AAC track; a timeline without audio produces the same video-only file as before.How
opentake-media— new pureencode::mixmodule: lays each clip's mono f32 PCM at its frame-derived sample offset, applies a per-sample gain envelope, sums overlapping clips, and hard-limits to[-1.0, 1.0].mono_f32_to_s16leproduces the mux wire format. No ffmpeg/domain deps — fully unit-tested.opentake-mediaencoder:finishnow mux's a supplied mixed buffer via a second ffmpeg pass (-c:v copy+ AAC/LPCM,-shortest). On mux failure it restores the video-only file rather than losing the export.mux_argsis pure and unit-tested; temp artifacts are cleaned up.export.rsorchestrator: decodes every audio/video clip's source window via the reusedextract_pcm(mix rate, mono f32), builds thevolume_atgain envelope, skips muted tracks and audio-less sources, mixes, and pushes to the encoder.export_integration.rs: ffmpeg+GPU-gated test builds a 440 Hz sine-audio fixture, exports, and asserts ffprobe reads back an AAC audio stream plus cleaned-up temps. The video-only test now also asserts no audio stream leaks in.Scope
Linear mixdown skeleton only — no resampling curve, pan/stereo field, or dynamics (follow-ups). H.264/.mp4 path; H.265/ProRes still gated as before.
Verification
cargo fmt --allcleancargo clippy -p opentake-media -p opentake-tauri --all-targets -- -D warningscleancargo test -p opentake-media(24 encode/mix unit tests) greencargo test -p opentake-tauri export— 10 unit + 2 ffmpeg+GPU-gated integration tests green (audio test genuinely ran, not skipped)Hard constraints honored: Tauri boundary returns
Err(String);opentake-domainuntouched (zero new deps); mixdown is pure logic inopentake-media; serde fields unchanged; unit tests under#[cfg(test)].🤖 Generated with Claude Code