[Cherry-Pick][Scheduler] Defer block recycling to accelerate LRU node freeing

[liyonghua0910]

Motivation

💡 If this PR is a Cherry Pick, the PR title needs to follow the format by adding the [Cherry-Pick] label at the very beginning and appending the original PR ID at the end. For example, [Cherry-Pick][CI] Add check trigger and logic(#5191)

💡 如若此PR是Cherry Pick，PR标题需遵循格式，在最开始加上[Cherry-Pick]标签，以及最后面加上原PR ID，例如[Cherry-Pick][CI] Add check trigger and logic(#5191)

Cherry-pick #7885 to release/2.6

Modifications

• Defer recycle_gpu_blocks calls inside the LRU freeing loop to a single batch call after the loop, reducing the overhead of repeated heap operations.
• Add defer_recycle parameter to _handle_free_gpu_node_without_cpu to support deferred block recycling.
• Fix the LRU leaf node freeing logic: disconnect the child node from its parent first, then check whether the parent should be added to the LRU heap, avoiding duplicate freeing.
• Add warning logs to help diagnose duplicate node issues in the LRU heap.

Usage or Command

No additional configuration required. The optimization takes effect automatically.

Accuracy Tests

Only affects KV Cache block recycling timing, no impact on model output accuracy.

Checklist

• Add at least a tag in the PR title.
  • Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
  • You can add new tags based on the PR content, but the semantics must be clear.
• Format your code, run pre-commit before commit.
• Add unit tests. Please write the reason in this PR if no unit tests.
• Provide accuracy results.
• If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

[paddle-bot]

Thanks for your contribution!

[PaddlePaddle-bot]

🤖 Paddle-CI-Agent | ci_status_monitor | 2026-05-22 21:49:51

CI报告基于以下代码生成（30分钟更新一次）:

• PR commit: 425cf62
• Merge base: b562b8d (branch: release/2.6)
• 查看完整 Diff
• CI 详情

---

1 任务总览

当前 Required 任务 9/10 通过，仍有 1 个 Required 任务失败（Approval，需人工审批）；等待处理的 Required 任务数 0。代码相关主测试 Run FastDeploy Unit Tests and Coverage / run_tests_with_coverage 已通过。Optional 区有 2 个失败和 1 个等待中，仅供参考。

总执行（rerun次数）	总任务	✅ 通过	❌ 失败	⏳ 运行中	⏸️ 等待中	跳过
36(0)	36	32	3	0	1	0

2 任务状态汇总

2.1 Required任务 : 9/10 通过

必选任务阻塞合并，失败需优先处理。

状态	任务	耗时	根因	修复建议	日志	重跑
❌	Approval	9s	需要 Approval	请通过人工审批	Job	-
✅	其余 9 个必选任务通过	-	-	-	-	-

2.2 可选任务 — 23/26 通过

可选任务不阻塞合并，失败仅供参考。

状态	任务	耗时	日志	重跑
❌	Run iluvatar Tests / run_iluvatar_cases	18m34s	Job	-
❌	Trigger Jenkins for PR	1m3s	Job	-
⏸️	CI_HPU	-	-	-
✅	其余 23 个可选任务通过	-	-	-

---

3 失败详情（仅 required）

Approval — 需要人工审批（置信度: 高）

根因摘要

该 Job 需要人工 Approval，完成审批后 CI 才会继续执行。

修复建议摘要

请通过人工审批后等待后续 CI 执行；如审批后仍未触发，请 rerun 对应 workflow。

[codecov-commenter]

Codecov Report

❌ Patch coverage is 68.18182% with 7 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (release/2.6@b562b8d). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
fastdeploy/cache_manager/prefix_cache_manager.py	68.18%	4 Missing and 3 partials ⚠️

Additional details and impacted files

@@              Coverage Diff               @@
##             release/2.6    #7886   +/-   ##
==============================================
  Coverage               ?   72.42%           
==============================================
  Files                  ?      381           
  Lines                  ?    54232           
  Branches               ?     8475           
==============================================
  Hits                   ?    39275           
  Misses                 ?    12184           
  Partials               ?     2773           

Flag	Coverage Δ
GPU	72.42% <68.18%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[PaddlePaddle-bot]

🤖 Paddle-CI-Agent | pr_review | 2026-05-22 11:45:25

📋 Review 摘要

PR 概述：Cherry-Pick #7885，通过延迟批量回收 GPU blocks 降低 LRU 驱逐循环中堆操作开销，并修复父节点重复入堆逻辑。
变更范围：fastdeploy/cache_manager/prefix_cache_manager.py
影响面 Tag：[KVCache] [Scheduler]

问题

级别	文件	概述
📝 PR 规范	-	Cherry-Pick 标题缺少原始 PR 编号 (#7885)
🟡 建议	prefix_cache_manager.py:1372	静默移除 node.cache_status = CacheStatus.CPU，未在 Modifications 中说明
❓ 疑问	prefix_cache_manager.py:1380	defer_recycle=True 路径下 node.reverved_dec_block_ids 未清空
🟡 建议	-	Checklist 中 Add unit tests 已勾选，但 diff 中无测试文件变更，且未说明不加测试的原因

强制规则：表格中每一条有 文件:行号 的问题均已在 comments[] 中创建对应行间评论。

📝 PR 规范检查

Cherry-Pick 标题格式不符：当前标题末尾缺少原始 PR 编号，不符合 [Cherry-Pick][Tag] 描述(#原PR号) 规范。

标题建议（可直接复制）：

• [Cherry-Pick][Scheduler] Defer block recycling to accelerate LRU node freeing(#7885)

PR 描述结构完整（Motivation / Modifications / Usage or Command / Accuracy Tests / Checklist 均已填写），无需调整描述。

总体评价

优化思路清晰，延迟批量 recycle_gpu_blocks 可有效减少热路径堆操作开销，父节点重复入堆的逻辑修复也属正确改动。主要关注点：cache_status 状态移除需确认下游无依赖，reverved_dec_block_ids 在 defer 路径下是否需要清空，以及 unit test checklist 项是否真实匹配。

[PaddlePaddle-bot]

🟡 建议 静默移除了 node.cache_status = CacheStatus.CPU，未在 PR Modifications 中说明。

原函数名为 _handle_free_gpu_node_without_cpu（无 CPU 卸载路径），原先设置 cache_status = CacheStatus.CPU 确实语义矛盾，疑似原始代码 bug。但此行为变更影响节点状态，建议在 PR 描述 Modifications 中补充说明该删除是有意修复，并确认无下游逻辑依赖 cache_status == CacheStatus.CPU 作为节点已释放的判断依据。

[PaddlePaddle-bot]

❓ 疑问 defer_recycle=True 路径下，blocks_to_recycle = list(node.reverved_dec_block_ids) + [node.block_id] 仅复制了列表，但原属性 node.reverved_dec_block_ids 未置空（原代码有 node.reverved_dec_block_ids = []）。

若节点对象存在池化复用（node_id 回收后重新分配给新节点），旧的 reverved_dec_block_ids 未清空可能携带脏数据。请确认节点对象是否会被复用，若会则需补充清空：

node.reverved_dec_block_ids = []