Skip to content

fix: release SV trial memmaps before temporary directory cleanup - #45

Open
xiangshengyun wants to merge 1 commit into
X-LANCE:mainfrom
xiangshengyun:fix/sv-trial-memmap-20261113
Open

xiangshengyun wants to merge 1 commit into
X-LANCE:mainfrom
xiangshengyun:fix/sv-trial-memmap-20261113

Conversation

@xiangshengyun

Copy link
Copy Markdown

Problem

When the SV pipeline finishes, it removes its temporary work directory. On
Windows an open np.memmap keeps the backing file locked, so cleanup raised
PermissionError [WinError 32] on trial_labels.uint8.mmap /
trial_scores.float32.mmap. That exception replaced the real evaluation
result or error, so every SV run failed on Windows regardless of the metric
outcome.

Fix

  • cosine_trial_scores/node.py: create the trial_scores/trial_labels
    mappings inside a try/finally and close both on every exit path, including
    the validation errors that previously leaked them (missing embedding keys,
    trial-count mismatch, zero-norm embedding, label-count mismatch).
  • sv/pipeline.py: release scores/labels in a finally around metric
    scoring so a failing metric no longer leaves them open. Add
    _release_trial_memmap(), which closes array._mmap.
  • Drop the array references before leaving the block so an exception traceback
    cannot keep the mappings alive.

Scoring results are unchanged; only the lifetime of the mappings changes.

Tests

Two regressions added to tests/test_sv_pipeline.py:

  • test_sv_cleans_work_dir_after_scoring
  • test_sv_cleans_work_dir_when_metric_fails

Both assert list(work_dir.iterdir()) == [] after evaluate_sv_files.
pytest tests/test_sv_pipeline.py → 9 passed on Windows.

Note

The visible failure only reproduces on Windows, because POSIX allows unlinking
a mapped file; Linux CI passed before this change too. The leak itself
(fd/mapping retention, plus arrays captured by exception tracebacks) is
platform-independent, so it is worth fixing upstream rather than only in a
fork.

An open np.memmap keeps its file locked on Windows, so removing the SV
pipeline temporary directory failed with PermissionError [WinError 32] and
masked the original result or error.

Close the trial mappings explicitly in the pipeline, release the
embeddings/scores/labels mappings on every cosine_trial_scores error path,
and add regression tests that assert the work directory is empty after both
successful scoring and a failing metric.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant