This repository provides a PyTorch implementation of the following paper:
DReAM-SAM: Dynamic Region-Adaptive Modulated Segment Anything Model for Fully Automatic Medical Image Segmentation
Kwanseok Oh1, 2, Dahye Lee2, and Heung-Il Suk1
(1Department of Artificial Intelligence, Korea University)
(2DEEPNOID Inc.)
Published in IEEE TMI: https://doi.org/10.1109/TMI.2026.3752818Abstract: The Segment Anything Model (SAM) has shown strong zero-shot segmentation capabilities across general visual domains. However, its effectiveness in Medical Image Segmentation (MIS) remains constrained by the medical-domain gap, reliance on explicit user prompts, and limited adaptability to small, heterogeneous, or ill-defined targets. In particular, feature adaptation in existing SAM approaches often remains shared across anatomical targets, making it difficult to tailor local representations to target identity, morphology, and surrounding anatomical context. In this paper, we propose Dynamic Region-Adaptive Modulated SAM (DReAM-SAM), a fully automatic framework that introduces target- and region-adaptive representation learning for fine-grained MIS. Our framework introduces three key technical pillars: (1) Target-Conditioned Dynamic Modulation: We propose an agile CNN (ACNN) that complements SAM's encoder by adapting local feature extraction to target identity and morphology while promoting boundary-sensitive details. (2) Query-Adaptive Feature Interaction: We develop a bidirectional neighborhood attention mechanism to integrate complementary ACNN and SAM features over query-adaptive multi-scale contexts. (3) Target-Aware Self-Prompting: We further devise a self-prompting mechanism that exploits target-existence cues and target-aware representations to automatically generate surrogate prompts. Extensive experiments across six MIS datasets and diverse medical imaging modalities reveal that DReAM-SAM outperforms state-of-the-art SAM variants and task-specific MIS models.
The main contributions are summarized as follows:
- We propose DReAM-SAM, a fully automatic SAM adaptation framework that explicitly introduces target-/region-adaptive representation learning to address the lack of target-adaptive local representation in challenging MIS.
- We introduce an ACNN branch equipped with the DyRA-Block, which performs target-conditioned dynamic feature extraction via a shared--specific kernel decomposition and spatially adaptive sampling, complemented by kernel decorrelation and boundary-aware region decoupling.
- We further develop D$^2$NA for query-adaptive multi-scale interaction between the complementary ACNN and SAM representations; and a self-prompting module (TEC + TAPG) that derives surrogate prompts from inferred target cues without manual intervention.
- Extensive experiments across six MIS datasets and diverse imaging modalities demonstrate that DReAM-SAM outperforms state-of-the-art task-specific and SAM-based segmentation approaches, with particularly pronounced gains on small, heterogeneous, and ambiguous targets.
We evaluate DReAM-SAM on six MIS benchmarks covering CT/CTA, MRI/MRA, ultrasound, and X-ray.
| Dataset | Modality | Target | Train / Test |
|---|---|---|---|
| LIASD | CTA | Intracranial aneurysm | 1,186 / 152 |
| ADAM | ToF-MRA | Intracranial aneurysm | 79 / 34 |
| BTCV | CT | 13 abdominal organs | 21 / 9 |
| BraTS2021 | MRI (T1, T1CE, T2, FLAIR) | Brain tumor (ET / TC / WT) | 875 / 376 |
| TDSC-ABUS | 3D ultrasound | Breast tumor | 70 / 30 |
| QaTa-COV19 | Chest X-ray | COVID-19 pneumonia | 7,145 / 2,113 |
Performance is reported with DSC, boundary IoU (BIoU), and HD95. Unless otherwise noted, results correspond to DReAM-SAM with the 2D SAM (ViT-B) backbone.
DReAM-SAM is compared with 6 CNN/ViT-based models and 14 SAM-based methods, all reproduced under the same environment.
- DReAM-SAM achieves the best DSC on all six datasets.
- The margin over the second-best method is largest on datasets dominated by small, ill-defined targets (+8.18% on ADAM, +5.88% on TDSC, +3.99% on LIASD), and narrower on QaTa (+3.46%), where pneumonia regions are much larger.
- The framework is backbone-agnostic: built on MedSAM (2D) or SAM-Med3D (3D), it still outperforms every competing method in DSC.
- BTCV: DReAM-SAM ranks first on large organs (liver 97.9%, kidneys 96.1%) and shows larger gains on small structures such as the gallbladder (+5.7%) and adrenal glands (+3.8%).
- BraTS: DReAM-SAM improves ET / TC / WT by +5.3% / +4.6% / +4.2%. The largest gain is on ET, the smallest and most heterogeneous sub-region, which suggests the benefit grows with target difficulty.
On small or low-contrast targets, DReAM-SAM produces tighter and more coherent boundaries. Competing methods often miss the lesion entirely or over-segment into surrounding tissue. Green boxes show zoomed-in regions.
BTCV organs are split into small-volume (mean ≈ 13,263 mm³) and large-volume (mean ≈ 198,267 mm³) groups, about a 15× size gap, and models are trained with 10–100% of the labels. DReAM-SAM is compared with adaptation methods (SAMed, 3DSAM-adt) and with semi-supervised methods (H-SAM, CPC-SAM, SynFoC), which also use the remaining unlabeled data.
- Semi-supervised methods stay stable on large organs but drop sharply on small ones.
- DReAM-SAM uses no unlabeled data, yet it gives the best small-volume DSC at 50% labels and above.
Parameters, FLOPs, GPU memory, and inference time are measured on LIASD (1 target) / BTCV (13 targets) with 1024 × 1024 inputs on a single NVIDIA Quadro RTX 8000.
- Compared with AutoSAMUS, another CNN-augmented SAM, DReAM-SAM improves DSC by +27.0 / +10.5 %p and also runs faster (233.2 → 164.6 ms / 373.7 → 353.8 ms).
- Compared with Med-SA, DReAM-SAM improves DSC by +4.0 / +5.0 %p with ~4.2–4.6× fewer total parameters.
- These results suggest the gains do not come simply from added model capacity or computation.
- (a) Prototype similarity: On BTCV, bilateral organ pairs show among the highest off-diagonal similarities (kidneys 0.56, adrenal glands 0.58). This indicates that the learned prototype space reflects anatomical relationships between targets.
- (b) Prototype intervention: We replace the prototypes at inference without retraining. Using the mean prototype for every target lowers DSC from 89.50% to 81.92% (BTCV) and from 94.03% to 89.47% (BraTS). Shuffling prototypes across targets lowers it further, to 77.47% and 86.30%. Correct target–prototype correspondence is therefore essential for generating effective target-conditioned kernels.
The figure shows averaged feature maps from the ACNN (
- The self-generated dense prompt closely matches the GT in both location and shape.
- The original features mainly capture local textures (ACNN) and global semantics (SAM). After accentuation, both show finer detail and a clear focus on the target.
Representative failure cases on LIASD, annotated with the target-existence probability
-
(a) TEC false negative (
$y'$ = 0.29): the target gets too little guidance, and the lesion is missed. -
(b) TEC false positive (
$y'$ = 0.71): a localized activation in the dense prompt produces a spurious foreground. - (c) TAPG partial response: in a multi-lesion slice, the prompt responds strongly to the dominant lesion but only weakly to the smaller one.
- (d) TAPG misplacement: a spatially misaligned dense prompt produces a shifted and enlarged mask.
TEC decides whether target evidence is activated, and TAPG decides where it is concentrated. Uncertainty-aware existence estimation and multi-peak prompt generation are promising directions for handling small and multifocal lesions.
DReAM-SAM
├── segment_anything/ # SAM backbone and the proposed modules
├── utils/ # Helper functions for data preprocessing/postprocessing and objective functions
├── tests/dummy_run.py # Dummy-tensor smoke test (DyRA-Block kernels, L_kd, full forward/backward)
├── train.py # Training script
├── inference.py # Inference & evaluation script
├── config.yaml # Configuration file (directory, hyperparameters, and optimization settings)
If you find this work useful for your research, please cite the following paper:
@article{oh2026dreamsam,
title={DReAM-SAM: Dynamic Region-Adaptive Modulated Segment Anything Model for Fully Automatic Medical Image Segmentation},
author={Oh, Kwanseok and Lee, Dahye and Suk, Heung-Il},
journal={IEEE Transactions on Medical Imaging},
year={2026},
publisher={IEEE}
}
This work was supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS-2019-II190079, Department of Artificial Intelligence (Korea University)); the Institute of Information & Communications Technology Planning & Evaluation (IITP) under the Artificial Intelligence Star Fellowship Support Program to Nurture the Best Talents (IITP-2025-RS-2025-02304828) grant funded by the Korea government (MSIT); and the Technology Innovation Program (RS-2025-02221011, Development of Medical-Specialized Multimodal Hyperscale Generative AI Technology for Global Integration) funded by the Ministry of Trade, Industry & Energy (MOTIE, Korea).








