Bin Xia, Bohao Peng, Jiyang Liu, Sitong Wu, Jingyao Li, Junjia Huang, Xu Zhao, Yitong Wang, Ruihang Chu, Bei Yu, Jiaya Jia
The Chinese University of Hong Kong · ByteDance · The Hong Kong University of Science and Technology
- 2026-08-08: We release the DreamOmni3 project page and DreamOmni3Bench.
- Coming soon: Inference code, model weights, and an online demo.
DreamOmni3 introduces scribble-based image editing and generation, a flexible interface that combines natural-language instructions, one or more reference images, and freehand scribbles. Unlike binary masks that only specify where to edit, colored scribbles can express what each marked region means and how regions across multiple images correspond.
DreamOmni3 provides:
- Precise multimodal control. Colored marks establish explicit spatial and logical relationships between source and reference images.
- Seven tasks in one framework. The model covers four editing tasks and three generation tasks, ranging from instruction-guided manipulation to multimodal composition and doodle rendering.
- A natural fit for pretrained RGB models. Scribbles remain embedded in natural images, allowing unified generation and editing models to retain their pretrained visual understanding.
| Editing | Generation |
|---|---|
| Scribble and instruction-based editing | Scribble and instruction-based generation |
| Scribble and multimodal instruction-based editing | Scribble and multimodal instruction-based generation |
| Image fusion | Doodle generation |
| Doodle editing |
The following figure summarizes all four editing tasks and all three generation tasks supported by DreamOmni3.
The full-resolution source is also available as PDF. More qualitative results are available on the project page.
DreamOmni3Bench evaluates scribble-based editing and generation across all seven supported task types. It includes source and reference images, natural-language instructions, scribble conditions, and target results.
Note
Inference code and model weights are being prepared for release. Installation and usage instructions will be added here together with the public checkpoint.
DreamOmni3 is intended for research and responsible creative use. Users should comply with applicable laws, licenses, and platform policies. The authors are not responsible for misuse of the released resources.
If DreamOmni3 is useful for your research, please consider starring this repository and citing our paper:
@article{xia2025dreamomni3,
title={DreamOmni3: Scribble-based Editing and Generation},
author={Xia, Bin and Peng, Bohao and Liu, Jiyang and Wu, Sitong and Li, Jingyao and Huang, Junjia and Zhao, Xu and Wang, Yitong and Chu, Ruihang and Yu, Bei and others},
journal={arXiv preprint arXiv:2512.22525},
year={2025}
}If you have questions or suggestions, please open an issue or contact Bin Xia.
