Skip to content

About

This project is the official implementation of 'DreamOmni3: Scribble-based Editing and Generation''

Resources

Stars

41 stars

Watchers

2 watching

Forks

Latest commit

 

History

2 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 

Repository files navigation

DreamOmni3: Scribble-based Editing and Generation

Bin Xia, Bohao Peng, Jiyang Liu, Sitong Wu, Jingyao Li, Junjia Huang, Xu Zhao, Yitong Wang, Ruihang Chu, Bei Yu, Jiaya Jia

The Chinese University of Hong Kong · ByteDance · The Hong Kong University of Science and Technology

Paper Project Page Benchmark GitHub

🔥 News

Introduction

DreamOmni3 introduces scribble-based image editing and generation, a flexible interface that combines natural-language instructions, one or more reference images, and freehand scribbles. Unlike binary masks that only specify where to edit, colored scribbles can express what each marked region means and how regions across multiple images correspond.

DreamOmni3 provides:

  1. Precise multimodal control. Colored marks establish explicit spatial and logical relationships between source and reference images.
  2. Seven tasks in one framework. The model covers four editing tasks and three generation tasks, ranging from instruction-guided manipulation to multimodal composition and doodle rendering.
  3. A natural fit for pretrained RGB models. Scribbles remain embedded in natural images, allowing unified generation and editing models to retain their pretrained visual understanding.

Supported Tasks

Editing Generation
Scribble and instruction-based editing Scribble and instruction-based generation
Scribble and multimodal instruction-based editing Scribble and multimodal instruction-based generation
Image fusion Doodle generation
Doodle editing

Visual Results

The following figure summarizes all four editing tasks and all three generation tasks supported by DreamOmni3.

DreamOmni3 editing and generation task summary

The full-resolution source is also available as PDF. More qualitative results are available on the project page.

DreamOmni3Bench

DreamOmni3Bench evaluates scribble-based editing and generation across all seven supported task types. It includes source and reference images, natural-language instructions, scribble conditions, and target results.

Quick Start

Note

Inference code and model weights are being prepared for release. Installation and usage instructions will be added here together with the public checkpoint.

Disclaimer

DreamOmni3 is intended for research and responsible creative use. Users should comply with applicable laws, licenses, and platform policies. The authors are not responsible for misuse of the released resources.

Citation

If DreamOmni3 is useful for your research, please consider starring this repository and citing our paper:

@article{xia2025dreamomni3,
  title={DreamOmni3: Scribble-based Editing and Generation},
  author={Xia, Bin and Peng, Bohao and Liu, Jiyang and Wu, Sitong and Li, Jingyao and Huang, Junjia and Zhao, Xu and Wang, Yitong and Chu, Ruihang and Yu, Bei and others},
  journal={arXiv preprint arXiv:2512.22525},
  year={2025}
}

Contact

If you have questions or suggestions, please open an issue or contact Bin Xia.

About

This project is the official implementation of 'DreamOmni3: Scribble-based Editing and Generation''

Resources

Stars

41 stars

Watchers

2 watching

Forks

Releases

Packages

Contributors