This report presents a systematic study on the architectural evolution, sampling mechanics, and multimodal guidance of diffusion-based generative systems evaluated on the CIFAR-10 dataset. To address the inherent trade-offs between generative quality, inference latency, and geometric structural fidelity in color image synthesis, our research proceeds across three evolutionary stages.
In Stage 1, we establish a foundational exploration of neural network backbones (UNet vs. Diffusion Transformer, DiT), sampling paradigms (DDPM, DPM-Solver, and Flow Matching), and image quality enhancement mechanisms, specifically Classifier-Free Guidance (CFG) and Edge-Aware Structural Loss (EASL).
In Stage 2, we construct a rigorous benchmark matrix comprising seven distinct model configurations (M1--M7) to systematically isolate the contributions of backbone design, sampler speed, and structural constraints. Empirical evaluations across Fréchet Inception Distance (FID), Inception Score (IS), sampling latency, and edge sharpness identify M6 (UNet backbone with Flow Matching, enhanced by CFG and EASL) as the optimal baseline architecture.
In Stage 3, we extend M6 into a multimodal framework by incorporating a pre-trained Text Encoder and Cross-Attention modules, facilitating fine-grained conditional generation controlled by continuous semantic embeddings and dynamic CFG scales. Experimental results confirm that our evolutionary pipeline significantly optimizes stroke fidelity, class stability, and cross-modal alignment.
- Stage 2 Benchmark (M1-M7): Complete training and evaluation scripts for comparing UNet/DiT backbones and DDPM/DPM-Solver/Flow Matching samplers.
- Stage 3 Multimodal Fine-tuning: Cross-Attention integration and CLIP Text Encoder conditional sampling for the optimal M6 baseline.
- Evaluation Suite: Integrated scripts for calculating FID, IS, and Edge Sharpness.
- Python 3.10+
- PyTorch (Configured with CUDA for optimal training throughput on high-end hardware like the RTX 5070)
- Torchvision
Please refer to the stage2_benchmark/ and stage3_multimodal/ directories for detailed model implementations, training loops, and sampling scripts.