Applications of Diffusion Models to Discriminative Tasks: Classification, Segmentation, and More

Diffusion models have demonstrated outstanding performance in image generation tasks, surpassing traditional approaches such as GANs and VAEs. In addition to the high quality of generated data, the internal representations of diffusion models contain rich semantic and structural information that can be valuable for discriminative tasks, including segmentation, detection, and the extraction of depth and object boundaries. In this work, we investigate the use of diffusion features at different timesteps of the generation process for depth and edge prediction tasks. We compare architectures based on convolutional layers and attention mechanisms, and we also examine the effect of the temporal component t during training. As a baseline, we adopt the Readout Guidance architecture, which we further modify and extend in a series of experiments. Additionally, we analyze the effectiveness of using features from CleanDIFT, which are free from noise and temporal embeddings. Our results show that attention maps used as feature sources lead to improved performance in depth prediction, while features from CleanDIFT demonstrate advantages in object boundary detection. This work highlights the potential of leveraging internal diffusion model representations for discriminative tasks and suggests directions for future architectural improvements.