Rina Oh

Automatic medical segmentation is crucial for assisting doctors in identifying disease regions effectively. As a state-of-the-art (SOTA) approach, generative AI models, particularly diffusion models, have surpassed GANs in generating high-quality images for tasks like segmentation. However, most diffusion-based architectures rely on U-Net designs with multiple residual blocks and convolutional layers, resulting in high computational costs and limited applicability on general-purpose devices. To solve this issue, we propose an enhanced denoising diffusion implicit model (DDIM) that incorporates lightweight depthwise convolution layers within residual networks and self-attention layers. This approach significantly reduces computational overhead while maintaining segmentation performance. We evaluated the proposed DDIM on two distinct medical imaging datasets: X-ray and skin lesion and polyp segmentation. Experimental results demonstrate that our model achieves, with reduced resource requirements, accuracy comparable to standard DDIMs in both visual representation and region-based scoring. The proposed lightweight DDIM offers a promising solution for medical segmentation tasks, enabling easier implementation on general-purpose devices without the need for expensive high-performance computing resources.

In CycleGAN, an image-to-image translation architecture was established without the use of paired datasets by employing both adversarial and cycle consistency loss. The success of CycleGAN was followed by numerous studies that proposed new translation models. For example, StarGAN works as a multi-domain translation model based on a single generator–discriminator pair, while U-GAT-IT aims to close the large face-to-anime translation gap by adapting its original normalization to the process. However, constructing robust and conditional translation models requires tradeoffs when the computational costs of training on graphic processing units (GPUs) are considered. This is because, if designers attempt to implement conditional models with complex convolutional neural network (CNN) layers and normalization functions, the GPUs will need to secure large amounts of memory when the model begins training. This study aims to resolve this tradeoff issue via the development of Multi-CartoonGAN, which is an improved CartoonGAN architecture that can output conditional translated images and adapt to large feature gap translations between the source and target domains. To accomplish this, Multi-CartoonGAN reduces the computational cost by using a pretrained VGGNet to calculate the consistency loss instead of reusing the generator. Additionally, we report on the development of the conditional adaptive layer-instance normalization (CAdaLIN) process for use with our model to make it robust to unique feature translations. We performed extensive experiments using Multi-CartoonGAN to translate real-world face images into three different artistic styles: portrait, anime, and caricature. An analysis of the visualized translated images and GPU computation comparison shows that our model is capable of performing translations with unique style features that follow the conditional inputs and at a reduced GPU computational cost during training.