TY - CONF

T1 - Synthetically Generated Microscope Images of Microtopographies Using Stable Diffusion

AU - Siemens, Stefan

AU - Kästner, Markus

AU - Reithmeier, Eduard

N1 - Conference code: 1262309

PY - 2023/8/11

Y1 - 2023/8/11

N2 - This study presents a method to generate synthetic microscopic surface images by adapting the pre-trained latent diffusion model Stable Diffusion and the pre-trained text encoder OpenCLIP-ViT/H. A confocal laser scanning microscope was used to acquire the dataset for transfer learning. The measured samples include metallic surfaces processed with different abrasive machining methods like grinding, polishing, or honing. The network is trained to generate microtopographies with these machining methods, with different materials (for example, aluminum, PVC, and steel) and roughness values (for example, milling with Ra=0.4 to Ra =12.5). The performance of the network is evaluated through visual inspection, and the objective image quality measures Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), and Frechet Inception Distance (FID). The results demonstrate that the proposed method can generate realistic microtopographies, albeit with some limitations. These limitations may be due to the fact that the original training data for the Stable Diffusion network used mostly images from the Internet, which often show people or landscapes. It was also found that the lack of post-processing of the synthetic images may lead to a reduction in perceived sharpness and less finely detailed structures. Nevertheless, the performance of the model demonstrates a promising and effective approach to surface metrology and materials science, contributing to fields such as materials science and surface engineering.

AB - This study presents a method to generate synthetic microscopic surface images by adapting the pre-trained latent diffusion model Stable Diffusion and the pre-trained text encoder OpenCLIP-ViT/H. A confocal laser scanning microscope was used to acquire the dataset for transfer learning. The measured samples include metallic surfaces processed with different abrasive machining methods like grinding, polishing, or honing. The network is trained to generate microtopographies with these machining methods, with different materials (for example, aluminum, PVC, and steel) and roughness values (for example, milling with Ra=0.4 to Ra =12.5). The performance of the network is evaluated through visual inspection, and the objective image quality measures Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), and Frechet Inception Distance (FID). The results demonstrate that the proposed method can generate realistic microtopographies, albeit with some limitations. These limitations may be due to the fact that the original training data for the Stable Diffusion network used mostly images from the Internet, which often show people or landscapes. It was also found that the lack of post-processing of the synthetic images may lead to a reduction in perceived sharpness and less finely detailed structures. Nevertheless, the performance of the model demonstrates a promising and effective approach to surface metrology and materials science, contributing to fields such as materials science and surface engineering.

KW - confocal laser scanning microscopy

KW - image generation

KW - machine learning

KW - microtopography

KW - stable diffusion

KW - surface metrology

UR - http://www.scopus.com/inward/record.url?scp=85173433713&partnerID=8YFLogxK

U2 - 10.1117/12.2673643

DO - 10.1117/12.2673643

M3 - Abstract

T2 - SPIE Optical Metrology, 2023, Munich, Germany

Y2 - 26 June 2023 through 30 June 2023

ER -