Stable diffusion infrared texturing

Posted on March 21, 2024
Ground truth labels and detection results in an infrared image. From Kjønås (2021).

Background

Stable diffusion is a deep generative artificial neural network that can be used to generate images from text input. It can also be used generate textures for 3-dimensional models. In contrast to previous text-to-image models it is open source and can be run on consumer-grade computers.

Scope

The goal of this project is to simulate infrared textures for the digital twin Autoferry Gemini, which is used for simulations of the autonomous ferries milliAmpere 1 and milliAmpere 2.

Proposed tasks for the specialization project

  • Make yourself familiar with latent diffusion models in general and Stable Diffusion in particular.
  • Consolidate suitable infrared training data for experimenting with Stable Diffusion in the context of Autoferry Gemini.
  • Train Stable Diffusion on collected data, and generate high-fidelity simulations.
  • Analyze the output in quantitative and qualitative terms.

Proposed tasks for the MSc thesis

The MSc thesis will build on the specialization project, and pursue a topic such as one of the following in greater depth:

  • Can simulated infrared textures be used to train infrared detectors?
  • Can simulated infrared textures be used to test infrared detectors?
Workflow of Stable Diffusion used to generate optical textures on boats.

Prerequisites

This is a list of recommended prerequisites for this master project.

  • Strong programming skills in Python and/or C++.
  • You should have had courses in machine learning and/or computer vision.

Contact

For more information, contact main supervisor Edmund F. Brekke.

Relevant literature