Background
Neural radiance fields (NeRFs) have gained momentum as a tool to reconstruct complex three-dimensional scenes from two-dimensional images. Such reconstructions can be used as part of a digital twin.
Scope
The goal of this project is to build content for the digital twin Autoferry Gemini, which is used for simulations of the autonomous ferries milliAmpere 1 and milliAmpere 2. This includes both the environment with harbor facilities, water and buildings, and other vessels and vehicles in the vicinity.
Proposed tasks for the specialization project
- Make yourself familiar with literature and software for NeRFs.
- Consolidate suitable training data for experimenting with NeRFs in the context of Autoferry Gemini.
- Train NeRF models on collected data, and generate high-fidelity simulations.
- Analyze the output in quantitative and qualitative terms.
Proposed tasks for the master thesis
The master thesis will build on the specialization project, and pursue a topic such as one of the following in greater depth:
- To which extent can NeRFs be used to simulate realistic sea conditions?
- To which extent can NeRFs be used to generate dynamic objects such as ships, cars, pedestrians, birds, etc?
- Can we also use NeRFs to estimate the motion of dynamic objects using online data from cameras or lidar?
Prerequisites
This is a list of recommended prerequisites for this master project.
- Strong programming skills in either Matlab, Python or C++.
- You should have had courses in machine learning and/or computer vision.
Contact
For more information, contact main supervisor Edmund F. Brekke.
Relevant literature
- Yang et al (2023): “UniSim: A Neural Closed-Loop Sensor Simulator”, CVPR 2023.
- Huang et al. (2023): “Neural LiDAR Fields for Novel View Synthesis”, ICCV 2023.