Sensor fusion for ReVolt

Background

The ReVolt is an unmanned and zero-emission ship concept designed for short sea shipping along the coast of Norway. The ReVolt was designed for low energy demand and the propulsion system is fully electric. The ReVolt has 2 stern pods and 1 retractable azimuth thruster in the bow. A 1:20 scale model of the ReVolt has been built to explore and demonstrate the vessels autonomous capabilities and to test other design features.

The ReVolt model-scale vessel.

During the spring 2019 ReVolt has been equipped with a 360 degree camera system and a lidar, enabling it to perceive its surroundings. An MSc student is currently working on the development of a real time detection and tracking system on ReVolt. This system will use convolutional neural networks for camera detection, clustering algorithms for lidar segmentation, and a multi-sensor version of the target tracking system described in (Wilthil 2017).

The Ladybug 360 degree camera and the Velodyne Puck lidar mounted on ReVolt

Scope

The goal of this combined 5th year and master project will be to demonstrate sensor fusion as part of a closed loop collision avoidance system for ReVolt. The project can possibly be expanded into different projects for two or more collaborating students.

Proposed Tasks for the 5th year project

1. Conduct experiments early in the autumn semester to gather data from relevant scenarios which will guide the development of the sensor fusion and collision avoidance system.
2. Investigate the experimental data with a particular focus on challenges such as multi-target situations, occlusions, false tracks and misdetections, adverse weather conditions, problems with computational complexity, navigation uncertainty etc.
3. Based on identified challenges, propose solutions and mitigations. The candidate should check whether these can be based on previous and current developments in the Autosea, Autoferry and Autosit projects, or open source implementations on the web, to avoid reinventing the wheel unnecessarily.
4. Implement solutions to the most critical challenges, and verify on recorded data. Algorithmic developments should also as much as possible be tested in simulations.
5. Write report and make a plan for the MSc project.

Proposed Tasks for the master thesis

For the MSc project the goal will be to develop a closed-loop collision avoidance system, that ideally should be tested in real-world experiments during April or early May 2020. An MPC-based collision avoidance system for ReVolt exists already (Midjås 2018), but it has never been tested with sensor fusion as an input. Inclusion of hazard regions due to land is currently under development. Some revisions of the MPC method itself are likely to be needed to make the closed-loop system work, and a good interface between the sensor fusion and the control system is also needed as part of the MSc project.

Contact

Main supervisor Edmund F. Brekke
Co-advisors: Geir Hamre, Tom Arne Pedersen, Vegard Kamsvåg (DNV GL), Inger Berge Hagen (NTNU)

References

• Alfheim, H. and Muggerud, K. (2017): “Development of a Dynamic Positioning System for the ReVolt Model Ship”, Master thesis, NTNU.
• Wilthil, Flåten & Brekke (2017): “A target tracking system for ASV collision avoidance based on the PDAF”, in Sensing and Control for Autonomous Vehicles, Springer.
• Midjås, T. (2018): “Collision Avoidance for the ReVolt Model-Scale Ship”, Specialization project, NTNU.
• Norbye, H. G. (2019): “Camera-Lidar sensor fusion in real time for autonomous surface vehicles”, Specialization project, NTNU.