Background
Collision avoidance and situational awareness for autonomous surface vessels depend on methods for detection and tracking to know where other vehicles are and where they are heading. Vessels in confined waters such as canals and harbors often need to pass each other with small margins of error. This poses requirements to the tracking system, which only can be fulfilled by extended object tracking (EOT) utilizing the high resolution of modern sensors such as lidars, cameras or high-frequency radar.
 |
 |
|
|---|---|---|
| The Munkholmen tourist ferry as seen by a lidar (different colors are data from different time steps). | Setup where the lidar were recorded. It is evident that the boats in the background will cover several degrees in the field of view. |
The goal in EOT, as opposed to point-based tracking, is to estimate the extent, or shape, of other vessels, in addition to their kinematic states. Thus, any EOT method must utilize some kind of representation of the extent. Representations such as random matrices, fixed shapes, Gaussian processes or Fourier expansions all have shortcomings. Ships come in all kinds of shapes and sizes, and it is difficult to specify a model that both is sufficiently flexible, computationally efficient and which achieves the ideal precision.
As a solution to this, we have recently proposed a model where the ship perimeter is expressed by means of principal component analysis (PCA) over a large set of CAD models, which include many different shapes of relevance. This approach can be thought of as a form of soft classification or machine learning, where a fundamental form of the model is given, and the parameters are learned partly from offline training data, and partly from online sensor data. A major benefit with this approach is that is able to represent a wide range of ship shapes with a low number of parameters.
 |
| On the left: Comparison between ground truth, Fourier shape and PCA shape. On the right: A complete boat model generated by PCA model. |
Scope
The goal of this project is to contribute to the development of a complete EOT method that uses this kind of representation that is learned from offline candidate shapes. You will be working closely with a PhD candidate who developed this representation.
Proposed tasks for the specialization project
During the specialization project, you will be working closely with a PhD candidate who developed this representation. The specialization project will pursue answers to research questions such as the following:
- How is it best to do state estimation using this model? Is it preferable with a filtering approach or a smoothing approach?
- How can data association be done using this model? To which extent can feature information (colors, signal strength) aid the data association?
- For which sensors is this approach suitable? Much of the motivation stems from research on lidar tracking. The model is also relevant for stereo cameras, which also can provide a similar point cloud, although with lower accuracy. A 2-dimensional version may also be relevant to high-resolution radar.
Proposed tasks for the MSc thesis
The MSc thesis will continue in the same direction as the specialization project, or address research questions that were left for the future during the work with the specialization project. The following topics are likely to be of relevance:
- Multi-sensor fusion, using e.g. cameras and radar.
- Integration with established tracking systems using variations of probabilistic data association or multiple hypothesis tracking.
- Computational efficiency, to enable closed-loop testing.
Prerequisites
This is a list of recommended prerequisites for this master project.
- Strong programming skills in either Matlab, Python or C++.
- You should have had the course TTK4250 Sensor Fusion, or take it in parallel with the specialization project. Background in computer vision will also be useful.
Contact
For more information, contact main supervisor Edmund F. Brekke.
Relevant literature
- Ruud, K., Brekke, E., & Eidsvik, J. (2018): “LIDAR Extended Object Tracking of a Maritime Vessel Using an Ellipsoidal Contour Model”, SDF 2018.
- Granström, K., Baum, M. & Reuter, S. (2016): “Extended Object Tracking: Introduction, Overview and Applications”, Journal of Advances in Information Fusion.
- Lopez, E., Brekke, E., Mester, R. and Stahl, A. (2023): “Multiscan Shape Estimation for Extended Object Tracking, Fusion 2023.
 |
| Radar and lidar data recorded in the Trondheim City Canal. |
See also some of the media coverage that the autonomous ferry project has received: