The PMBM filter 2 - Radar-AIS fusion

Posted on April 24, 2019

Background

To safely navigate among other ships, an autonomous ship needs means to keep track of where other ships are as well as their motions. On the fundamental level, there are two ways to achieve this. On the one hand, ships may broadcast their locations and other relevant information to each other. All ships of a certain size are obliged to do this through the Automatic Identification System (AIS). On the other hand, ships can perceive their environment including other ships by means of exteroceptive sensors such as radar, lidar, optical cameras and infrared cameras.

AIS is easy to work with because AIS data are inherently tidy: There is a unique measurement from each boat that has an AIS transmitter. For radar, on the other hand, one must always solve the problem of data association: Of several returns in a radar image one must decide which return comes from which ship. However, radar or other exteroceptive sensors are always needed because not all ships have AIS, and because AIS depends on satellite communication with its own reliability issues.

In the Autosea project, both radar and AIS have been used as information sources in collision avoidance (COLAV) experiments. However, the two data sources were never used together, because a reliable algorithm for fusing radar and AIS never was developed.

It is not immediately obvious how radar and AIS should be fused in order to benefit maximally from the two data sources complementary properties. One can, however, expect that optimal fusion only is possible if the fusion algorithm is based on an optimal approach to multi-target tracking. Such an approach is provided by the Poisson Multi-Bernoulli Mixture (PMBM) filter (Williams 2015), which has emerged as the gold standard in multi-target tracking during the last decade.

Scope

During this assignment, the candidate will develop a multi-target tracking algorithm that fuses both radar data and AIS data within the PMBM framework.

Proposed Tasks for the 5th year project

In the specialization project the main focus will be on mastering the fundamental building blocks of PMBM and radar-AIS fusion. The following tasks are proposed for the specialization project:

  • Make yourself familiar with multi-target tracking, random finite sets, MHT and PMBM.
  • Design simple toy scenarios that will enable careful analysis of the expected benefits of radar-AIS fusion.
  • Revise the hypothesis structure of the PMBM filter so that its hypotheses not only are able to account for associations between measurements in consecutive radar scans, but also associations between these and AIS tracks.
  • Revise the expressions for the probabilities of the association hypotheses according to the revision of the hypothesis structure.
  • Revise the estimation of target kinematics according to the two data sources.
  • Test the revisions in simulations.
  • Write report.

Proposed Tasks for the master thesis

In the MSc thesis the work should move from toy scenarios to more comprehensive simulations and real data. It is also desirable to strengthen the theoretical foundations of the work using the tools of random finite set theory such as functional derivatives. The following topics are expected to be of central importance.

  • Study mixture reduction techniques to limit the number of hypotheses.
  • Benchmark against other approaches to AIS-radar fusion, such as Liland (2017), Gaglione (2018) or Hem (2021).
  • Generate more elaborate simulations and/or participate in field experiments to record real data.
Giorgio Kufoalor during Autosea experiments in the Netherlands where AIS data were used
Erik Wilthil during Autosea experiments in Trondheimsfjorden where radar tracking was used

Prerequisites

The candidate should have had the course TTK4250 Sensor Fusion, or take it in parallell with the specialization project. Experience with sensor fusion from extracurricular activities will also be useful.

References

Williams, J. (2015). “Marginal multi-Bernoulli filters: RFS derivation of MHT, JIPDA, and association-based member.” IEEE Transactions on Aerospace and Electronic Systems, vol. 51, no. 3.

Liland (2017): “AIS Aided Multi Hypothesis Tracker”, Master’s thesis, NTNU.

Gaglione, D., Braca, P. and Soldi, G. (2018). “Belief Propagation Based AIS/Radar Data Fusion for Multi - Target Tracking.” Proceedings of FUSION.

Hem (2021): “Maritime multi-target tracking with radar and asynchronous transponder measurements”, Master’s thesis, NTNU.

Contact

Supervisor: Edmund Brekke.

Example application 1: The autonomous ferry Milliampere should be able to use AIS for warm track initiation on the Munkholmen ferry when it approaches under the bridge.
Example application 2: The same theory can be used for fusion of drone transponders with 3D airport surveillance radar.