Heads Up! Supporting Maritime Navigation using Augmented Reality

Abstract

Augmented Reality (AR) is a technology that shows potential for the improvement of maritime safety. Today, the ship bridge suffers from a lack of standardization and integration. Head-Mounted Displays (HMDs) may alleviate these challenges by showing information when relevant and enhancing operator mobility. Microsoft HoloLens 2 (HL2) is such a HMD. Prior research shows the potential of HMDs in the Maritime AR domain (Rowen et al., 2019). Limited research has been conducted however on the design of AR User Interfaces (UIs) for maritime applications leveraging HMDs. As a result, no framework exists to test new UI designs in the real world, which is necessary due to many variables that cannot be accurately modelled in a lab setting. This led to the research questions (RQs) 1. What makes an effective head-mounted AR UI for maritime navigation? (RQ1); and 2. How can HL2 be used as a ship bridge system? (RQ2) A Research through Design (RtD) process is detailed where a UI design and functional prototype was developed in collaboration with end-users. The prototype, named Sjør, implements the aforementioned interface, provides a framework for in-context UI testing and can be viewed as the next step towards standardizing AR UIs for the maritime industry. The design and development process led to three contributions to the Maritime AR domain. Firstly, a framework for the visualization of location-based data about points of interest on predefined canvases co-located in the real world was developed (Technology Readiness Level (TRL) 6), which runs on the HL2. This first contribution is defined in Section 4 and provides an answer to RQ2. Secondly, using this framework, an interface design (including interactions) is developed in collaboration with end-users and proposed as an answer to RQ1. This process is described in Section 5. The third contribution is a research agenda which provides insights into how contemporary and future research can leverage the developed framework. Section 7 discloses this research agenda.