Sketch-based Storytelling for Cognitive Problem Solving: Externalization, Evaluation, and Communication in Geology

Abstract

PROBLEM solving is an important part of all engineering and scientific activities. It is present, for instance, when experts want to develop more fuel-efficient cars or when they are searching for oil and gas in the subsurface. Many alternatives have to be examined and evaluated before the optimal solution is found. Solving such problems is not only performed inside the mind of the scientist, but it is also an interaction between mind and scribbles, sketches, or visualizations on papers, on blackboards, and on computers. For problem solving in expert teams, this externalization through sketches and visualizations also plays an important communicative role. This dissertation presents research for assisting the problem-solving process on the computer, through novel technological advances in the fields of illustrative visualization and sketch-based modeling. Specifically, it targets problems that are related to evolutionary processes. Firstly, inspired by storytelling, the domain experts can express their ideas for solution as stories. These stories are based on sketches that the experts draw, utilizing a novel temporal-sketching interface inspired by a flip-over canvas metaphor. Further, the dissertation describes a set of sketching proxy geometries, such as the box-proxy geometry, that the experts can take advantage of when drawing three-dimensional (3D) sketches. These proxy geometries support the task of mapping a two-dimensional input (2D), e.g., a mouse or a digitizer tablet, to a 3D sketch. Solving difficult problems require that many different solutions are evaluated to identify the most optimal one. This dissertation introduces the story-tree, a tree-graph data structure and visualization, which manages and provides access to an ensemble of alternative stories. The story-tree also provides an interface where the stories can be evaluated and compared. This playback of the stories is done through automatic animations of the 2D sketches. The third challenge addressed in this dissertation is to communicate the optimal solution to decision-makers and laymen. By combining the animated 2D story sketches with illustrative visualization techniques it is possible to automatically synthesize and animate 3D models. These animations can be combined with new cutaway visualization techniques to reveal features hidden inside such 3D models. All of these contributions have been investigated in the context of the problemsolving tasks relevant to the early phase of petroleum exploration. This phase is characterized by having very little ground-through data available. Thus, a large solution space needs to be explored. Even so, the geologists need to produce models that can predict if petroleum is present. In addition to working with few data, the geologists also work under heavy time constraints because of the competition between the oil companies exploring the same area. The contributions from this dissertation have created enthusiasm among the domain experts and already, a new research initiative has materialized from the work described in this dissertation. Based on the feedback from the domain experts, we can conclude that the contributions presented in this dissertation form a valuable step towards better tools for problem solving, involving the computer, for the domain investigated here.