On Latency Compensation and Its Effects for Body Motion Trajectories in Virtual Environments

Student:吳健榕 Advisor: 歐陽明

國立臺灣大學資訊工程學研究所

Abstract

Overall system latency, the elapsed time from input human motion to the immediate response of that input in the display, is one of the most frequently cited shortcoming of current virtual environment (VE) technology when the latency is relatively large. In a head-mounted display (HMD) based VE system with head tracker, if the latency is longer than 200 ms for instance, it will cause motion sickness for long time wearing. To compensate for latency, there are methods proposed that use prediction in tracking.

This dissertation proposes a novel head motion prediction method based on Grey system theory. In Grey system, the behavior of information are neither deterministic (white) nor totally unknown (black), but are partially known (grey). Because the behavior of the tracker output is also grey, the Grey system theory can be applied to predict the head motion in an HMD system. Our results show that the Grey system based prediction can greatly reduce the latency by at least one half and reduce image jittering. In order to measure the latency, a convenient way to precisely measure it for a VE is also proposed.

For latency compensation, several prediction methods have been analyzed by simulation. However, the degree of presence of the “man in the loop” should be concerned as well but never been properly studied. After all, users of VE are human beings. VE related researches including human reaction in real applications will be more practical. In measuring the performance on various head motion prediction methods, our purpose is to make a comparison both in simulation and in real tasks among (1) the Grey system based prediction, (2)the Kalman filtering, (3) a simple linear extrapolation, and (4) the one without prediction. Our results show that the former two methods are significantly better than the one without prediction, and within the best two they perform equally well. However, computation complexity of Grey system based prediction is significantly lower than that of the Kalman filtering based prediction.

Similarly, our auditory system may be sensitive to time lags in a VE. However, there is yet no research data available that describes the relationship between localization of sound source and head tracker latency. Since the power of the modern personal computer is enough to deal with the massive computation of 3D sound, a highly optimized program was implemented to play 3D sound in real time. Preliminary results of two experiments indicate that (1) localization of sound in space is easier with head movement than without and (2) if a 3D sound system has latency involved, latency compensation software helps the performance of localization in 3D space. Note that the two results are original in real tasks. Furthermore, two experiments has been designed and conducted involving experiments on the effects of latency compensation in a real application, the SpaceWalker building walkthrough system, both in the aspects of visual and acoustic fidelity.