Virtual Reality technologies are technologies which support an experiential interaction with in a computationally sustained environment. Virtual Reality technologies represent a fundamentally new way for humans and computers to interact. For not only do these technologies translate natural human actions of communication, such as speaking eye-movements and body gestures into computer commands , but they also render information to the human in multiple sensory modalities, that is spatialized audio, 3D graphics and various somato-sensory forms.
Introduction
To date the major effort of companies developing these technologies has been primarily to cater to the military, entertainment and construction industries. This is very unfortunate. While no one will argue the economics' of this bias, there is a strong humanitarian consideration that is being neglected. It is true that these technologies will profoundly impact the entertainment industry, and they should; for interactive media is truly a vehicle for participatory governments of the future. However, if we miss this opportunity to fully exploit the positive humanitarian potentials in EDUCATION and MEDICINE this period in history will be looked upon as one of the truly great missed opportunities in techno-social evolution. Here at the Center for Really Neat Research in Loma Linda University we have a moral imperative to "Dare to Care" and an operational mode of action of "Lead by Example" in our efforts to fuse High Tech with High Touch in the fields of medicine and education. The following are some examples of our efforts:
Immediate Medical Applications for Virtual Reality Interfaces
Normal people are naturally enabled. They are born with the capacity to interact with the world and willfully manipulate their environments. Disabled people have lost the capacity for such interaction and manipulation through either trauma or disease. Advanced human-computer interface technology that has been developed as natural user interfaces for interaction with virtual reality has immediate application in re-enabling the disabled persons. While virtual reality promises to solve many problems in the future, the immediate application of these advanced interfaces can improve the lives of millions of today. At the Loma Linda University Medical Center we have had many successes in utilizing these technologies. The utility of these devices has already been demonstrated as augmentative communication devices, as environmental controllers, as therapeutic tools in rehabilitation and as tools for quantitative assessment for diagnostic evaluation. Patients who have lost the ability to communicate verbally have successfully used an instrumented glove configured in a gesture to speech mode. Spinal cord injury, stroke and traumatic brain injury patients have virtual reality technology to manipulate virtual objects and practice specific skilled motor tasks. Quadriplegics have used physiological input devices to move objects on the screen with only their eyes and to play virtual instruments merely by contracting face and neck muscles. These are just a few examples of immediate uses for this promising technology that can profoundly improve the quality of life of real people today.
The Virtual Reality technologies also are showing great promise in the field of psychiatry. In a recent experiment a real-time performance animation system was used to encode facial expressions of an actor and then generate a 3D talking head with realistic facial expressions that could interact with children that were in the hospital. This Virtual Teacher taught several classes on anatomy to a group assembled in a classroom and then made individual bedside appearances in the children's room over the hospital television system. The reaction of the kids (and the doctors) was overwhelming. The potential for a system such as this to augment the quality of life of hospital bound children is profound. Now for some theory.
Remapping the Human-Computer Interface for Optimized Perceptualization of Medical Information
Virtual Reality is a paradigm shift in the way we think about mass communication and information technologies. Consider the following: In the distant past Medicine was an art, the practice of medicine was guided mostly by refined heuristics and intuition. All the external senses were used in the evaluation of the patient. Visual, auditory, tactile, olfactory and gustatory cues were all integrated to give the healer a perception of what to do. With the development of science and technology the practice of medicine has slowly shifted from being intuition based to being guided by the results of objective tests. In many ways this is progress, though in other ways we seem to have forsaken our own senses in favor of machines, thus removing ourselves from the determination of the problem. The ever increasing ability of technology to quantitate complex physiological parameters and to image volumetric anatomical structures are taking us to a point where we will soon be unable to assimilate all the available information through the traditional means (i.e., numbers and graphs). Recent attempts to solve this problem have focused primarily on advanced visualization techniques. While much progress has been made in this field, the visual sense is finite and is reaching its saturation level.
Enter Virtual Reality. Virtual reality technology is primarily interface technology that renders computer information onto multiple human sensory systems to give a sustained perceptual effect (i.e., a sensation with a context) while monitoring human response in the form of gestures, speech, eye movements, brain waves and other inputs. This interface also allows for a natural interaction with abstract data sets providing an integrated experiential encounter with information. This new technology provides us with the capacity to move into a new paradigm, a paradigm where the physiological integration of a pansensory rendering of medically relevant information provides an enhanced capability to discriminate between classes of complex dynamic interactions involved in pathophysiological processes.
Much attention has been given to enhanced visualization techniques. Dynamic volumetric stereoscopic rendering methods have greatly enhanced our capacity for visual assessment of medical information. We need however to be careful that we do not become photo-chauvinistic and forget that we have other senses. There are relevant concepts from sensory physiology that are now within the resolution of the interface technology. This new technology increases the number and variation of simultaneous sensory inputs, thus making the body a sensorial combinetric integrator. A good working knowledge of sensory physiology and perceptual psychophysics can help us optimize our future interactions with the computer. Aside from the basic neuroscience issues of modality, duration, intensity, distribution, frequency, spatial displacement, contrast, inhibition, threshold, adaptation, transduction, conductance and transmission (to name a few) we must identify the optimal perceptual state space parameters with in which information can best be rendered. We must also identify which types of information are best rendered by each specific sense modality.
New technologies and techniques have recently become available that allow for the rendering of data via auditory means. Not only can we now represent any data set in the form of sound but we can also spatialize the displacement of multiple sound sources giving us simultaneous exposure to different dynamic data sets. In these spatialized environments we can shift our attentional focus from source to source for real time comparison of multiple sets of data. Devices now exist which can stimulate the sensation of pressure, vibration, texture and temperature. This is a relatively untouched field as far as abstract data representation is concerned. These modalities combined with somatotopic placement also provide for spatial coding of the rendered information. The implementation of vision, hearing and touch technologies allow for simultaneous sensation of multiple independent and dynamic data sets that can be integrated physiologically into a single perceptual state.
Yet to be fully embraced by the virtual reality community are the olfactory and gustatory senses, smell and taste. While their current integration is questionable, their potential impact is quite profound. Recent work in olfactory science has identified at least 30 basic smells. Technologies under current development will be able to deliver quantified combinations of these smells for a wide range of distinct perceptual states.
In the area of taste the development of automated food processing will eventually allow for the doctor to get a taste of complex data. The use of smell and taste to help convey the state of complex systems may seem like quite a reach of the imagination. However, the possibility that these senses may help discern subtle changes in complex systems warrants investigation. We are embarking on an adventure that promises to change our relationship with the computer forever. With the immersion of all the external senses into virtual reality, our ability to perceptualize medically relevant information in an interactive mode will greatly enhance our capacity for improvisational investigation (stand up research). This is truly a paradigm shift and the beginning of a new era of computer assisted medicine.