 |
|
Before Pulsar, a small, but dedicated group of doctors and technologists from Loma Linda University Medical Center(LLUMC) began implementing various technologies developed for the military, entertainment, and aerospace industries in medical and educational applications. Pulsar director, Dr. David J. Warner led this effort while a Loma Linda MD/Ph.D student (1988-1995). From his college experience, Warner brought to LLUMC a highly cultivated interest in the complexities of human perception/expression. He knew then an understanding of neurobiology was needed to even begin asking questions; his Ph.D course was neurophysiology.
The first and most basic question: what are the dynamic and empirical features of a brain as it perceives or expresses some set of data/information? A set of ideas Warner collectively named Neurocosmology formed the philosophical ground for this inquest into brain and mind. Neurocosmology's central notion: that which can be perceived and/or expressed is limited only by the human nervous system's ability to collect, process and render to consciousness data coming into/leaving the body. Warner believed information systems to be a powerful and timely way for extending the normal sensory gates. Late 80's technology started making novel 3-dimensional interactive graphical environments (i.e., virtual reality) as well as creating new tools for data collection from the body. With these tools Warner's research into human expression began.
Advanced instrumentation for the acquisition and analysis of medically relevant biological signals
Exploration began by creating novel glimpses into the form and flow of the human brain. Using themselves as subjects, Warner and Jeff Sale took the clinical tools of the Neurologist (Electroencephalogram [EEG], Magnetic Resonance Imaging [MRI]) and fused them with the then new tools of high end digital graphics and supercomputing: Advanced Visualization Systems [AVS] Stardent Workstations, respectively. Unified, these methods gave wholly new kinds of clinically useful data. Instead of the EEG's normal array of "squiggly" lines, the new techniques generated visually rich 3-dimensional contoured surfaces, termed isosurfaces. By taking the basic data measured by 20 electrodes attached to the skull and inputting it into AVS' graphical rendering software modules the very same data could now look like this. Information in this format is vastly more powerful and more easily cognized then the abtuse trace lines of the traditional EEG: Breakthrugh! Jeff Sale then developed innovative methods for interpreting the new data based on the emerging mathematics of non-linear dynamical analysis, or chaotropic dynamical systems theory.
Moving beyond experimentation with themselves, Warner's group began to work with LLUMC patients from Outpatient Neurology and Pediatric Rehabilitation. Two general categories characterize the medical applications of information technologies:
quantitative assessment
augmentative communication
Qauntitative Assessment: Advanced informatic systems which augment the general flow of medical information providing support for the health professional
Dr. Warner had been exploring the VPL Data Glove, an early virtual reality interface device, when he realized the potential the glove afforded to the clinical neurologist. Many of the local nuerology patients were sufferring from Parkinson's disease. Traditional neurological examination relied entirely on the doctor's subjective, albeit skilled, visual and physical examination of patient symptoms and response to treatment over time. There was no way to precisely measure the exact nature and movement of Parkinsonian tremor. Fitted with the Polhemus 3-Space Tracker (6 degrees of freedom: x,y,z, yaw, pitch, roll), the data glove allowed a patient's sensory motor dynamics to be measured down to any needed increment. A"taxonomy of tremor" was born: Breakthrough! Eventually, the data glove was used during parkinson's palidotomy surgery. As the surgeon went through the operation, awake patients were asked to perform hand movements which were measured. The data clearly showed the positive effects in the post lesioned patient. Neurosurgeon, Dr. Iacona praises Warner's solution as there was never any immediate way of measuring the effect of deliberate brain lesioning during surgery.
Some American Neurological Association abstract titles demonstrate the powerful applications of the various technologies at the time:
Another instance of precise quantitative assessment was with pressure sensor technologies. Geriatric patients confined to beds in constant positions are always at risk for decubidi ulcers (i.e., pressure sores). Here again, clinical need for precise measurement of the phenomena arises: what is the weight of the patient; how long can they stay in a particular position before needing to be moved; which parts of the body develop decubidi quicker? These among others are questions with only vague and variable answers until a large pressure sensor was installed in a bed or wheelchair. The enhanced visual rendering of this data by AVS and Stardent was also a Breakthrough. The specific pressure data is then a basis for making effective decisions about how to best handle the patient's body.
Tekscan Pressure Sensors used for 2 and 3 dimensional color-coded visual renderings of presssure data. To indicate pressure criticality the image would be morphologically isometric in the negative. The degree of peak height is the significant unit. Also used for patient gait analysis and prosthetic device fitting.
The Neuro-Rehabilitation Workstation is a system designed by Warner et al., to assist in the clinical neurology setting. NRW employs all the technologies in a single platform for diagnostic and interventional medical informatics (e.g., Parkinson's, alzhiemers, etc). Data gloves, EEG, MRI, ECG, PET, CT, EMG, MEG and pressure sensors of various types all feed out of the patient into a central processor which then fuses all relevant data into representational forms more rich to the clinical gaze (i.e., expert knowledge system). Rendered information may take the form of squiggles or else more powerful and exotic forms such as the beautiful and dense isosurfaces wrought by the AVS softwares. The NRW is among the most advanced concepts for neurological diagnostics to date. NRW allows for precise quantitative measurement of all forms of nuerological exam.
1. Captured Physiological Output
2. Quantification of Motion
3. Complex Graphical Rendering Systems
The Convulvitron Chair was a 3-D sound and contour pliant somatic interface. The objective was to discern possible psychological and emotional effects of sound. Sound managed and applied through the chair could conceivably bring a subject into subtle altered states of consciousness by various specialized 3-dimensional renderings of sound.
Motion Platforms for designing realistic virtual worlds. The platforms also allowed experimentation with vestibular system sense of orientation in simulacrum space. Motion sickness was a problem early on for VR interfaces. Visual system perception of space events and movements conflicted physiologically with the vestibular system’s perception of no movement resulting in naseua.
Augmentative Communication: New interface devices for persons with disabilities
The data glove was applied in Gesture to Speech systems. This application is very promising for the various Aphagias. By establishing a patient specific set of hand gestures corresponding to phonetic structures a patient could regain some level of verbal communicability.
Qaudriplegic children were the real impetus for rethinking the connection between human and computer. As Warner the early, more abstract thinking was focused on how to measure thought so as to begin willfully orchestrating thought to make thinking itself an input modality (i.e., autocerebrascope: mind controlled computer), a more real problem became abundanly clear: the debilitating effects of neuropathology and paralysis. The work would then become very focused on the problem of interface development for people no longer able to interact with the world in "normal" ways. While a great deal of work had succeeded in innovatively collecting data from the body, the goal of enhancing expressional output was still in limbo. This would change in Spring of 1991.
On a single day in March, 1991 Dr. Warner took the opportunity to test an EMG driven interface called Biomuse. EMG (electromyograph) is a medical device which records the electrical output of flexed muscles. Using EMG sensors on a skin surface, Biomuse was the first technology allowing physiologically generated interaction (as opposed to mechanical interaction: mice and keyboard) between a computer and a human (This was the beginning of the re-cognition of the interface as a project no less important than processor speed and of greater social significance). Originally, Biomuse transduced a leg/arm EMG into a musical tone. Warner had another idea: what if they attached EMG sensors to the faces of quadriplegics and coached them in moving the muscles? Using a very simple program in place of Biomuse's musical output, LLUMC Pediatric Rehab patient Crystal Earwood (18 months) was the youngest quadriplegic to ever control a computer with voltage output from her eyes. The somatic basis for human-computer interaction had been initiated in earnest. Warner’s lab purchased Biomuse in Fall of 1992. At this time, then high school student, Patrick Keller, became Warner's chief "lab slave." Patrick would be instrumental in the real world applications of this Biomuse breakthrough and all subsequently related work into augmentative communication.
Programmer Jo Johansen was hired by Warner to write software allowing the computer to moniter EMG signals and then translate those signals into commands. Bio-enviornmental Control (BEC) was the first version of this "gesture recognition" software. Neat Software supplanted BEC and continues to be revised. Biomuse and all its associated rehab devices were run on a 386 PC.
The work reached its most elaborate and successful point with a seven year old C1 quadreplegic, Ashley Hughes. Neat Software was in fact designed right from Ashley’s facial muscle capacities. The following technologies were used in developing a physiological interface for Ashley:
A very significant ‘quality of life’ enhancing feature of the last three elements of the list is their allowing others to interact with Ashley. Mario Bros. allowed Ashley to simultaneously play with her peers. Biocar and Cindy Cyberspace greatly enticed people around Ashley to play with her. With these off the shelf technologies, other children would be inclined to pilot Cindy around Ashley's house and yard while she sat stationary in her wheelchair and directed them to climb a tree or swim in the pool. As Ashley controlled Biocar, her friends and family members would follow it around the house and move it from place to place. They could put it in practically any space they wanted Ashley to "be" in and she would immediately be there becuase of telepresence. Such technologies begin to break down the common barriers normal children run up against when confronted by disabled peers.
In 1994/95 David Warner and Solamo Maturnen developed THNG 1. THNG 1 is a 4-channel EMG signal transducer which supplanted Biomuse. Biomuse is a $20K device. This price would prevent most potential users form having access to it. THNG 1 is more efficient, compact and less than $200.
1995 Completion of M.D. Program
1995-Present: Nason Fellowship, Syracuse University
Consistent with the original goal of giving the disabled access to information systems and ultimately the World Wide Web Dr. Warner has succeeded in his work with:
Eyal Sherman: Like Ashley, Eyal is a C1 quadreplegic. With similar technologies as were used at LLUMC and going beyond Eyal is one of the first quadreplegics to navigate the WWW using inputs from his face.
Educational systems that adapt to the users ability to learn.
Julie Jacoby: Julie is limited cognitively, bahaviorally and linguistically by neural retardation. Smart Desk is a project instigated by Warner to give Julie and others like her greater power with the limited expressional capacities she has. Smart desk is the state of the art in multi-sensory input technology for interaction with informatic systems.