SELECTED I3 Papers

SAMARITAN PROJECT

LEADING BY EXAMPLE

INTELLIGENTLY IMPLEMENTING INTERACTIVE INFORMATION TECHNOLOGY

in

HEALTH CARE AND EDUCATION

to

IMPROVE QUALITY OF LIFE

MISSION:

To improve quality of life by intelligent use of interactive information technology in the fields of Health Care and Education.

METHODS: Provide the knowledge and resources required to empower people to make a positive difference with information technology.

Lead in the development of socially responsible applications of information technology.

Guide and assist in the intelligent implementation of advanced information technologies.

Identify, acquire and implement advanced information technology in exemplary applications.

Actively share information and experience with others who are willing to become involved in a socially responsible utilization of advanced information technology.

It is the intent of this project to seek out key areas where information technology can be effectively utilized to improve quality of life, i.e. in health care and/or education, and then actively participate in facilitating implementation.

WHAT IS INTERACTIVE INFORMATION TECHNOLOGY? Information technology is not new; clay tablets and quill pens are information technologies. The radio, television and telephone are information technologies. Computers and advanced interfaces are only the most resent additions.

Interactive information technology is any technology which augments our human ability to dynamically create, express, retrieve, analyze, process, communicate, or experience information.

WHAT HAVE WE DONE WITH THIS TECHNOLOGY SO FAR ?

For the past 4 years a small, dedicated group of socially conscientious technologists have been actively implementing off the shelf technologies that were developed for the military, entertainment, aerospace industries in medical and educational applications.

The primary role that we play is one of a general medical-scientist liaisons between the medical community and high-tech development companies, specifically for the purpose of technology transfer and application assessment of new technologies for possible medical uses. Our experience in the area of human computer interaction combined with our current access to clinical medicine allows us the unique opportunity to facilitate the rapid exchange of relevant information between the high-tech industry and the medical community in general. We are particularly active in technology transfer of aerospace and other defense derived technologies to more socially useful applications.

Project areas currently active include;

1. Advanced instrumentation for the acquisition and analysis of medically relevant biological signals

2. Advanced informatic systems which augment the general flow of medical information and provides decision support for the health care professional.

3. Public access to health information databases designed to empower the average citizen to become more involved in their own health care.

4. Advanced training technologies which will allow the rapid dispersion of newly developed techniques.

5. New interface devices for persons with disabilities.

6. Educational systems that adapt to the users ability to learn.

HOW CAN YOU HELP ?

The finacial status of the Samaritan Project is completely dependent on donations from people who see what we are doing and provide support so that these efforts will continue. Donations to the Samaritan Project are managed through LOMA LINDA UNIVERSITY, a non-profit organization, which has established a special account for the project. All contributions to the Samaritan Project are tax deductible and a letter of verification will be provided to the contributor by the University

WHAT WILL THE FUNDS BE USED FOR?

The funds for the Samaritan Project are to support the following:

1. EQUIPMENT (computers, communication technologies, interface devices, research instruments....)

2. SUPPLIES ( office supplies, film, video tapes, paper, computer disks.....)

3. PERSONNEL ( wages for part time/full time/contract workers , room and board for volunteers...)

4. EDUCATIONAL ACTIVITIES (travel, tuition, expenses for key persons involved with the Samaritan project to attend/participate in academic and professional functions...)

5. TRAVEL EXPENSES (for attending professional meetings, to network with strategic partners, for invited lectures .....)

BIO-CYBERNETICS

A Biologically Responsive Interactive Interface

"Adventures In The Next Paradigm Of Human Computer Interaction"

Dave Warner, Jeff Sale, Todd Anderson, Jo Johanson Human Performance Institute

Dept. of Rehabilitation Engineering,

Loma Linda University Medical Center

The capacity of computers to receive, process, and transmit massive amounts of information is continually increasing. Current attempts to develop new human-computer interface technologies have given us devices such as gloves, motion trackers, 3-D sound and graphics. Such devices greatly enhance our ability to interact with this increasing flow of information. Interactive interface technologies emerging from the next paradigm of human-computer interaction are directly sensing bio-electric signals (from eye, muscle and brain activity) as inputs and rendering information in ways that take advantage of psycho-physiologic signal processing of the human nervous system (perceptual psychophysics). The next paradigm of human-computer interface will optimize the technology to the physiology -- a biologically responsive interactive interface.

"BIOCYBERNETICS"

INTERACTIVE INFORMATION TECHNOLOGY

Interactive information technology is any technology which augments our ability to create / express / retrieve / analyze / process / communicate / experience information in an interactive mode. Biocybernetics optimizes the interactive interface, promising a technology that can profoundly improve the quality of life of real people today. The next paradigm of interface technology is based on new theories of human-computer interaction which are physiologically and cognitively oriented. This emerging paradigm of human computer interaction incorporates multi-sense rendering technologies, giving sustained perceptual effects, and natural user interface devices which measure multiple physiological parameters simultaneously and use them as inputs. Biologically optimized interactive information technology has the potential to facilitate effective communication. This increase in effectiveness will impact both human-computer and human-human communication, "enhanced expressivity". Work in human-computer interaction is an ongoing endeavor in many areas. These efforts have captured the attention of several professional societies; the entertainment industry, the aerospace industry, communications and educational technologies industries, as well as medicine. These diverse areas will all be impacted in multiple ways by advances in technologies that enhance human-computer interaction.

Optimizing the human computer interface will rely on the knowledge base of physiology and neuroscience, that is, the more we know about the way we acquire information physiologically the more we know the optimum way for a human to interact with intelligent information systems. The next paradigm will see the "THINNING" of the human-computer interface to a biological sheer as the interface will map very close to the human body.

PHYSIOLOGICALLY ORIENTED INTERFACE DESIGN

Knowledge of sensory physiology and perceptual psychophysics is being used to optimize our future interactions with the computer. By increasing the number and variation of simultaneous sensory inputs, we can make the body an integral part of the information system, "a sensorial combinetric integrator". We can then identify the optimal perceptual state space parameters in which information can best be rendered. That is what types of information are best rendered to each specific sense modality, "a sense specific optimization of rendered information. Research in human sensory physiology, specifically sensory transduction mechanisms, shows us that there are designs in our nervous systems optimized for feature extraction of spatially rendered data, temporally rendered data, and textures. Models of information processing based on the capacity of these neurophysiological structures to process information will help our efforts to enhance perception of complex relationships by integrating visual, binaural, and tactile modalities. Then by using the natural bioelectric energy as a signal source for input; electroencephalography, electroocculography, and electromyography (brain, eye and muscle) we can generate highly interactive systems in which these biological signals initiate specific events. Such a real-time analysis enables multi-modal feedback and closed-loop interactions.

"BIOCYBERNETIC CONTROLLER"

Interactive interface technology renders content specific information onto multiple human sensory systems giving a sustained perceptual effect, while monitoring human response, in the form of physiometric gestures, speech, eye movements and various other inputs. Such quantitative measurement of activity during purposeful tasks allows us to quantitatively characterize individual cognitive styles. This capability promises to be a powerful tool for characterizing the complex nature of normal and impaired human performance. The systems of the future will monitor a user's actions, learn from them, and adapt by varying aspects of the system's configuration to optimize performance. By immersion of external senses and iterative interaction with biosignal triggered events complex tasks are more readily achieved.

This paradigm shift of mass communication and information technologies is providing an exciting opportunity to facilitate the rapid exchange of relevant information thereby increasing the individual productivity of persons involved in the information industry. Areas such as computer-supported cooperative work, knowledge engineering, expert systems, interactive attentional training, and adaptive task analysis will be changed fundamentally by this increase in informatic ability. The psycho-social implications of this technologically mediated human-computer and human-human communication are quite profound. Providing the knowledge and technology required to empower people to make a positive difference with information technology could foster the development an attitude of social responsibility towards the usage of this technology and may be a profound step forward in modern social development. Applications which are intended to improve quality of life, such as, applications in medicine, education, recreation and communication must become a social priority.

USING TECHNOLOGY TO IMPROVE QUALITY OF LIFE

The potential of this technological capability to improve quality of life can be best understood when it is actualized into the lives of real people with real needs. The Human Performance Institute at Loma Linda University Medical Center is an interdisciplinary research center which is leading the effort to utilize the latest in human computer interface technology to "make the world a better place". The primary research areas are in developing interactive interfaces which enable severely disabled individuals to lead productive lives, and in the design of environmental systems which support experiential interaction with information systems in such a way as to help maintain a state of general good health.

The following are real world cases that demonstrate the utility of this technology to change the future of disabled individuals:

Crystal, an 18 month old "C1 quadriplegic" (complete paralysis from the neck down, requires a respirator in order to breathe) was the first person to use this biocybernetic technology in a medical setting. Processing of electropotential changes along the eye and adjacent muscles into a biological signal enabled this child to interact in real time with the displays on the monitor, in short, "her eyes became her hands" in generating commands to the screen. The activity was direct, the implications profound: She was able to enter into a unified feedback loop where direct real time response to a physiological signal was used to modify and improve that psycho-physiological source. In this case, her capacity to learn and interact with the world willfully was restored.

Andy, a 10 year old C2 quadriplegic whose speech is confined to the breathing patterns of his respirator to such an extent that it requires better than a minute to make a verbal request found himself in a spatialized environment where commands from facial muscles enabled him to "fly around" in a 3d computer environment. This was the first time in 5 years where he was able to willfully control something in his environment without the aid of others. A 17 year old car accident victim who was motivated to rehabilitate his impaired psycho motor skills through an "air guitar" interactive system which converted the weak bioelectric signals from his impaired muscles into "rock and roll" music.

We have also developed the BioCar, a primitive yet functional demonstration of telerobotic devices under direct biocybernetic control. The BioCar is a simple demonstration of how the biosignals can be used to control objects within an environment. For this demonstration a remote control car from Radio Shack was modified so that it can be controlled from the parallel port of a standard IBM compatible PC. Since there are only seven discrete functions (there is no proportional control) that the car can perform (forward, forward left, forward right, stop, reverse, reverse left and reverse right) then it takes a minimum of three sets of electrodes to control all of the functions (23=8). The BioCar software is responsible for interpreting the bioelectric signals from the user and sending commands to the remote control car.

Michael, a 27 year old engineer recently paralyzed in an auto accident was able to navigate the BioCar through a very complicated course using the muscles of his face and arms. The same system that allowed him to control this toy car could be easily adapted to control his wheel chair or some type of robotic arm. The potential to empower the disabled to become functional members of society can be realized through biocybernetic interface design.

The next effort of our lab was to expand the utility of this biocybernetic controller. We modified a nintendo game to accept commands from our system as if they were coming from the regular hand controller. This simple modification allows disabled children to use whatever muscle activity they have control of to play the same games as normal children. This generalized biocybernetic controller opens up an enormous resource of compelling games which can be integrated into rehabilitative therapy. From the control of virtually nothing to really something, we can get coordinated motion from patients at a much earlier time. Instead of some arbitrary task, they can work with computer generated objects that have specific motions associated with them; getting the associated feedback of watching themselves pick up a virtual object even though you may lack the physical strength to pick up a real object.

Future efforts will focus on adapting the biocybernetic controller beyond games and toys to functional information systems. The capacity to operate interactive educational multimedia systems will open a whole new area where human expressivity can be optimized in applications that customize an educational environment to the capabilities of an individual.

CYBERNETIC HEDONISM

The other focus of our efforts is in developing highly interactive, biocybernetic systems where biological signals can modify an environmental chambers' parameters allowing the user to bioelectrically interface with spatialized environments. We believe that such physiologically modulated environmental systems may have a health preserving function. Interfaces to control stimulation can adaptivly utilize any biosignal. The result is the capacity to create a stimulus regime that accelerates relaxation and facilitates stress reduction. This is an application of wellness maintenance technology. "The Nirvana Express"

THE MICROSCOPE OF THE MIND

The goal is to extend these environmental control systems into new methods of investigative research. Such as a test of basic cognitive functionality or the capacity to maintain attentional focus necessary to complete an iterative series of cognitive tasks. Data fusion of sensor data with user interaction parameters will allow meaningful correlation's to be made across various performance modalities. A goal of this application is to seek to identify a qualitative difference between the two performance/behavior states and then investigate various methods of quantifying that difference in a way that can be generalized.

It is postulated a difference will be seen in the modulation of some of the natural rhythms. It is also postulated that a cognitively induced modification would be consistent in an individual but would most likely be different between individuals. The psycho-social-behavioral nature of individuals factors into initial assessment of their cognitive function. Other indicators of cognitive function are short-intermediate-long term memory, sound judgment and the ability to identify similarities in related objects. Performance of these cognitive functions is a strong indicator of the biologic health of the brain. Poor performance is highly correlated with organic brain dysfunction.

THE POTENTIAL OF THIS NEW PARADIGM OF BIOCYBERNETICS IS LIMITED ONLY BY THE IMAGINATION(and funding) OF THE USERS

Special thanks to Dave Gilsdorf and Patrick Keller for their ongoing efforts in making the world a better place

REAL MEDICAL APPLICATIONS OF VIRTUAL REALITY TECHNOLOGY

Dave Warner

Medical Neuroscientist Human Performance Institute

Loma Linda University Medical Center

Virtual Reality technologies are technologies which support an experiential interaction with in a computationally sustained environment. Virtual Reality technologies represent a fundamentally new way for humans and computers to interact. For not only do these technologies translate natural human actions of communication, such as speaking eye-movements and body gestures into computer commands , but they also render information to the human in multiple sensory modalities, that is spatialized audio, 3D graphics and various somato-sensory forms.

To date the major effort of companies developing these technologies has been primarily , to cater to the military, entertainment and construction industries. This is very unfortunate. While no one will argue the economics' of this bias, there is a strong humanitarian consideration that is being neglected. It is true that these technologies will profoundly impact the entertainment industry, and they should, for interactive media is truly a vehicle for participatory governments of the future. However, if we miss this opportunity to fully exploit the positive humanitarian potentials in EDUCATION and MEDICINE this period in history will be looked upon as one of the truly great missed opportunities in techno-social evolution.

Here at the Center for Really Neat Research in Loma Linda University we have a moral imperative to "Dare to Care" and an operational mode of action of "Lead by Example" in our efforts to fuse High Tech with High Touch in the fields of medicine and education.

The following are some examples of our efforts:

Immediate Medical Applications for Virtual Reality Interfaces

Normal people are naturally enabled. They are born with the capacity to interact with the world and willfully manipulate their environments. Disabled people have lost the capacity for such interaction and manipulation through either trauma or disease. Advanced human-computer interface technology that has been developed as natural user interfaces for interaction with virtual reality has immediate application in re-enabling the disabled persons. While virtual reality promises to solve many problems in the future, the immediate application of these advanced interfaces can improve the lives of millions of today. At the Loma Linda University Medical Center we have had many successes in utilizing these technologies. The utility of these devices has already been demonstrated as augmentative communication devices, as environmental controllers, as therapeutic tools in rehabilitation and as tools for quantitative assessment for diagnostic evaluation. Patients who have lost the ability to communicate verbally have successfully used an instrumented glove configured in a gesture to speech mode. Spinal cord injury, stroke and traumatic brain injury patients have virtual reality technology to manipulate virtual objects and practice specific skilled motor tasks. Quadriplegics have used physiological input devices to move objects on the screen with only their eyes and to play virtual instruments merely by contracting face and neck muscles. These are just a few examples of immediate uses for this promising technology that can profoundly improve the quality of life of real people today.

The Virtual Reality technologies also are showing great promise in the field of psychiatry. In a recent experiment a real-time performance animation system was used to encode facial expressions of an actor and then generate a 3D talking head with realistic facial expressions that could interact with children that were in the hospital. This Virtual Teacher taught several classes on anatomy to a group assembled in a classroom and then made individual bedside appearances in the children's room over the hospital television system. The reaction of the kids (and the doctors) was overwhelming. The potential for a system such as this to augment the quality of life of hospital bound children is profound.

Now for some Theory.

Virtual Reality is a paradigm shift in the way we think about mass communication and information technologies. Consider the following:

Remapping the Human-Computer Interface for Optimized Perceptualization of Medical Information

In the distant past Medicine was an art, the practice of medicine was guided mostly by refined heuristics and intuition. All the external senses were used in the evaluation of the patient. Visual, auditory, tactile, olfactory and gustatory cues were all integrated to give the healer a perception of what to do. With the development of science and technology the practice of medicine has slowly shifted from being intuition based to being guided by the results of objective tests. In many ways this is progress, though in other ways we seem to have forsaken our own senses in favor of machines, thus removing ourselves from the determination of the problem. The ever increasing ability of technology to quantitate complex physiological parameters and to image volumetric anatomical structures are taking us to a point where we will soon be unable to assimilate all the available information through the traditional means (i.e., numbers and graphs). Recent attempts to solve this problem have focused primarily on advanced visualization techniques. While much progress has been made in this field, the visual sense is finite and is reaching its saturation level.

Enter Virtual Reality. Virtual reality technology is primarily interface technology that renders computer information onto multiple human sensory systems to give a sustained perceptual effect (i.e., a sensation with a context) while monitoring human response in the form of gestures, speech, eye movements, brain waves and other inputs. This interface also allows for a natural interaction with abstract data sets providing an integrated experiential encounter with information. This new technology provides us with the capacity to move into a new paradigm, a paradigm where the physiological integration of a pansensory rendering of medically relevant information provides an enhanced capability to discriminate between classes of complex dynamic interactions involved in pathophysiological processes.

Much attention has been given to enhanced visualization techniques. Dynamic volumetric stereoscopic rendering methods have greatly enhanced our capacity for visual assessment of medical information. We need however to be careful that we do not become photo-chauvinistic and forget that we have other senses. There are relevant concepts from sensory physiology that are now within the resolution of the interface technology. This new technology increases the number and variation of simultaneous sensory inputs, thus making the body a sensorial combinetric integrator. A good working knowledge of sensory physiology and perceptual psychophysics can help us optimize our future interactions with the computer. Aside from the basic neuroscience issues of modality, duration, intensity, distribution, frequency, spatial displacement, contrast, inhibition, threshold, adaptation, transduction, conductance and transmission (to name a few) we must identify the optimal perceptual state space parameters with in which information can best be rendered. We must also identify which types of information are best rendered by each specific sense modality.

New technologies and techniques have recently become available that allow for the rendering of data via auditory means. Not only can we now represent any data set in the form of sound but we can also spatialize the displacement of multiple sound sources giving us simultaneous exposure to different dynamic data sets. In these spatialized environments we can shift our attentional focus from source to source for real time comparison of multiple sets of data. Devices now exist which can stimulate the sensation of pressure, vibration, texture and temperature. This is a relatively untouched field as far as abstract data representation is concerned. These modalities combined with somatotopic placement also provide for spatial coding of the rendered information. The implementation of vision, hearing and touch technologies allow for simultaneous sensation of multiple independent and dynamic data sets that can be integrated physiologically into a single perceptual state.

Yet to be fully embraced by the virtual reality community are the olfactory and gustatory senses, smell and taste. While their current integration is questionable, their potential impact is quite profound. Recent work in olfactory science has identified at least 30 basic smells. Technologies under current development will be able to deliver quantified combinations of these smells for a wide range of distinct perceptual states.

In the area of taste the development of automated food processing will eventually allow for the doctor to get a taste of complex data. The use of smell and taste to help convey the state of complex systems may seem like quite a reach of the imagination. However, the possibility that these senses may help discern subtle changes in complex systems warrants investigation. We are embarking on an adventure that promises to change our relationship with the computer forever. With the immersion of all the external senses into virtual reality, our ability to perceptualize medically relevant information in an interactive mode will greatly enhance our capacity for improvisational investigation (stand up research). This is truly a paradigm shift and the beginning of a new era of computer assisted medicine.

THE NEUROREHABILITATION WORKSTATION:

A CLINICAL APPLICATION FOR MACHINE-RESIDENT INTELLIGENCE

Dave Warner, Jeff Sale, Stephen Price, Doug Will

Human Performance Institute

Loma Linda University Medical Center

ABSTRACT

The Neurorehabilitation Workstation is described. The need to maintain a clinical perspective motivates the comprehensive nature of the system, which integrates multiple data acquisition devices, interface technologies, advanced analytical techniques, and multi-sensory rendering capabilities. Emphasis is placed on machine-resident intelligence embedded at several levels.

INTRODUCTION

The field of Rehabilitation applies techniques and resources from many disciplines and is constantly seeking to improve the measurement of human performance and the assessment of therapeutic efficacy. We have had considerable success recently in our attempts to transfer new technologies into the clinical setting for such purposes. Devices such as gloves to measure hand motion dynamics, surface EOG and EMG sensors for eye movement and muscle contraction, and lightweight pressure sensor arrays for gait analysis show great promise in therapy. At the same time, our efforts to make these transfers permanent have been impeded by the lack of standard platforms, interfaces, inaccessible file formats, as well as the medical community's lack of time, technical expertise, and adequate budgets. Until now no cost-effective solution appeared possible. Recent developments in human-computer interface hardware and software, data analysis, and expert systems suggest this is no longer the case. We are currently exploring a solution, the Neurorehabilitation Workstation (NRW), which integrates these technologies and methods into a comprehensive system designed specifically for the clinic. In addition, we hope it may be generic enough to act as a standard for other similar applications. The success of the NRW depends on four things; modular design (for distributed processing and adaptability), integration of several data input devices into a single platform within a common interface protocol, implementation of machine-resident intelligence (neural nets, fuzzy logic) on several levels, and creation of a development environment driven by clinical needs. We detail aspects of these features below.

DATA INPUT

A necessary feature of the NRW is the integration of a variety of data input devices into a single system to include EEG, EMG, EOG, ECG, dynamic bend sensors, pressure sensors, audio and video digitizers, etc. The resulting capacity for data fusion allows for meaningful correlations to be made across various performance modalities. The devices and their hardware boards connect to an external module, and a high speed bus will route the data both to a central multi-tasking server and to the rendering subsystem for immediate feedback. The server should be intelligent enough to automatically implement a custom configuration of input device parameters, interface functionality, and relevant records based on the device(s) connected and the identity of the operator(s) and patient(s) currently at the system.

DATA MANAGEMENT

The maintenance of medical record integrity is a significant issue. Such integrity is achieved through security protocols, standardized data formats, error handling, and semi-automated database archiving. The data management subsystem tasks also include linking the device data with the patient record and specifying sensor-specific data formats and structures.

INTERACTION MODALITIES/METHODOLOGIES

The user interface will be based on new theories of human-computer interaction methodologies , computer-supported cooperative work, knowledge engineering, expert systems, and adaptive task analysis The system will monitor a user's actions, learn from them, and adapt by varying aspects of the system's configuration to optimize performance. Adaptable on-line knowledge-based help using text, graphics, and animated tutorials provide interactive learning and navigation.

DATA ANALYSIS

Effective therapeutic intervention relies on a comparative evaluation of a patient's progressing or digressing state. The nature of the change in this state may often be quite subtle, even imperceptible using traditional techniques. Given that the data acquisition subsystem can detect these changes, the data analysis subsystem is designed to enhance them in ways that may then be rendered to optimize the operator's sensory modalities. Linear and nonlinear multivariate analysis tools will be capable of processing multiple data sets in a variety of ways, including graphical analysis (phase portraits, compressed arrays, recurrence maps, etc.) and sound editing (mixing, filtering). Automated detection of trends and correlations using fuzzy logic may be performed in the background or in a post-processing mode. The user may then be alerted by the system if it detects areas worthy of further investigation. Such a feature should expedite the creation of a taxonomy of lesion-specific impairments.

USER CLASSIFICATION

We have defined five types of users. These types help define discrete levels of user functionality. Therapists, the primary users of the system, are responsible for data acquisition, data management, basic analysis, and patient-oriented interactive biofeedback modes. Technicians are responsible for simple data acquisition. Physicians will use the more comprehensive data analysis tools. Researchers will focus on the data analysis but their use of the system will be unconstrained. They will explore and develop custom analytical techniques. Patients will primarily use the therapeutic biofeedback features of the system, usually in a supervised setting.

DATA RENDERING MODALITIES

With multi-sensor data acquisition and advanced analytical characterization, the rendering capacity of the system becomes extremely vital. The NRW will implement multi-sensory rendering by combining recently developed 3D sound and tactile feedback systems with advanced visualization technologies. Research in human sensory physiology has shown the eye to be optimized for feature extraction of spatially-rendered data, the ear for temporally-rendered data, and the tactile sense for textures [9]. Thus the NRW will enhance perception of complex relationships by integrating visual, binaural, and tactile modalities. The rendering subsystem has a near real-time biofeedback mode for use in a therapeutic paradigm and a data perceptualization mode for use in an analytical paradigm. Outputs from sensing devices and analytical operations are parsed and routed to the combination of rendering modalities best suited to render that information.

CONCLUSION

The goals of the NRW are twofold; 1) to provide an open hardware platform and modular infrastructure which will expedite the implementation of new technologies into the clinic, 2) to augment clinical therapy with new methods of interaction and analysis. Success should result in providing neurorehabilitation, and the medical community in general, with a powerful tool for characterizing the complex nature of normal and impaired human performance.

Interventional Informatics Healing with Information

Dave Warner

The convergence of communication and computational technologies has created informatic systems which may be applied to social and cultural problems

When used appropriately, informatic systems have demonstrated the capacity to enhance the quality of life and to facilitate wellness of being through their applications in areas such as health care and education

Interaction between informatic systems and systems which they influence can be qualitatively and quantitatively modeled by generalized theories of complex adaptive dynamical systems.

Interventional Informatics is the intentional proactive utilization of informatic systems as complex adaptive systems to alter the course or outcome of a particular behavior languaging tools

INTERVENTIONAL INFORMATICS SPATIALIZATION OF INFORMATION

EXPRESIONAL SPATIALISM EXPANSIVE SPATIALIST

We live in the information age. Our lives are continually influenced by new applications of information technology. Informatic systems impact society at all levels, from personal and interpersonal dynamics, to global systems of immense complexity. Informatic systems have become so complex that their implementation causes certain unexpected and often undesirable behaviors to emerge in the individuals or groups involved. This often evokes a shortsighted linear reactive response in an attempt to gain some predictability, though this usually results in a worse state than before (take the public education and health care systems, for example). There appears to be a fundamental lack of understanding of these systems when they go beyond a certain level of complexity. Is there a way to resolve this?

A Suggested Solution

As part of our eternal journey to make the world a better place for expression at all possible levels, we recognize the need for a basic theoretical understanding of the dynamic relationship between information technology and society. If we understand how information technology is capable of affecting our lives, we may respond proactively to avoid some of these affects and to cause others. More commonly we find, however, that we have a need to understand information technology in order to respond reactively when it behaves unpredictably (understanding rarely means 100% predictability). In a reactive state, we look for critical points, state transitions, where familiar patterns may manifest in some system at some level and which hopefully will provide a meaningful link with an existing model. A major goal for complex information systems such as education and health care should be to reduce the need for a costly reactive response by taking proactive steps with information technology. With this in mind, we have coined the term Interventional Informatics (II) to describe the pre-emptive, proactive, or preventive use of relatively small amounts of information and information technologies at critically sensitive points on a system’s information state trajectory.

Theoretical Foundation

Most readers are familiar with the term "intervention" as it applies in a social setting (addiction therapy), or in a political setting (first world intervenes at a critical state in a third world country), or even a pharmacological setting (drug therapy). We may even call a vaccine an immunological intervention. Even though these all involve some form of information and they consist of acting either reactively or proactively at a potentially critical point in a system’s evolution to hinder or alter an imagined outcome, they do not necessarily depend on informatics systems to operate normally. Neither, for example, does the interventional act of informing someone that they may reduce the risk of AIDS by using condoms depend on informatics. Rather, interventional informatics involves putting an information-based mechanism in place which understands and uses the tremendous potential of small amounts of information, when applied at critical junctures over a range of scales, to alter outcomes in a positive way. How do we transition from a reactive to a proactive state? We are now at the point where we can benefit from applying principles of complex adaptive dynamical systems to large systems like the global communications network (Internet), although probably not quite yet with the same quantitative precision of neurophysiological or biochemical research. However, we are drawn by a strong qualitative similarity in the dynamics of seemingly different systems. To understand why this is so and what use this might serve, it is essential to recognize that at the very least, the behavior of many complex spatially-distributed dynamical systems appear to provide a metaphorical basis for some powerful interdisciplinary languaging tools. These tools may help us identify system behaviors familiar to us in other more well understood systems, at which point more formal methods of analysis may be applied.

Social Systems and Their Respective Levels of Complexity

It is useful to distinguish levels of social complexity at which information technology’s importance manifests. They include: the individual, interpersonal (intimate/non-intimate), group, community, district, city, county, state, region, country, continent, hemisphere, and planet. In academic medicine, it is also useful to consider a common non-intimate interpersonal system which manifests in different contexts but exhibits similar behaviors. It consists of one person who is being assessed and the other doing the assessment. These are: the physician /patient, the physician /student, and the student/patient. To a less obvious extent, there are also the License Commissioner/physician or practice, and the Medical school Accreditor/Dean et al. In each of these systems, interaction is bi-directional and, contrary to popular belief, when functioning properly the boundaries between the assessor and assessee are blurred. Each of the two system components should be providing informative feedback to the other about their performance, and adapting accordingly. These are reasonably simple examples of complex adaptive systems whose behavior differs in detail, but are quite similar in how information technology may be beneficially applied. They each involve a process of learning and assessment through informative feedback. In order to maintain proper function, it is critical to determine how information technologies are best applied to these systems.

The Role of Information Technologies in Assessing Learning and Performance

In general, improved retention, recall, and understanding may result from either of two basic processes. The first involves improved fidelity of the information being processed, i.e. a more accurate rendition of an existing representation. Examples would be modeling and simulation of a system, or in academic medicine, the case-based curriculum. The other learning process involves creation of a rendition richer in information because of changes in the nature of meaningful associations made with an existing rendition, perhaps through an increase in clarity, strength, or number of associations. This process includes principles of dual-coding, perceptualization (see below), and active learning. Interactive information technologies (i.e. any technology which responds in a nonarbitrary way to input from a user, and which in turn evokes a response from that user), and especially virtual reality technology, may potentially merge these two learning processes and serve to emphasize the important role of creativity in both information-rich (rule-based scientific) and entropy-rich (artistic narrative) behavior. In this regard, it is particularly important for our children to become adequately informed and capable of expressing themselves and informing others at all levels. Given the opportunity to develop their own expressive abilities and desires unhindered, we are certain they would turn both the education and health care systems around, and many others as well.

Case Studies in Interventional Informatics: Work In Progress

Recently, we have come to know an eight year-old girl named Ashley and a nine year-old boy named Trent. Ashley, who has been a C1 spinal (high quad) nearly her whole life, is an amazing young girl. She lives with her grandparents, and it was they who contacted us after seeing a television broadcast about our work with virtual reality and the disabled. Trent has only recently been physically challenged. Since we first met, they have learned to play video games, interact in meaningful ways with 3D objects in a virtual world, and drive a car, all with just their faces! By integrating off-the-shelf computer and data acquisition technologies with custom interactive software and hardware, we have created a system in which an individual may control their environment through their body's bioelectric signals , such as their muscles, eyes, and brain. Figure 1 represents a system in which a user may use their ability to control these signals in order to remotely control a robotic device, in this case the speed and direction of a small battery powered car equipped with stereographic cameras, microphones, speakers, and lights. The car receives visual and audio information from its environment, transmits it to the user, and receives controlling signals and audio from the user in response. Where is the informational intervention in the process of Ashley and Trent becoming more empowered to learn? Figure 1 represents an information trajectory which we have found to be useful in identifying system components and key points of intervention. The trajectory delineates the informational basis of a cognitive cybernetic loop. It is divided into a path representing the direction in which the user’s abilities are manifested, and a path representing the motivational feedback which makes the interaction a reality.

Ability

In Figure 1, the path representing ability contains one of the most important points of intervention, the bioelectric controlling software (BEC). This kind of informational intervention at the device level is more basic and inconspicuous than other levels, but no less critical. This software is used to calibrate the muscle signals and to define complex sets of gestures for various navigational tasks. These gestures suggest the term cyberlinguistics, and they may be looked upon as a sort of ‘cyberpidgin’; a collection of functional but grammarless phrases useful in commonly occurring interactions. It is possible, even likely, that these pidgins may develop into Creoles and ultimately full-blown languages as the need evolves. This suggests the potential for information technology as a foundation for some remarkably unique languaging tools. The task is to develop the cyberlinguistic navigational heuristics to use them. The successful operation of the system by Ashley or Trent hinges on the adaptability of this software. It is not yet auto-adaptive (it must be configured semi-manually), which would be yet another way of intervening at a critical point to enhance a user's expressive capacity.

Motivation

In rehabilitation, as in other types of learning, motivational feedback can be critical to rapid therapeutic progress. In the motivational component of this system, intervention is critical in the choice of rendering modes. By selecting binaural (3D spatial perception) and stereographic display modes, we have committed to mediums significantly more faithful to the source than text or 2D sound and video, and which provide a foundation for increased perceptual associativity. Ashley and Trent have become mobilized spatially-distributed vehicles of sight and sound. Through their learning and adaptive skills with their face muscles, they are now spatiotemporally extended well beyond where they were only a very short time ago. And of course, with the proper interactive informatics toolkits, these navigational skills may be used in a much more abstract way than simply driving a car. We have not yet given Ashley or Trent the opportunity to meet together in a world of their own making, but someday soon we hope to. Thanks to Trent and Ashley and a few others like them, in just a short time we have learned a great deal about complex adaptive systems; just look at their faces while they’re playing.

Modeling Complex Adaptive Behaviors

There are several properties of complex dynamical systems which may be useful in modeling the effect of information on society. Two important properties are nonlinear response and control parameters. The property of nonlinearity implies that the magnitude of a system’s response to some input is not directly proportional to the perceived value of that input, in fact it is usually far from linear. Additionally, the state of the population prior to the input of information strongly influences the response of the population to that input. There exists certain parameters in the population which strongly bias the response potential for a given input. These control parameters may be modified either internally or externally, thus providing a choice of mechanisms for altering a population's response potential.

Useful Similarities in Apparently Different Systems

Let’s consider two systems as examples, (1) the global information networks such as the Internet, and (2) bioelectric-controlled telepresence such as Ashley using electromyographic (EMG) signals from her face in order to perform telerobotics. Control parameters in the Internet might be the number of on-line information servers globally, or the number of users with hypermedia-based network browsing programs like Mosaic or Netscape, and a measure of complexity (an order parameter) might be the average rate of information flow. Analogously, the number of sensors capable of being attached to Ashley’s face, or the number of gestures available simultaneously for her to manipulate might serve as control parameters in that system. As we vary these control parameters, the complexity of the systems evolves, and we expect to observe rapid, distinct, and highly nonlinear transitions in their behavior. Evidence supporting this has been observed in the rate of increase in traffic (information flow) on the Internet, probably as a function in part of the control parameters mentioned above. Traffic jumped in such a way in the beginning of 1993 when the first multiple OS World Wide Web browsing program (NCSA’s Mosaic) began distribution1. We suggest that a corresponding situation exists in bioelectric telepresence, such that increasing the number of degrees of freedom (sensors) available to Ashley or the combinatorial capacity of the gesture definition software should lead to critical transitions in information flow. We have not tested this empirically, although such behaviors have been observed in learning procedural tasks similar to those in this system2. To do this with Ashley or Trent would require an extended study involving in part some intentionally controlled limitation of their capacity to interact with their world. This kind of study may hopefully be an easier one to consider in the future, but at present we feel it is not worth the aggravation and frustration such constraints place on children who are already constrained by their disability.

The Future

Where else might interventional informatics be applied? We used information technology proactively by anticipating the general need for Ashley and Trent to communicate with their families and friends, but we were surprised at the creative behavior of siblings and friends and what it implied: that not only were Trent and Ashley more empowered to interact with their world, the car provided part of Trent’s world, his siblings and friends, with a "vehicle" to interact with him, too! They were jumping in front of it, picking it up and flying it around. This sort of unanticipated behavior may emerge in other ways at other levels, and it should be accounted for in an accurate model of the system. We are actively exploring the areas of somatosensory and vestibular experiences, although we are still in the early stages. Many of those who are confined to a wheelchair lack much of the usual variety of these perceptual experiences. We are exploring how immersion in low frequency high amplitude sound, using speakers external and internal to the equivalent of a high tech Lazy Boy, can cause a meaningful somatosensory-induced deep visceral experience. We are also attempting to combine this with a vestibular component using motion platforms or suspension systems. These efforts represent our emphasis on perceptualization, which we define as the integrative experience of consciousness as the primary rendering state. The term originated in response to the severe visual bias so common in information technologies, and it refers to the integration of multiple sensory processors and their basic spatiotemporal pansensory features such as frequency, intensity, and complexity. On a more global scale, the telecommunications media have provided us with an initial link to some of those in society who could dramatically benefit from these technologies, but this is not a reliably predictable medium for a formal informatics mechanism. Our ability to manipulate the telecommunications media in it’s current state is severely limited as a result. It is critical that a more formal informatic infrastructure be put in place which virtually eliminates any reliance on luck to reach patients in need. The information superhighway offers promise towards this goal. It certainly allows us to test some of the most important ideas about complex adaptive systems. Interaction occurs on the Internet at many levels, with varying complexity, but with a common purpose, to proactively share information. The concept of "pulsing the net" at a critical state with information evokes images of echolocation, cybersonar.

Conclusion: Dynamical Similarity at Varying Social Scales

Finally, at a more basic and personal level, if we recognize that each one of us has emergent adaptive qualities in common with our environment, we may more readily recognize critical points in information flow between components of a system, and across systems of varying levels of complexity. Herein lies the essence of proactive informative behavior that we observe with the bioelectric controlling software, and in the global information infrastructure, and in many other systems dependent at least in part on information technology. Our relentless need to communicate, combined with an understanding of these technologies, as well as a recognition of the abilities and critical state of relevant social systems, should result in a dramatic increase in expressive capacity for society at all levels, all towards making the world a better place.