Story by Judy Holmes
jlholmes@syr.edu
Phone: 443-3784
Story ran in the April 6, 1998 Syracuse Record
Suzanne Henderson Kendrick knew her 6-year-old daughter, Brooke, understood colors, numbers and letters just like any other kindergartner. The problem: Brooke has cerebral palsy, which has left her unable to speak or control her body's movements. She couldn't tell her teachers what she had learned.
Eyal Sherman, 17, wants to learn to surf the Internet just like his classmates at Nottingham High School, and he wants to use the computer as a communications tool. The problem: Complications from a life-saving operation to remove a tumor from his brain stem when he was 4 years old left him totally paralyzed-he is unable to speak, move or breathe on his own. His parents, family and close friends have learned to read his lips. But as Eyal looks forward to college, he needs to find a better way to be understood. Enter the Center for Really Neat Research (CRNR) at Syracuse University, comprising University faculty, alumni and students who are dedicated to helping people with disabilities access computers to improve their quality of life.
Using a combination of tools they created-widgits, TNGs (pronounced "things") and NeatTools software-CRNR researchers have given Brooke and Eyal an opportunity to break through their physical barriers and share their thoughts, talents and creativity with the world, using the computer as their voice.
Humans commonly interact with computers through a keyboard and a mouse. But neither device works for people who cannot move their arms or use their hands. David J. Warner, CRNR founder and Nason Fellow at SU's Northeast Parallel Architectures Center (NPAC), has turned the computer world upside down by looking beyond traditional human/computer interfaces.
The goal is to find a path for communication between humans and computers, he says. Once people who are disabled can use a computer to communicate, they can become employed, productive participants in society.
"We look at what movements humans can do and then figure out a way to use computers to display it," Warner says.
Here's how the system works: Body movements create signals that are captured by sensoring devices called widgits. The widgits send the signals to an interface box called the TNG-for "totally neat gadget"-and the TNG converts the signals into a stream of data that NeatTools software understands. NeatTools uses the data to communicate with the main computer system. When the connection is complete, the user can control a mouse and access any program on the computer, including a Web browser.
"If people can blink their eyes, then we can make an eye-blinking detector. If they can wiggle their cheeks, we can make a cheek wiggler detector. And so on," Warner says.
Warner and Edward Lipson, professor of physics and a CRNR researcher, call the system "an expressional interface." "We want to be able to capture anything humans can do as an intentional communication," Lipson says. "`Expressional' implies there is an intention."
Accessible and affordable are the key ingredients in all the interfaces CRNR researchers create. NeatTools, developed by Warner and Yuh Jye Chang, a programming theory doctoral student in the L.C. Smith College of Engineering and Computer Science, can be downloaded free of charge from the center's Web site ( http://www.pulsar.org ).
The TNG-3 interface box, developed by Lipson, costs about $100.
The widgits that Eyal and Brooke are using were created by research teams that included SU industrial design students. They were made from spare parts, toys and inexpensive devices the teams obtained at discount and electronic supply stores.
"If we can't fix it at midnight, if we can't find it at Radio Shack or Wal-Mart or some generic place, then we are in the same trap as the people we are trying to replace," Warner says. "We don't want to become what we seek to destroy-expensive, proprietary, function-limited, hard-to-access technology."
Rahul Panesar, an NPAC research assistant whom Warner "commandeered" to help build the CRNR's research laboratory, was put in charge of Brooke's case. Warner and Panesar took an inventory of Brooke's movements to determine what kind of widgits would be needed to help her access a computer.
Last fall, Panesar recruited four industrial design undergraduates in the College of Visual and Performing Arts to create widgits for Brooke. The fourth-year design students-Michael McKibben, Jonathan Provost, Michael Lashombe and G. Banning Rowles-took on the project as part of a class assignment. They visited Brooke, now a first-grader-in her classroom at Lakeland Elementary School, and on Friday evenings at the CRNR worked with her, her mom and her physical and occupational therapists.
The experience opened Brooke to a whole new visual world with which to interact. Panesar's team introduced Brooke to age-appropriate computer games. She quickly learned that by manipulating the experimental joysticks the students created for her, she could control what was happening on the computer screen. For the first time, Brooke was showing her therapists what she was capable of doing.
"Brooke was learning faster than we were at the CRNR," says Jean LeManczyk, Brooke's physical therapist. "The students were great. They treated her just like any other kid."
Each student worked on a different solution for Brooke. Some succeeded, others didn't. "While we studied and mapped out her motion patterns and inventoried her movements, 99 percent of what we did were educated guesses," Provost says. "It's not an exact science."
During the process, a special bond developed between Brooke and the students. Brooke breaks out into a beaming smile when you ask her about "college and her boys."
"It's really rewarding to create things where you see a direct, positive impact," Provost says. "We made things for Brooke that really helped her."
Today, Brooke is practicing her mouse skills with a "duck-billed" joystick handle that the students developed for her and a button she clicks with her head. NeatTools, installed on a computer at school, enables Brooke to interface with a variety of developmental programs. When her precision improves, Brooke will be able to use special software and the joystick/mouse to type her thoughts on the computer.
"The computer will give her a voice," Kendrick says.
Lipson first introduced Warner to Eyal two years ago, before TNG-3 was developed and when CRNR researchers were using an earlier version of what is now NeatTools that was designed in Warner's California laboratory. When Warner and Chang scrapped the old software and began building NeatTools at SU, many of the functions they wrote into the software were the direct result of what they learned from Eyal and later from Brooke.
"We did a little bit, and they showed us where it was weak," Warner says. "So we changed things and did a little more. Without Eyal and Brooke, we would not have the software we now have. It is almost real-time science. We go from the basic research to application and production in a very short time cycle."
Lipson led a CRNR team in determining how to capture the motions Eyal could make and use them to access the computer. The process has taken two years. Eyal can move his cheek, forehead and chin muscles. Industrial design students helped develop widgits to hold cheek sensors and a chin joystick so he could gain mouse control.
Panesar and Michael Konieczny '97 (a VPA senior at the time), who is now working at the CRNR, designed the chin joystick. They developed their prototype by tearing apart old remote control radios and incorporating the devices that enable the knobs to move in different directions into the joystick.
The primary challenge of the "widgiteers" was finding a way to mount sensors near Eyal's face. A variety of widgits were developed, but until a few months ago none of the ideas were very practical.
Then, one day, Warner was hanging out in a Kay-Bee toy store when he stumbled upon Power Rangers action figures. The jointed arms, he theorized, might help resolve the problem. He showed the toy to Konieczny and Panesar. A competition ensued among all the widgit-builders to see who could find the cheapest Power Ranger. They searched garage sales, their homes and discount stores.
"We started butchering GI Joes and Power Rangers to build arms that would hold the sensors on Eyal's face," Konieczny says.
The Power Ranger arms connect Eyal's eye glasses to the cheek sensors. The chin joystick is held in place on his chest by a table-top extension arm. TNG-3 allows analog signals from the chin joystick and digital signals from simple mechanical switches mounted on Eyal's cheeks to flow simultaneously into NeatTools.
The real breakthrough came when Lipson used NeatTools to create a program that allows Eyal to precisely control the position and speed of the mouse with his chin joystick. The Joy Mouse, as Lipson calls it, was introduced to Eyal in February. For the first time, he was able to surf the Internet and independently use the computer to communicate his thoughts to the world.
Click here to return to the release menu