The Phantasticon.

To Use the Sense of Touch to Control a Computer
and the World around You


Carl Sjöström, Ph D student Calle.Sjostrom@certec.lth.se
Bodil Jönsson, Ph D, Ass prof Bodil@certec.lth.se
Center of Rehabilitation Engineering Research at Lund University,
Box 118, S 221 00 Lund, Sweden

The 4th European Conference for the Advancement of Assistive Technology (AAATE'97)
Thessaloniki, Greece, September 29 - October 2, 1997




Table of Contents

Abstract
1. Introduction
2. A short description of the PHANToM
3. Making a Phantasticon out of the PHANToM
4. Software development for the Phantasticon
5. Work in progress
6. "Touch Windows"
7. Conclusions
References
Contact persons


Abstract





l. Introduction

Computers are becoming everyday technology for more and more people. Computers have opened up many doors for disabled people, for example it is now rather easy for a blind person to access written text. Any text in a computer can be read either with a one row Braille-display or a speech synthesizer. This is done in real time and is of course much more flexible and less space consuming than books with Braille-text on paper. There is a big problem though: Since the introduction of graphical user interfaces computers are getting easier and easier to use for sighted people, but GUIs has the opposite effect for non sighted people. The many fast accessible objects on a Windows desktop becomes a big mess for a user who can't see them. And if you don't know what is on the screen it is almost impossible to control the computer.

The PHANToM
This is where the haptic interfaces can be a good help. With the PHANToM or a similar device it is possible to feel things represented in a computer. CERTEC has developed a set of programs which demonstrate different ways for a blind user to control a virtual reality with finger movements and to get feedback via the sense of touch. One of the big tasks for the future is to make the Microsoft Windows environment completely accessible through haptics. If you can feel the START-button in the lower left corner etc. it is not only possible to control the environment, but it is also possible to start speaking about how to do things since both sighted and non sighted users have a common ground to start from.

CERTEC is working with the meeting between human needs ana technical possibilities. Normally we start with the human needs and develop technical solutions from that aspect. Sometimes though it is motivated to start from the other side.

In this case we have used a technical solution which has been developed for other purposes and modified it to correspond to the needs of a disabled person. We have turned the PHANToM into the Phantasticon.


2. A short description of the PHANToM

The PHANToM is a small robot which acts as a haptic interface between a human and a computer. A normal program uses vision, sound, a keyboard and a mouse for the interaction with a user. The PHANToM adds a new dimension to human computer interaction, namely haptic interaction. Haptic interaction uses both the sense of touch in a small scale and movement in a slightly bigger scale.

It is not unusual to connect a robot to a computer as is done with the PHANToM. The special thing in this case is that movement and sense is used for interaction between the human and the computer. With the PHANToM a user can feel objects which are represented inside the computer. At the same time one can use movement to give commands and to get feedback from the program.

When activated the PHANToM works together with the computer to interpret the users finger position in three dimensional space and apply an appropiiate and variable resisting force. This process is completed 1000 times per second.


3. Making a Phantasticon out of the PHANToM

When the PHANToM is extended to meet the needs of disabled persons it becomes a complete system. This system includes the PHANToM itself, and the software from CERTEC. It also includes a lot of ideas and thoughts about what can be done for people with special needs using this hardware and software. Altogether this is what we call the Phantasticon (Fantomaten© in Swedish).


4. Software development for the Phantasticon

CERTEC is continuously developing programs for the Phantasticon. At this moment we have the following programs ready:

"Paint with your fingers"
A program with which the user can paint computer pictures with a finger. One can choose a colour from a palette and paint with it on the screen. The harder you push with your finger, the thicker becomes the line. Each color has an individual structure. When you are painting you can feel the structure which is being painted. You can also feel the structure of the whole picture by changing program mode with a simple click on the space key.

"Mathematical curves and surfaces"
Mathematics is a partially visual subject. That is often noticed by people who try to explain mathematics for blind persons. With the help of the Phantasticon also blind persons can learn to understand equations as curves and surfaces. CERTEC has developed a program which makes it possible to feel any mathematical curve or surface with the PHANToM.

"Submarines"
"Submarines" is a PHANToM variant of the well known battleship game. The player can feel 10x10 squares in a coordinate system. In the game your finger is a helicopter which is hunting submarines with depth charge bombs. If you put your finger on the "water surface" you can feel the smooth waves moving up and down. The surface feels different after you have dropped a bomb, and it also feels different if a submarine has been sinked.

This computer game uses the PHANToM, the screen and the keyboard for the interaction with the user. It also uses sound effects as most games do nowadays. "Submarines" is one of the first computer games which can be played by a deaf blind person.

Marrie



5. Work in progress

The big efforts at this moment is laid on developing a general user interface which is easily accessible for blind people. As a test bench for haptic interface objects and at the same time a haptic computer game we have developed Haptic Memory. The task for the user is to find pairs of sounds which are played when the users pushes different buttons. The Memory program is a good base to find out how different parts of a haptic interface should be designed to work as good as possible for low vision users.

The Haptic Memory has also been expanded into "the HOuSe". The HOuSe is CERTECs "first steps towards a Haptic Operating System". The HOuSe is a bigger haptic memory with five floors and five buttons on each floor. With this program we can gain some experience about how blind persons can use haptics to build inner pictures of complex environments. That knowledge is an important cornerstone when we start building the complete haptic windows system or other haptic programs for visually disabled people.

"The HOuSe" has been tested with four children and five adults. All of them are blind. A reference group of 21 children in the age of 12 has tested a program with the same function and layout, but with a graphical user interface. Since the haptic interface has the same layout as the graphical interface and both programs work the exactly same except for the way of interacting with the user it is possible to compare to results of the blind users with the results of the sighted users. All of the blind testers had a little more than one hour of experience with the phantom before the test started.

The tests show that a majority of the blind users could complete the task using about as many button pushes as the sighted users, but the blind users generally r.eeded more time. The differences in the results where bigger in the group with blind users and two of them did not finish the task at all.

The results imply that it is meaningful to keep trying to make graphical user interfaces accessible for blind people using haptic technology. Most of the blind users showed big confidence when using the haptic interface even with the rather limited experience they had and the differences in time will probably be lower after more training.


6. "Touch Windows"

As mentioned in the introduction there is a big problem for non sighted users that computer nowadays mostly have graphical users interfaces. However, Windows and other graphical user interfaces are widespread and accepted, so almost all new programs are made for those environments. The good thing is that if one can make windows accessible for non-sighted users then almost automatically the windows programs are accessible as well. That is why we have started the "Touch Windows" project.

Our idea is that haptics with the Phantasticon can provide the overall information and that sounds, speech synthesis and Braille-displays can provide the details of the interface. In this way it is possible to use the same or at least similar interface metaphors for both sighted and non sighted users. People can work together and help each other and it doesn't make a big difference whether you see what's on the screen or if you feel it, you get the same inner picture anyway.


7. Conclusions

One of the key elements in the process of learning is variation. This implies that an already existing knowledge is a vital precondition for new luiowledge. For a blind person, vital parts of the experiences of the world around them are touch based from the very beginning. Thus, it is a most exhilarating possibility to use the Phantasticon for widening the touch based experiences and learning outside an armlength's distance. So far, we have not come across any unclimbable obstacles.


References

CERTECs homepage and our page about the Phantasticon:

Sensable Technologies are the makers of the PHANToM haptic interface:

MIT Touch Laboratory, are working with the PHANToM:

University of Delaware, another developer of PHANToM applications for the blind:

MIT Artificial Intelligence Laboratory, where the PHANToM was born:



Contact persons

Carl Sjöström
Bodil Jönsson Ph D

CERTEC, Lund University
Box 118
S-221 00 LUND
Sweden

Tel: +46-46-222 46 95
Fax: +46-46-222 44 31