Isaac - A Personal Digital Assistant
for the Differently Abled

Bodil Jönsson and Arne Svensk

Proceedings of the 2nd TIDE Congress, pp 356-361,
Paris, France, April 1995


Designed to be used as an aid for individuals with cognitive dysfunctions, Isaac combines in one unit a pen-based computer, a digital camera, a GPS satellite navigation receiver, and cellular phone channels for both voice and data. This paper deals with Isaac as an experimental product, the arrangement and procedure of the ongoing user test and its results so far, and the challenging questions and ideas raised by the mere existence of Isaac.

1. Introduction

The aim of the Isaac project is to increase the independence of differently abled persons by enabling them to do more on their own and to decide themselves when they want to use the option of interacting with their relatives or the staff. A secondary purpose of the Isaac project is to find out how advanced technology affects daily routines and influences organizational structure.

The project has grown out of an existing cooperation between group homes in Lund for adults with cognitive dysfunctions and CERTEC, the Center for Rehabilitation Engineering at Lund University. Since September 1994, when the initial technical phase was completed, there has been a systematic search for different kinds of user experiences.

Fig 1. Isaac is based on Sharp´s version of Apples Newton
with a CCD camera and phone handset integrated in a modified cover.

2. Isaac - an experimental product

Isaac combines in one unit a pen-based computer, a digital camera, a GPS satellite navigation receiver, and cellular phone channels for both voice and data. The handheld unit contains a miniature still camera. A picture is taken when a viewer window on the screen is touched, and a moment later a greyscale image appears. The picture can be immediately transmitted to the support center or just stored to be included in the personal picture data base and later printed on paper with photorealistic quality.

A receiver for the GPS satellite navigation system provides accurate coordinates in most outdoor locations. The latest position is stored and can be retrieved by the support center at any time. When privacy is preferred, this function can easily be disabled by the Isaac user.

Sequences of images on the screen can be used to guide the user in navigating on a walk, performing a repetitive task at work, cooking a meal, or doing the morning ritual properly.

The user operates Isaac through an LCD touch screen capable of showing greyscale images with reasonable quality. Interaction is based on symbols and pictures, avoiding text according to the user needs. The telephone directory is shown on the screen as faces instead of names and phone numbers. Dialling is automatic after touching the appropriate picture, and the back of the screen contains the handset.

A number of such mobile units can be in wireless contact with a support center providing assistance over the phone based on pictures, position data and other kinds of information managed by the system.

Isaac in use in six different situations
Fig 2. A wealth of capabilities

Although targeted for a special application, Isaac has the potential for a much wider use. The emphasis on multimedia and communication puts Isaac in the forefront of PDA technology as an example of future personal computing. Further information on the Isaac technology and its wealth of capabilities can be found in [1]. The chief scientist and project manager of the technical part of the Isaac project was Professor Lars Philipson.

The support center could be placed at home, at a day center or anywhere where relatives, friends or personnel are always available. The support center may even be mobile. It has a Windows PC running a set of applications as an integral part of the Isaac system. All incoming photos are marked with the name of the user and stored in the database. The map where the position of a user can be shown is managed by a fullfledged Geographical Information System capable, for instance, of locating street addresses. Most activities during an ordinary day can be scheduled and alarms set to remind the user if necessary. On the Isaac screen, activities appear as pictures or symbols (pictograms) positioned relative to a vertical time line. The present time of day is always located at the top, and the screen shows the next few hours of activity.

Up to 60 pictures can be stored temporarily in the bag and later transferred using a high speed wire connection to the support center computer. As part of this procedure all pictures can be previewed, some possibly discarded, others perhaps annotated.

The accepted pictures are stored in the database and can later be searched and selected interactively to be printed out as part of pages for a photo album or a diary. A similar procedure is used to select pictures for guiding sequences and personal telephone directories.

Isaac User

Fig 3. The handheld unit

3. User test arrangement and procedure

The main interest in testing the usage of Isaac is to reveal new knowledge about how a tool like Isaac may influence the life of a person with cognitive dysfunctions, his/her relatives, the personnel and the surrounding community. The experimental product series comprises 25 handheld Isaac units. The user field tests are being performed in the south of Sweden by three different centers: CERTEC at Lund University and (starting January 1995) the Rehabilitation Center Lund-Orup and HADAR in Malmö.

Autumn 1994 was a probationary period with a target group of mainly personnel working with differently abled people. There was a twofold reason to start with this group to get user experiences. Firstly, the usage of Isaac might influence not only the everyday life of mentally retarded persons but also the everyday life of people in their surroundings. Relations and roles might change. Thus, efforts to catch glimpses of the personnel´s attitudes towards Isaac and their willingness and abilities to work with it have a value per se.


Secondly, the initiation of carefully prepared tests involving differently abled persons has to rely on persons (personnel or relatives) who are both interested and competent in taking over the responsibility of a user test involving people with mental retardation. The responsibility is not restricted to the prevention of dangers but also includes ingenious invention of relevant Isaac possibilities for each person in question. The personnel is needed also for improvisations in completely new situations and to help evaluate the experiences.

More than 60 people from personnel at group homes, day centers and schools for differently abled have attended the first user tests. On four different occasions during a fortnight, each of them visited the Isaac resource center to get the first insights into how Isaac and the computer at the support center work. During the days in between the visits to the Isaac center, the participants have had Isaac units at their disposals. They have used them in their spare time as well as during working hours. So, Isaac has made its debute also in the heads and hands of differently abled persons. The first one who ever held an Isaac unit in his hand chose three different objects to photograph: a very handsome young girl, a CD-player and a clock radio.

The results of the probationary period so far are :

a. Isaac works as a liberating pedagogy for personnel. Everything around Isaac is very concrete. To use it, you have to consider that a person always is some-where, that a picture always represents something, that an occurrence always happens at a certain moment. The discussions initiated at the Isaac center were not only concrete but also tight. One could not avoid difficulties and stick to prestige words. One had to consider what, when, where, and how.

b. The partipicants came to focus on Isaac´s ways of changning the working day and the content of the work. Isaac trains another ability than the existing one since it forces you to think of possible happenings (and illustrate them through sequences) in advance. Isaac might enable personnel to be absent to a degree not known today, giving the freedom to the differently abled persons to seek contact when they want. They will get new chances to learn stealthily. They will also get new chances to show what is important to them when they have the pictures as a means of communication.

c. People in the surroundings showed a new interest in the personnel and their work when they brought an Isaac.

d. Digital pictures and the possibilities of easily combining pictures to form different patterns awoke a lot of interest. And so did picture communication and the possibilities of learning on the basis of pictures.

e. A lot of ideas arose about the next step involving individuals with mental retardation. Some of the participants during the probationary period initiated and will be part of projects at school, at day centers and in group homes. These will start in February 1995.

4. Discussion. Challenges

The technological revolution in our western civilisation has resulted in an abstract way of dealing with the world around us. This causes great problems for people with cognitive disabilities. Still, it is possible to find structures that can act as guiding lights - as theories of technology for people with mental retardation. Once you are familiar with these criteria, the complex becomes manageable. It is like the picture below that appears to be a spiral although it is really made up of circles (check for yourself).

Fig 4. The background pattern tricks the brain into perceiving the
circles as asingle spiral. In technology for the mentally retarded
and elsewhere, help is needed in taking away the background.

Humans are said to err in their technological surroundings. Most often the truth is that the things we use are badly conceived and designed. This has been very well illuminated by D. Norman in The Psychology of everyday things [2]. For those with learning disabilities the situation is even worse. Their possibilities of handling the modern world are much more limited. So are their supply of cognitive aids [3].

This absurd situation was the background of the birth of Isaac: a tool that could give people with mental retardation a technical advantage over the elite in society and business. CERTEC´s basic assumptions are that robots should be intended primarily for people with physical disabilities, modern optics for people with visual impairments, and cognitive aids for people with mental retardation [4]. MMI, Man Machine Interaction, should not stand for a meeting between man and machine but for a meeting between human needs, wishes and abilities at the one hand and technological possibilities at the other.

The challenges now stimulate our imagination and push its limitations. Surely, Isaac will enter school work during spring 1995 and enable a pedagogy based on picture communication. But how wide are the doors that it will open? Isaac will be introduced in group homes and day centers to serve as a tool for differently abled people and as a bridge to their personnel. But which will be the most prominant features of the increased freedom and safety? The attitudes of the surroundings towards mentally retarded people will be influence by their usage of Isaac. But how? Books on technology and cognitive science, e.g. The computer and the mind [5], could be a guide, but not more. We are entering a new field with new possibilities, inspired by many different scientists. One of them, Philip W. Anderson, a condensed-matter theorist, states in [6]:

More is different.

At each level of complexity entirely new properties appear.

The understanding of new behaviors requires research.

At each stage entirely new laws, concepts, and generalizations are necessary, requiring inspiration and creativity to just as great a degree as in the previous one.

Isaac might enable many differently abled persons to experience much more. More is different. The complexity will shape a situation with entirely new properties, laws and concepts. We hope to be able to develop a research based on inspiration and creativity, having fruitful discussions with research colleagues and each other as well as with experienced personnel and relatives.


[1] Isaac, A personal Digital Assistant for the Differently Abled. Brochure from CERTEC, LU, Box 118, S-221 00 Lund, Sweden, 1994

[2] Donald A. Norman, The Psychology of everyday things. ISBN: 0 465 06709 3. Basic Books, 1988

[3] Arne Svensk and Bodil Jönsson, Teknik och förståndshandikapp. ISBN: 91 27 04135 2. Natur och Kultur, 1994 (in Swedish). To appear in English "Technology and Differently Abled People"

[4] CERTEC, Brochure, LU, Box 118, S-221 00 Lund, Sweden, 1994

[5] Philip Johnson-Laird, The computer and the mind, ISBN: 0 00 686299 3. Fontana Press, 1988

[6] Philip W. Anderson, More Is Different, Science 177 (1972) 393-396