Human Computer Interaction Institute
School of Computer Science
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213-3891
(412) 268-7565
FAX: (412) 268-1266
bb46@andrew.cmu.edu
http://www.cs.cmu.edu/~pebbles
There are many signs of the approach of the ubiquitous computing era [Weiser 1993]. People are carrying Personal Digital Assistants (PDAs), such as Palm Pilots or Windows CE devices, and therefore have computing with them everywhere. "Smart Environments," where computing is embedded in offices and homes, are becoming a reality. Already, most conference rooms and classrooms at Carnegie Mellon University (CMU) have a built-in computer with a projector. And of course, every professor and most students have at least one computer on their desks in their offices. Most companies are similar, since virtually every white-collar worker today uses a computer. Homes are just starting to be set up with embedded computing, and the "Smart Home" has been widely anticipated [Levy 1999].
One aspect of ubiquitous computing that has not been adequately studied is how a user will use multiple devices at the same time, so the devices work seamlessly together. Most of the research and development about hand-held and mobile computers has focused on how they can be used to replace a personal computer (PC) when the PC is not around. The conventional model for PDAs is that the data is "synchronized" with a PC once a day using the supplied cradle, and otherwise the PDA works independently. This will soon change. For example, CMU has installed a Lucent Wavelan wireless network (using the 802.11 protocol) throughout the campus, in a project called "Wireless Andrew" [Hills 1999]. Many Windows CE hand-held computers can be connected to this wireless network using a Wavelan PCMCIA card. Other mobile devices will support this protocol soon. Next year, the BlueTooth standard for small device wireless radio communication will finally be available, and most PDAs, cell-phones, and other computerized small devices are expected to support it. Therefore, we expect that connecting the PCs and hand-helds together will no longer be an occasional event for synchronization. Instead, the devices will frequently be in close, interactive communication.
The Pebbles research project (http://www.cs.cmu.edu/~pebbles) has been studying the implications of this trend, especially on how the functions, information and user interfaces can be shared across multiple devices in use at the same time. For example, there are many ways that a PDA can serve as a useful adjunct to a personal computer to enhance the interaction with existing desktop applications. New applications may distribute their user interfaces across multiple devices so the user can choose the appropriate device for each part of the interaction. A key focus of our research is that the hand-held computers are used both as output devices and as input devices to control the activities on the other computers. The following scenarios illustrate some of the capabilities we are already investigating:
There are many significant research issues involved in bringing these visions to fruition, which we are investigating. We are particularly interested in the appropriate ways to distribute the user interfaces across multiple devices, how to support multiple people interacting with the same screen using their various devices as auxiliary input and output devices (which is sometimes called "single-display groupware" [Stewart 1997]), the automatic creation of appropriate and usable control panels from high-level specifications, and usability issues with multi-device interaction techniques.
The Pebbles research project has made substantial progress by building example applications, releasing them for general use, and formally testing them in usability experiments. Several of our existing applications support meetings where the participants are co-located. All participants' PDAs are in continuous two-way communication with the main computer which is often projected on a screen to serve as the focal point of the discussion. Some of our initial applications use the PDAs as remote mice and keyboards so that everyone in the meeting can control the main computer. The PDA might be used to control a PowerPoint presentation while displaying the slide notes and titles on the PDA, as a shared whiteboard that supports multiple inputs simultaneously, for private side messages via a "chat" program, and to display multiple cursors for pointing and scribbling on arbitrary applications [Myers 1998]. We are currently investigating a number of groupware issues, including appropriate floor control mechanisms, and how to fluidly move information between the public and private displays. Another set of applications supports a single person using the PDA as an extra input and output device to enhance desktop applications. The PDA can be used as a scrolling device, as a general-purpose button panel (to create screens of "shortcuts"), as an index page or table of contents for web surfing, and to cut and paste information back and forth from the PDA to the PC. These applications have been downloaded over 15,000 times already, and are available from our web site: http://www.cs.cmu.edu/~pebbles.
Benjamin Bostwick of the Pebbles research project would like to participate in this workshop because we are interested in discussing how the devices should adapt to the situation and context, especially in terms of which other computing, input, and output devices are available in the area. If there is a big display on the wall, what parts of the user interface should migrate there? But what if someone else is using the big display for a side discussion? The consideration of context would enhance our work on the use of multiple devices at the same time. Conversely, considering the use of multiple devices simultaneously will expand the range of issues in situational context.
[Hills 1999] Hills, A., “Wireless Andrew.” IEEE Spectrum, 1999. 36(6)June.