| The Design and Use of History List Systems for Rehabilitation Robots: AUTHORS and AFFILIATIONS:
H.F. Machiel Van der Loos, Ph.D. Biomedical Engineer Rehabilitation R&D Center VA Palo Alto Health Care System 3801 Miranda Ave. #153 Palo Alto, CA 94304-1200
Larry J. Leifer, Ph.D. Design Division Dept. of Mechanical Engineering Stanford University Stanford, CA 94305-4021 office: 650/723-1869 650/723-3521 fax
ACKNOWLEDGMENTS: This paper is based on the 1992 Ph.D. dissertation by H.F.M. Van der Loos, whose academic supervisor was L. Leifer. The authors gratefully acknowledge the major contributions by David Lees, Ph.D., the designer and programmer of the original DeVAR interface, and Joy Hammel, O.T.R., Ph.D., clinical collaborator and researcher. The authors are also indebted to Bob Yee, who used and helped evaluate DeVAR for two years in his office: his operation of the system generated the history list data on which this work is based. Biographical sketches: Machiel Van der Loos, Ph.D., has been involved in the VA/Stanford Rehabilitation Robotics Program since 1979 in various roles: graduate student, mechanical designer, system integrator and, most recently, project director. He received the Ingénieur Mécanicien degree from the Swiss Federal Institute of Technology in Lausanne in 1979, and an Engineer's Degree (1984) and Ph.D. (1992) from Stanford University in Mechanical Engineering, all in robotics. He has a special interest in the use of mechatronics in rehabilitation, human-machine interface design and assistive device assessment for the clinic. Larry Leifer, Ph.D., received a BSME in 1962, a MS degree in Product Design in 1963 and his Biomedical Engineering doctoral thesis in 1969, all from Stanford University. From 1969 to 1973 he studied human information processing at the NASA Ames Research Center and at the MIT Man-Vehicle Laboratory. From 1973 to 1976 he was an Assistant Professor of Biomedical Systems Analysis at the Swiss Federal Institute of Technology in Zurich. A member of the Stanford faculty since 1976, he teaches a Graduate Mechatronic Systems Design course and a Design Theory and Methodology Forum, and directs the Engineering Design Affiliates Program. He developed the Smart Product Design Laboratory and taught that curriculum from 1978 to 1988. He is founding director of Palo Alto VA Rehabilitation R&D Center and the Stanford Center for Design Research (CDR). Special interests include: a) design knowledge capture and reuse; b) assistive telerobots for physically limited individuals; c) distributed concurrent product realization environments; and d) Synalysis-Exercise modules for design education and learning assessment. In an effort to disseminate assistive devices developed at Stanford and the VA, he co-founded of the Tolfa Corporation (1989) and Independence Works, Inc. (1992). IWInc is an orphan product development enterprise that uses the Internet World-Wide-Web to bridge the technology transfer gap between research laboratories and disabled consumer markets. ABSTRACTIn interactive computer controlled systems such as robots, history lists store records of user and system events. The history of commands, system states and actions taken by an intelligent robot is important to safety, performance, service and quality assurance. These records are commonly referred to as "history lists". They are an objective account of system operation that can serve the needs of users, analysts, designers, management and regulatory agents. A well-known example of a history list system is the aircraft flight recorder. They are also used in chemical process control and software usability testing. These areas are distinguished by the high cost of system malfunction and the high value placed on continuous availability. For semi-automated health care systems such as human-service robots in rehabilitation, surgery and patient care, a well-designed history list recorder and analyzer can help ensure safe and effective system operation. However, history list usage is largely unexplored in robotic systems. This paper presents a history list design methodology for interactive robots. The design framework consists of five guidelines, which are defined and discussed. The guidelines are derived from two years of experimental history list usage, during which time 3000 hours of history list data and 30 hours of videotaped robot operation were recorded. The history lists were recorded by a data acquisition system originally designed into the interface of DeVAR, a VA/Stanford-developed, voice-controlled, desktop robot for individuals with physical disabilities. Analysis of the data, using the five guidelines as a framework for discussion, shows how history lists can be a useful tool for designers and operators. Results focus on methods for debugging robot interfaces and developing chronicles of actual use. INTRODUCTIONNeed for history list softwareA computer system interfaced between a human operator and a physical process is capable of recording and storing the operator commands and process events that it mediates. A collection of such time-stamped data records is a history list (HL). History lists provide objective operational data on processes such as a telerobot handling task, a computer programming session, aircraft piloting, and power plant supervisory control. The social cost of malfunction and the potential benefit of performance enhancement are commensurate with the mass, energy, and information flow in these process domains. The ability to do task analysis, cognitive modeling, usability assessment, accident analysis, simulation, playback, and operator training on the basis of objective historical records provides a powerful tool in the overall design process [Hartson & Hix, 1989]. Although HL systems exist for the aircraft and process control industries, there is no such tradition in health care and human service automation, both of which share similar human safety and system performance issues. This paper describes a set of guidelines for effective HL design in the area of interactive human service robotics. Definition of a History ListIn the absence of an accepted definition of the term "History List" in the literature, and to standardize the meaning for this paper, we offer the following definition: A History List is a chronological record of system activity. The following conditions are required: 1. The record must be generated by an objective observer. 2. The system includes operators, machine controllers, and equipment. 3. The context of system activity must accompany the chronology. Implications for system designThe decision to install a history list recorder in a user interface -- and what its function(s) will be -- needs to be made early in the system design process. The HL recorder can reside in the application software or operating system. It may also be a separate unit linked to the inputs and outputs of the host system. The cost associated with the incorporation of a HL recorder is linked to the benefits it is designed to provide to its users. Since the use of HLs implies the review and reuse of historical data to influence future system operation, history lists provide an informational feedback path in the design lifecycle for many potential clients (Figure 1). Open-loop real-time systems (in process control, for example) are acknowledged to be simpler and less expensive than closed-loop implementations, but difficult to optimize and less resistant to outside disturbances. The incorporation of HLs provides the same trade-offs: not only do HLs need to be recorded, but also viewed, stored, analyzed, collated and distributed to the system operators. The end result, in terms of enhanced function, must justify the investment in the HL system.
----- FIGURE 1 APPROXIMATELY HERE ----- BACKGROUNDThe literature on history list use is divided into the following two large domains: 1. Command and status feedback to the user of a computer or dynamic system, emphasizing short timeframe interaction information reuse. 2. System usage storage and feedback to design staff for post-hoc analysis of interface design, system operation and malfunction. These two are discussed below, followed by a section on robot-specific HL examples. History lists and computer user interfacesA programmer using a command-line interface (CLI) benefits from HLs for single-command reuse, script writing, single and multiple level undo capability, and data recovery. In CLI systems that use windows to organize context, data and command line reuse is implemented by the selection of text anywhere on the screen using the mouse and its buttons. A `paste' mouse or menu action then copies the selected text to the current command line at the cursor. The size of the screen (or extended screen, if the window can be scrolled down to show older commands) determines how far back reuse can get data. A distinction between command line interfaces (CLIs) and graphic user interfaces (GUIs) is the function of the display screen. In CLIs, the command is a record of a system change request. The request, and potentially any system messages as a result of the change, become part of the history list. For GUIs, the display can reflect only the current state of the system: the display cannot show a record of user interactions graphically as would a scrolling window of past system commands. At best, vestiges and shadows of past GUI actions can be maintained in the current view (e.g., previously selected spreadsheet cells that stay highlighted, reduced-size snapshots of past displays). Aside from the simple UNDO command, there are other approaches to operator-oriented data reuse in GUIs, such as: * to aid in navigation, screen images of previous states are stored and can be shown serially so that a user can choose which one to revisit (e.g., HyperCard's GO RECENT command); * a display window with a command line description of the user actions can serve as a template for reuse (e.g., the EXCEL spreadsheet macro capability). Programmers in CLI and GUI environments have similar needs, but the nature of the interface dictates a different relationship with undo and recover features. Since GUIs show system states, any reversal of history must change not only the data, but also the display. Scripts offer the same value for GUI as CLI operators: they permit a series of operations to be assembled and executed as a set. Whereas CLI scripts are verbatim command lines contained in editable text files, GUI scripts are command-line equivalents to direct manipulation actions, and therefore more difficult for the user to interpret since it is an unfamiliar language. The `undo' function (restoring data to a previous state) with any computer system is of particular interest to interface designers, since operators do make mistakes. Different strategies for robust implementations in computer operating systems are discussed in Linxi & Habermann [1986, p. 38]. However, computer systems that communicate with external processes, such as robot systems, industrial production facilities, and complex dynamic systems will rarely be able to offer this function except at the syntactic level. Semantically-related side-effects, such as the fusing of two metal parts after a welding command has been given to a robot, are impossible to undo. This is similar to other fundamentally irreversible computer-mediated processes such as sending an e- mail message to another computer or printing a document. Predictive and adaptive interfaces depend on HL manipulation. They can offer individualized, context-sensitive environments for software use based on the history of user actions. History lists in process controlComplex dynamic systems are characterized by subsystems that are loosely connected by the flow of mass, energy and data. Each subsystem is influenced by independent disturbance forces and failure causes. The operator (or team of operators) of the entire system controls each subsystem independently, with the goal of achieving an acceptable and safe global system performance. Examples are power plants, paper mills, and satellite data transfer scheduling. History lists have been used in different capacities in such systems. Operator support, management support and accident analysis typify data reuse functions. The complexity of power plant operator consoles has historically followed the evolution of the plants themselves: as plants became more complex, interfaces maintained the one gauge per function (`mirroring') convention that sufficed for simple systems. However, studies in supervisory control [Rasmussen, 1990] and analyses of the causes of accidents such as the one at Three Mile Island (TMI) have shown that present-day power plant complexity has rendered the `mirroring' approach incompatible with human cognitive and information-processing capabilities [NSAC, 1980]. With the recent trend of ever-more disastrous incidents, the interface has recently become a prime target for redesign. It has been demonstrated that the set of system states may need to be displayed, combined, and integrated in a number of different ways to aid the operator in viewing the fault from different perspectives [Rasmussen, 1990], and to aid in the identification and correction of the fault condition. Rasmussen's interface design guidelines imply access to HL data, and confirm their importance for the operation of all complex dynamic systems. Use of history lists with robotsNo commercially-available robot operating systems surveyed in 1984 offered any type of history list support [Gruver, Soroka & Craig, 1984]. A more recent survey of industrial robot systems and safety [Graham, 1991] cites no industrial use (but discusses development work) in history lists. It has been demonstrated that there is potential for history lists to objectify and streamline analysis of robot and computer interfaces [e.g., Van der Loos, 1992; Hammel et al., 1989; Maguire & Sweeney, 1989], to document robot fault situations more objectively than operator interviews [Bonney & Yong, 1985], and to enhance user productivity [e.g., Lee, 1992]. Aside from DeVAR (discussed in Results, below), there are several data collection systems of note in experimental and emerging robotics programs: * The University of Western Australia has developed a sheep shearing robot whose controller was designed with error detection and recovery features [Trevelyan, 1987]. It contains a data collection system to store motion planning, robot position, and end-effector sensor data. When a fault is detected, several seconds of the most recent data are stored on disk for later analysis. The history list system is reported to have been valuable in system debugging, shearing algorithm development, and performance assessment. * The HelpMate hospital fetch-and-carry robot from Transitions Research Corporation has a critical-event data logger that can be reviewed on a daily basis by technical personnel, as well as a plug-in temporary unit to compile several hours of low-level events for use in debugging and system software and hardware development [Engelberger, 1992]. * The Hugh MacMillan Rehabilitation Center (Toronto, Canada) is evaluating a wheelchair-mounted manipulator, MANUS [Kwee et al., 1989]. The arm and gripper are controlled by several buttons to change modes and a joystick to actuate the motors. A camcorder records video and one audio channel during a manipulation task. MANUS itself has no history list system, so a single-board computer has been added to the robot controller to tap into the joystick, button, and display signals [Bishop et al., 1992]. Events are recorded digitally on the other sound track. For analysis, this track is played back, and the events decoded, time-stamped, and stored in a history list file. * At the NASA Jet Propulsion Laboratory (JPL), an operator controls two robots with two force-reflecting six-axis joysticks. The user also employs voice or keyboard-initiated commands for mode changes and discrete interventions. This experimental system contains a data collection subsystem to record forces, torques, and robot joint positions at a 1 msec sampling frequency, and is used primarily to collect performance data during operator control of the robots [Das et al., 1991]. * Sheridan's telerobot systems, as those at JPL, record robot position and torque data samples, and are used in experimental situations to study human control of multi-axis telemanipulation [Sheridan, 1984]. History lists analysis environmentsFor computer development in general, a number of systems have been developed to support user interface testing. The following list, though not exhaustive, illustrates several approaches: * Playback reroutes keyboard and host computer output signals through the data collection system, permitting keystroke-level timings [Neal & Simons, 1983]. Playback has been used to investigate the use of database systems. * HIMS (Human Interface Monitoring System) [Maguire & Sweeney, 1989; Theaker et al., 1989] is an external hardware system that reroutes signals to and from user I/O devices (display, keyboard, mouse, serial port). It stores video and user audio on a videotape and user interaction data in a computer file. The system allows playback and supports analysis. * AutoMonitor is a set of extensions to HyperCard on the Apple Macintosh computer to record a user's navigation through cards of a stack [Kornbrot & Macleod, 1990]. Events (e.g., mouse actions, button presses, field manipulations) are time-stamped and stored in a buffer, which can be copied to disk for later analysis in a spreadsheet program. * VAM (Video Analysis Method) combines into one descriptive grammar the transcript of the videotape with the session history list [Prasse, 1990]. The videotape is important to establish user goals (intentions) and to record the user's comments and gestures, which are then added to the computer history list of actual interactions. * The Hypertext navigation aid, implemented in HyperCard on the Apple Macintosh [Nielsen, 1990], is a software-based set of tools to record and facilitate stack navigation. * MUSiC, a European ESPRIT Project [Bevan, 1992] has begun work on the standardization of usability testing methodology. A data logging facility called DRUM has been developed to integrate video and other data into a single analysis environment. Analysis Techniques of History ListsThe primary goals of HL analysis are testing and modeling. Testing of operators and systems can be done by measuring events such as keystrokes, voice commands, speed, decision choices and error rates; modeling can take the form of, for example, graphical, flow, grammatical or structural representations that reduce data and allow their visualization. Several major analysis techniques are discussed below: Real-time performance analysis methods: In telerobotics, history lists of commands and real-time inputs have been used to replay sequences of user actions and to review the performance of experimental controller designs [Bejczy & Hannaford, 1988; Sheridan, 1984]. In industry, data acquisition systems (analog signal history list recorders) assist designers in system monitoring during the quality control phase of robot construction and in making decisions on acceptability. Pattern analysis methods: Behavior pattern analysis has largely remained a manual process for sociological and user interface field studies [Schatzman & Strauss, 1973; Suchman, 1987; Wright & Monk, 1989]. Some analysis tools for videotape transcription and note organization, such as NoteCards [Halasz et al., 1987], VAM [Prasse, 1990] and CVideo [Knowledge Revolution, 1992] offer computer-aided methods. A method developed by Siochi [1989] to extract Maximal Repeating Patterns (MRPs) offers promise to automate history list analysis. A second computerized pattern analysis method determines locality, or "clustering of user interactions", which may be associated with user activities [Lee, 1992, p. 51]. Performance models: The GOMS (Goals, Operators, Methods, Selection) model, and its precursor, the Keystroke model [Card et al., 1983] are task descriptive and performance predictive methods. GOMS has been used to study expert, error-free operation of complex tasks such as computer-based text editing [Card et al., 1983], telephone operators [John, 1990], and input strategies for computer users [Lee, 1992]. Task and language grammar models: In language analysis, notations such as BNF (Backus-Naur form) can be used to generate a formal representation of grammar rules. The universal applicability of BNF has made it attractive as a base analytic tool. However, it is weak in certain areas most relevant to interface analysis, such as modeling class similarities [Payne & Green, 1986] and including semantic aspects of rule definition [Reisner, 1981]. In response, Payne and Green [1986] present the Task-Action Grammar (TAG) as a formalism based on sets of semantic components and rule-schema manipulations of them. Modeling with state transition networks (STNs): Graphical methods complement language-descriptive methods by showing system states as nodes and the allowable transitions between them through node connections. The topology of the representation evokes the salient aspects of the structure of the interface itself. The value of an STN lies not only in its graphical representation for visual inspection, but also in its relation to a production-rules to render the system available for computational uses. Miller [1985] illustrated the use of transition networks in the study of human control of finite state machines. Based on that work, Mitchell and Miller [1986] developed the Operator Functional Model as a means to analyze the use of a simulated industrial production line in terms of decision-making, error occurrence, and performance. Another, termed the `USE' methodology [Wasserman, 1985], provides a rapid prototyping design environment for user interfaces. A different graphical representation, termed "Statecharts" [Harel, 1986], is based on Venn diagram conventions, and allows complex systems to be represented in a modular and hierarchical display. Kieras and Polson [1983] developed an enhanced STN tool termed the Generalized Transition Network (GTN), which allows representation of hierarchical structures and supports modularity (see Results section for its application to DeVAR). METHODSDescription of DeVAR systemThe Desktop Vocational Assistant Robot (DeVAR) was designed in 1987 in the context of a rehabilitation robotics program co-sponsored by the Rehabilitation R&D Service of the U.S. Department of Veterans Affairs (VA) and the Department of Mechanical Engineering of Stanford University. At the VA Palo Alto Health Care System, DeVAR development work has proceeded primarily at the Rehabilitation R & D Center (RRDC), with clinical evaluation activities coordinated through the adjacent Spinal Cord Injury Center (SCIC). DeVAR is based on two previous desktop robot designs dating back to 1979 [Leifer, 1982]. DeVAR has a voice-controlled, task-level interface designed for use by individuals with high-level spinal-cord injuries. Typical tasks are taking medication, getting a drink from a cooler, inserting floppy disks and laser disks in their drives, retrieving a mouthstick, turning computer fanfold pages, and manipulating printer output. The interface uses an X-10 environmental control unit (ECU) to operate lights, computer, fan, and a call-help alarm. Telephone control (answering a call, dialing a number) is included. The tasks are designed to cover the daily living and vocational needs of an individual in the work setting during a normal workday, with only morning set-up, lunch-time, and evening assistance by an attendant. This project has been described extensively in Van der Loos et al. [1988], Hammel et al. [1989], Van der Loos et al. [1990], Van der Loos, Hammel & Leifer [1991] and Hammel, Van der Loos & Perkash [1992]. Research program retrospectives are contained in companion papers by Hammel [1995] and Van der Loos [1995]. Design of the HL-related softwareThe first version of ROBOT9, the DeVAR interface program, was written in 1987. It was based on eight years of experience with two previous versions, neither of which had a built-in history list system. With these DeVAR precursors, system error diagnosis and recovery was entirely dependent on the presence of an expert at the time of the fault. The use of error log sheets for fault and status reporting by clinicians working with test subjects proved not to be a viable alternative since it was rarely possible to capture enough of the fault context to permit a diagnosis. The decision to include a history list system in ROBOT9 was also influenced by experiences in other domains: (1) Lakin [1988] developed `vmacs', a text-graphic editor that includes a powerful history list system for session playback; and (2) we were familiar with NASA flight simulators and their data logging systems to support pilot modeling and performance analysis. ROBOT9 was written in TurboPascal 5.0 [Lees, 1991]. Of its 7000 lines of source code, approximately 1% is devoted to HL recording. At the start of each user session, a text file is created (sequentially numbered) and a header written to include date, time and user name. Every time an event occurs, whether user initiated or reported by the robot controller, a line of text is appended to the file. Figure 2 contains a file excerpt. To analyze the HL files at the task rather than command level, the program FORHIS was written to parse the event files. Figure 3 shows an extract of the resulting file, ANALYZE.OUT, which is a compilation of all tasks performed over the data recording period. This file was the basis for subsequent analysis.
----- FIGURE 2 APPROXIMATELY HERE ----- ----- FIGURE 3 APPROXIMATELY HERE -----
Test subject and data recording environmentThe data set for this analysis consisted of records from 320 days of real-world usage by one user in a vocational setting. Three typical days were videotaped and transcribed as a corroborative data source. A subset of the same data was manually analyzed as part of a study to assess the effectiveness of DeVAR to replace attendant care in the workplace [Hammel, Van der Loos & Perkash, 1992]. That experience was an additional incentive to automate certain aspects of the HL analysis. DEFINITION OF GUIDELINES FOR HISTORY LIST DESIGNOur guidelines for HL system design are based on five principles. Figure 4 shows the relationship between the five guidelines: although they are vertically ordered according to importance, the interconnections emphasize the non-temporal nature of the HL design process.
----- FIGURE 4 APPROXIMATELY HERE -----
Guideline 1: Identify the Data Client. The recipient of the history list information benefits from using the history list data. The term `client' is used to accentuate the fact that a cost is incurred in terms of resources, time and training. The client can be the software user, the computer itself, the system designer, management or a regulatory agent (See Table 1). The client establishes the function of the history data and drives the design of the history list facility.
----- TABLE 1 APPROXIMATELY HERE -----
Guideline 2: Identify the Data Function. The client formulates the use of the data, and thereby the requisite data reduction, modeling, and analysis functions. The function set drives the data acquisition system and data design (Table 2). Since data use is an interactive, time-evolving process, clients will devise new functions and alternate data reuse strategies after system design has been finalized. A client's initial function definition should therefore be viewed as belonging to a class of functions, with the system supporting alternative functions within that class.
----- TABLE 2 APPROXIMATELY HERE -----
Guideline 3: Define the Data Acquisition System. The software and hardware requirements for data collection depend on client and function definitions. Through the life of the system, client and function may change, and the data acquisition and reporting features may need to be redefined. Design flexibility must be built into the code architecture responsible for the timing and formatting of the data collection. These, in turn, depend on environmental and operational constraints. Guideline 4: Define the Data Structure. The destination and the intended function of the data determine the data structure, its granularity, format, and content categories. These factors are largely independent of the data acquisition system itself. This guideline drives the accessibility of the data to the client. Guideline 5: Determine the Limits of History List Applicability. Since the use of a robot system can be expected to change over time, it is important to define at the outset what the history list system can and cannot be expected to do. At times, it may be necessary to supplement the history lists with data from other sources, such as audio or videotape recorders, process-control machines, or external data acquisition systems. RESULTSThe purpose of this section is to illustrate data reuse and analysis techniques by providing results from the use of DeVAR data, using the five guidelines as a framework for discussion. The goal is not to provide a comprehensive report of DeVAR usage over the 320 days the data cover. More detail is available in Van der Loos [1992]. Guideline 1: Identify the Data Client.Although DeVAR has only one operator, it has several HL users: data records are presented to each in the most appropriate manner. The design and evaluation team is the primary user of the HL system. The operator and the robot controller are also HL users. Operator: A scrolling window of the last 20 commands was part of the user interface screen. These were used for reference, for example when the user could not hear the spoken confirmation of a past command, or when a long period of time had elapsed since the previous command and a reminder was needed. A `demonstration' mode to train new users in DeVAR task capabilities used an artificial history list, produced off-line, to lead the robot automatically through a sequence of commands. This HL was annotated with voice messages and detailed screen information. Robot controller: A `playback' capability allowed the operator to step backwards and forwards through a robot motion program that had been interrupted. While robot tasks were running, each motion step was interpreted and held in a buffer by the controller (the primary "user" and interpreter of the data). After being interrupted, the program could revert back to a previous robot location. This feature was prototyped but never fully implemented. Clinician: The HL data has been used in conjunction with video, questionnaire and other data sources to record usage patterns and user preferences [e.g., Hammel et al., 1989; 1992]. As an example, Figure 5 shows the usage results from the throat lozenge "CANDY" task over the 320 days of data collection. Task views of this nature illustrate user characteristics, long term trends and potential areas of needed training or system improvements.
----- FIGURE 5 APPROXIMATELY HERE -----
Designer: The primary use of HL data has been to analyze the usage, responsiveness and robustness of DeVAR in laboratory and field testing. Table 3 shows the overall use of DeVAR during the test period. Other metrics that the data provided include verbal commands per task, errors per task and trends over time (see Van der Loos [1992] for more detail). The following paragraphs illustrate how the analyses provided additional feedback to the designer.
----- TABLE 3 APPROXIMATELY HERE -----
Guideline 2: Identify the Data Function.This section focuses only on analysis functions for the designer, since for DeVAR these have been dominant over the past years. The Generalized Transition Net (GTN) model will be used to illustrate debugging and design related functions. The GTNs for the highest two levels of the ROBOT9 program (Figure 6) illustrate the designed grammar of overall DeVAR operation. Figure 7 shows one of these with superimposed usage metrics. This is also an example of the use of the GTN to debug a complex user interface. Even though ROBOT9 was flow-charted, the evolution of the program over six years caused aberrant behavior not readily inferable from the source code. The ability to inspect the true behavior of the system proved to be a powerful means of interface debugging.
----- FIGURE 6 APPROXIMATELY HERE ----- ----- FIGURE 7 APPROXIMATELY HERE -----
Guideline 3: Define the Data Acquisition System.Figure 8 shows the potential taps for history list data. The data users and functions largely determine the HL recording system structure. The interface for DeVAR incorporates an event-based (not time-based), internal (as opposed to external processor), software-only (as opposed to a dedicated storage device), cumulative (as opposed to a fixed-length circular buffer) HL recording system. This configuration results in a HL system that is unobtrusive to the user and the analyst and has negligible system overhead.
----- FIGURE 8 APPROXIMATELY HERE -----
Due to the multi-input/output nature of DeVAR (voice unit, keyboard, serial ports, robot), history list events are not saved to disk from one central procedure, but rather from at least six locations in ROBOT9. While this approach had the advantage of reducing overhead (events were handled by the procedures that created them), it had the disadvantage of being difficult to maintain through the design lifecycle. Guideline 4: Define the Data Structure.The design of HLs must follow from the uses they will see. As seen in Figure 2, the DeVAR HLs were originally designed for visual inspection and analysis. Subsequently, several fields were added and event types created to make computational analysis easier and automation feasible. To make the use of DeVAR as simple as possible, mode switches were minimized (e.g., ASLEEP vs. AWAKE) and the vocabulary was kept as small as possible. A by-product of this approach was that certain commands, like UP and DOWN, could be used in a task as well as outside a task. In other words, when analyzing a HL, the entire command chronology was needed to know whether the user was in a task or not, and the parser FORHIS was needed to recreate system state information during analysis. The problem could have been solved easily by recording more system states than the current three. It also illustrates the trade-off to be made in the design process between storage requirements and investment in analysis tools. In practice, the less structured the interface, the more state information is required to understand the context of operation. Guideline 5: Determine the Limits of HL Applicability.User intention is difficult to determine through observation since there are no representations of intention in the interface. Users deal with objects and actions. Interpretation of intention is made easier when the software objects (i.e., graphic or language-based representations) and actions are close to the real artifacts of a user's task. In other words, a text editor that contains structures for footnotes, tables of contents, and page numbering serves document preparation better than a pure character-oriented editor. Similarly, a robot interface can be organized around tasks (e.g., PICK_AND_PLACE) or motion-specific commands that may accomplish that task (e.g., the sequence: DOWN, GRIP, UP, LEFT, DOWN, RELEASE, UP). The former conveys user intention more readily. More direct user-oriented measurement, such as protocol analysis, interviews and videotape, can complement the event-oriented history lists. The client must determine whether the variables recorded in the history list cover the functional needs, and to what extent each variable or sensor has sufficient temporal and spatial resolution. Strategies the client can use to identify solutions to possible mismatches between needs and system capabilities are, in increasing order of complexity and cost: 1. Defining alternate variables that are correlated with the real quantity of interest. 2. Defining additions to the existing history list system, in terms of hardware and software. 3. Defining parallel recording systems to supplement the history list collector. 4. Changing the analysis if it is decided that data cannot be reliably collected. DeVAR history lists have not been completely adequate in the following areas: 1. Recording of non-robot tasks and actions at the workstation. 2. Finding causes of anomalous user actions in normal robot task execution. 3. Helping with voice recognition problems. 4. Uncovering certain robot operation problems. A DeVAR example covering all of the above points is illustrated and explained in Figure 9. It also shows how we have used video to understand context and intention. Videotape is the most common medium to record human behavior and situated actions [e.g., Tang, 1989; Minneman, 1991]. Analysis of videotape is time-consuming because human behavior is complex, and videotape recording provides no data reduction. The transcript of a videotape is an event HL whose contents are defined by the analyst, not by the designer of the system under observation. The comparison of transcribed events with history list recorded events provides designers with a mechanism to view the task under observation from the perspective of the user and the actions of the system, and to interpret user intentions from the observed actions.
----- FIGURE 9 APPROXIMATELY HERE -----
DISCUSSIONDeVAR HL clients and functions have increased in number as DeVAR has moved from the R&D phase to the product evaluation and marketing phase. From 1991-1994, the VA Technology Transfer Section evaluated DeVAR as a prescribable medical device for high-level quadriplegic veterans with a diagnosed need for assistance [Cupo & Sheredos, 1994]. The HLs were used as an important tool to debug installation and usage problems. With further enhancements to the HLs and its analysis tools, the DeVAR data can be used by a number of new clients: * the VA itself as a regulatory agent; * medical staff to record system activity and justify prescription; * medical staff to aid in training; * the manufacturer to chronicle use and plan maintenance; * the manufacturer and VA to review breakdowns and accidents; and * the user to customize the system based on usage preferences. The examples given in the Results Section, as well as other observations from the overall DeVAR HL analysis effort, establish the basis for the design of HL-software in the next development cycle of workstation robots. Advances in computer-controlled video manipulation software and sequential data analysis concepts are expected to have a large impact on the feasibility and theory of using historical data to enhance interfaces and understand human interaction with technology. CONCLUSIONHistory list design to date has been largely ad hoc. Still, with sufficient design lifecycle feedback, some good systems have been developed. With a framework and methodology for history list design, history list usage has the potential to become more routine and better structured. Table 4 shows the expected effect of HLs on robot system design parameters.
----- TABLE 4 APPROXIMATELY HERE -----
History lists have proved to be valuable in systems such as nuclear power plants and commercial aircraft, for which the social cost of system failure is high. It is especially important to instrument such systems due to the low frequency of accidents and the potentially devastating effect of each accident. Since computer control is becoming increasingly commonplace for electromechanical equipment and appliances, the implementation cost of history list systems continues to decrease. For equipment whose failure can incur high costs -- robots performing surgery, automated hospital diagnostic equipment, vehicle systems whose software performs decision-making and can control overall vehicle behavior -- it is essential that an objective data gathering system exists to support the design process. This becomes even more important as the complexity, speed, and size of systems render their operation inaccessible to humans. When history list design is grounded in method, the risk of their inadequacy is reduced and the usefulness of their output is enhanced. REFERENCESBejczy AK, Hannaford B. Man-machine interaction in space telerobotics. Proceedings International Symposium on Telerobotics and Control, July, 1988, 135-149. Bevan N. The MUSiC methodology for usability measurement. Proceedings (Posters and Short Talk), ACM Conference on Human Factors in Computing Systems, CHI'92, Monterey, CA, May 3-7, 1992, 123-124. Bishop JW, Verburg G, Milner M, Mifsud M. A command monitoring system for the MANUS robotic arm. Proceedings RESNA `92, June 6-11, 1992, Toronto, Canada, 328-330. Bonney MC, Yong YF. (eds). Robot Safety. Springer-Verlag, Berlin, 1985. Brooks JL, Wallace MG. Logistics support planning for standardized avionics. Proc., IEEE National Aerospace and Electronics Conf.NAECON, Dayton, OH, May 22-26, 1989, 1929-1936. Card SK, Moran TP, Newell A. The Psychology of Human-Computer Interaction. Laurence Erlbaum Inc., Hillsdale, NJ, 1983. Cupo ME, Sheredos S, Clinical evaluation of the Desktop Vocational Assistant Robot (DeVAR), Technology Transfer Section Report, U.S. Department of Veterans Affairs, Sept., 1994. Das H, Zak H, Kim WS, Bejczy AK, Schenker PS. Performance experiments with alternative advanced teleoperator control modes for a simulated solar maximum satellite repair. Proc. Fifth Annual Space Operations, Applications, and Research Symp., July 9-11, 1991, Houston, TX. Deming WE. Quality, Productivity and Competitive Position. MIT Center for Advanced Study, Cambridge, MA, 1982. Engelberger J. Personal Communication, 1992. Graham JE. (ed). Safety, Reliability, and Human Factors in Robotic Systems. Van Nostrand Reinhold, New York, 1991. Gruver WA, Soroka BI, Craig J. Industrial robot programming languages. IEEE Transactions on Systems, Man, and Cybernetics, 14, 4, 1984, 565-570. Halasz FG, Moran TP, Trigg RH. NoteCards in a nutshell. Proceedings Human Factors in Computing Systems CHI'87 Conference. Toronto, Canada, April 5-9, 1987, 4552. Hammel J, Hall K, Van der Loos HFM, Leifer LJ, Perkash I. Clinical evaluation of a desk-top robotic assistant. Journal of Rehabilitation R&D, Vol. 26, No. 3, 1989, 1-16. Hammel J, Van der Loos HFM, Perkash I. Evaluation of a vocational robot with a quadriplegic employee. Archives of Physical Medicine and Rehabilitation, 73, July, 1992, 683-693. Hammel J. The role of assessment and evaluation in rehabilitation robotics research and development: Moving from concept to clinic to context. IEEE Trans. Rehabilitation Engineering, Vol. 3, No. 1, March, 1995, 56-61. Harel D. Statecharts: A Visual Approach to Complex Systems. Weizman Institute of Science Technical Report CS86-02, Rehovot, Israel, 1986. Hartson HR, Hix D. Human-computer interface development: concepts and systems for its management. ACM Computing Surveys 21(1), March, 1989, 5-92. John BE. Extensions of GOMS analyses to expert performance requiring perception of dynamic visual and auditory information. Proceedings Human Factors in Computing Systems CHI'90 Conference. Seattle, WA, April 1-5, 1990, 107-115. Kieras D, Poulson PG. An Approach to the formal analysis of user complexity. International Journal of Man-Machine Studies, 22, 1985, 365-394. Knowledge Revolution, Inc., User Manual of CVideo, San Francisco, 1992. Kornbrot D, Macleod M. Monitoring and analysis of hypermedia navigation. Proceedings, INTERACT'90 Conference, D Diaper, D Gilmore, G. Cockton, B Shackel, eds. North-Holland Publishers, Amsterdam, NL, 1990, 401-406. Kwee HH, Duimel JJ, Smits JJ, Yuinhof de Moed AA, van Woerden JA, van de Kolk LW, Rosier JC. The MANUS wheelchair-borne manipulator: system review and first results. Proc., 2nd Workshop on Medical and Health Care Robotics. Newcastle upon Tyne, UK, 1989. Lakin F. A performing medium for working group graphics. Computer-Supported Cooperative Work: A Book of Readings, I Greif, ed. Morgan Kaufman, San Mateo, CA, 1988. Lee A. Investigations into History Tools for User Support. Doctoral Thesis, Computer Science Department. University of Toronto, Canada, 1992. Lees DS. User Interface Software for a Desktop Vocational Robotic Assistant. Invention Disclosure filed with the VA General Counsel. Rehab. R&D Center, VA, Palo Alto, CA, 1991. Leifer, L. Restoration of motor function - a robotics approach, Uses of Computers in Aiding the Disabled, North-Holland Co., 1982, 3-18. Linxi C, Habermann AN. A History Mechanism and Undo/Redo/Reuse Support in ALOE. Carnegie Mellon Univ., Dept. of Computer Science Technical Report. CMU-CS-86-148, 1986. Maguire M, Sweeney M. System monitoring: garbage generator or basis for comprehensive evaluation system? People and Computers V, A Sutcliffe, L Macaulay, eds. Cambridge University Press, Cambridge UK, 1989, 375-394. Miller RA. A systems approach to modeling discrete control performance. Advances in Man-Machine Systems Research, Vol. 2, WB Rouse, ed. JAI Press, Greenwich, CT, 1985, 177248. Minneman SL. The Social Construction of a Technical Reality: Empirical Studies of Group Engineering Design Practice. Ph.D. Dissertation. Dept. of Mechanical Engineering, Stanford University, 1991. Mitchell CM, Miller RA. A discrete control model of operator function: a methodology for display design. IEEE Trans. Systems, Man, Cybernetics. Vol. SMC-16, May/June, 1986, 353-369. Neal AS, Simons RM. Playback: A method for evaluating the usability of software and its documentation. Proceedings, Human Factors in Computing Systems CHI'83 Conference. Boston, MA, December 12-15, 1983, 78-82. Nielsen J. The art of navigating through hypertext. Communications of the ACM 33(3), March, 1990, 296-310. NSAC: Nuclear Safety Analysis Center. Analysis of Three Mile Island - Unit 2 Accident. NSAC-80-1. Electric Power Research Institute, Palo Alto, CA, 1980. Payne S, Green TRG. Task-Action Grammars: A Model of the Mental Representation of Task Languages. Human-Computer Interaction 2, 1986, 93-133. Prasse MJ. The Video Analysis Method: An integrated approach to usability assessment. Proceedings of the Human Factors Society 34th Annual Meeting, 1990, 400-404. Rasmussen J. Mental models and the control of action in complex environments. Mental Models and Human Computer Interaction I, D Ackermann and MJ Tauber, eds. North-Holland Press, Amsterdam, 1990, 41-69. Reisner P. Formal Grammar and Human Factors Design of an Interactive Graphics System. IEEE Transactions Software Engineering, SE-7(2), 1981, 229-240. Schatzman L, Strauss AL. Field Research: Strategies for a Natural Sociology. Prentice-Hall, Englewood Cliffs, NJ, 1973. Sheridan TR. Supervisory control of remote manipulators, vehicles, and dynamic processes: experiments in command and display aiding. Advances in Man-Machine Systems Research, Vol. I, WB Rouse, ed. JAI Press, London, GB, 1984, 49-137. Siochi AC, Hartson HR. Task-oriented representation of asynchronous user interfaces. Proc. Human Factors in Computing Systems CHI'89 . Austin, TX, 1989, 183-188. Suchman LA. Plans and Situated Actions: The Problem of Human/Machine Communication. Cambridge University Press, Cambridge UK, 1987. Tang JC. Listing, Drawing, and Gesturing in Design: A Study of the Use of Shared Workspaces by Design Teams. Ph.D. Thesis. Stanford University Dept. of Mechanical Engineering, 1989. Theaker CJ, Phillips R, Frost TME, Love WR. HIMS: A tool for HCI evaluations. People and Computers V, A Sutcliffe, L Macaulay, eds. Cambridge University Press, Cambridge UK, 1989, 427-442. TimeLog Personal Project Management Software for the Macintosh. User Manual. Coral Research, Inc., 1991. Trevelyan JP, Nelson M. Adaptive robot control incorporating automatic error recovery. Proc. Third Intl. Conf. on Advanced Robotics ICAR'87, October, Versailles, France, 1987, 385-398. Van der Loos HFM, Michalowski SJ, Hammel J, Leifer LJ, Stassen HG. Assessing the impact of robotic devices for the severely physically disabled. Proc. First Intl.. Workshop on Robotic Applications in Medical and Health Care, Ottawa, Canada, June 23-24, 1988, 23-25. Van der Loos HFM, Hammel J, Lees DS, Chang D, Perkash I, Field evaluation of a robot workstation for quadriplegic office workers. European Review of Biomedical Technology, Vol. 5, No. 12, 1990, 317-319. Van der Loos HFM, Hammel J, Leifer LJ, Task-specific assessment of robot effectiveness: enhancing the independence of quadriplegics in the workforce. Proc. Fifth International Conference on Advanced Robotics, Pisa, Italy, June 19-22, 1991, 31-36. Van der Loos HFM. A History List Design Methodology for Interactive Robots. Ph.D. Thesis, Department of Mechanical Engineering, Stanford University, CA, 1992. Van der Loos HFM, VA/Stanford Rehabilitation Robotics Research and Development Program: Lessons Learned in the Application of Robotics Technology to the Field of Rehabilitation. IEEE Trans. Rehabilitation Engineering, Vol. 3, No. 1, March, 1995, 46-55. Wasserman AI. Extending state transition diagrams for the specification of human-computer interaction. IEEE Transactions on Software Engineering, Vol SE-11, No. 8, August 1985. Wright PC, Monk AF. Evaluation for design. People and Computers V, A Sutcliffe, L Macaulay, eds. Cambridge University Press, Cambridge UK, 1989, 345-358.
Figure 1. Design Lifecycle Diagram with History List Feedback The diagram shows data paths for design information flow. In particular, the history list system and its feedback paths illustrate the multiple potential recipients of the data and the numerous functions that maybe appropriate to each. The role of videotape recording is superimposed on the diagram. Note that videotape can directly access operator activity, while the history list acquisition system must rely on the operator interface for operator-based data.
** DeVAR Automatic History List Collector 4/3/91 6:32:29 ** User: boby.tem
NO. TIME TIME [sec] COMMAND SPEED MODE MOTION 95 07:02:37 25358 WAKE_UP_NOW 10 World Mode Step 96 07:02:43 25364 ATTENTION 10 World Mode Step 97 07:02:48 25368 PAGE_RIGHT 10 World Mode Step 98 07:02:50 25371 YES 10 World Mode Step 99 07:03:15 25395 (PAUSED) 100 07:03:18 25399 CLEAN 10 World Mode Step 101 07:03:21 25401 YES 10 World Mode Step 102 07:03:38 25418 Task Completed. 103 07:03:38 25419 Program completed 104 07:03:40 25421 PAGE_RIGHT 10 World Mode Step 105 07:03:43 25424 YES 10 World Mode Step 106 07:03:57 25437 (PAUSED) 107 07:03:58 25438 RIGHT 10 World Mode Step 108 07:04:00 25441 PROCEED 10 World Mode Step 109 07:04:52 25493 PLEASE_WAIT. 10 World Mode Step 110 07:04:54 25494 PLEASE_WAIT. 10 World Mode Step 111 07:05:14 25515 Task Completed. 112 07:05:15 25515 Program completed 113 07:05:19 25519 RELAX 10 World Mode Step 114 07:05:21 25522 YES 10 World Mode Step
Figure 2. History List Extract PGE0718.HIS The DeVAR files contain the following fields and field sets: 1. line identifier. Consecutive numbering of all events. 2. time of day in hours:minutes:seconds. 3. time of day in seconds from midnight, to calculate time differences manually. 4. event identifier. This name corresponds to the word appearing on the display screen. 5. Event parameters: SPEED, MODE, MOTION refer to the overall command speed setting, the piloting mode invoked (WORLD, PILOT OR JOINT), and the type of motion, either CONTINUOUS or discrete STEP. Certain event types (e.g., lines 99, 102), have no parameters.
NAME NUM DATE DW TIME T N S E SP CMDS WAKE_UP_NOW 44 33331 3 0.2888 0 0 0 0 10 ATTENTION 45 33331 3 0.2889 0 0 0 0 10 PAGE_RIGHT 4 33331 3 0.2890 50 2 0 0 10 YES (PAU PROC PAGE_LEFT 3 33331 3 0.2896 53 3 0 0 10 YES (PAU CLEA YES PAGE_LEFT 3 33331 3 0.2902 99 3 0 0 10 YES (PAU RIGH PROC PLEA RELAX 26 33331 3 0.2914 3 1 0 0 10 YES WAKE_UP_NOW 44 33331 3 0.2935 0 0 0 0 10 ATTENTION 45 33331 3 0.2936 0 0 0 0 10 PAGE_RIGHT 4 33331 3 0.2936 50 3 0 0 10 YES (PAU CLEA YES PAGE_RIGHT 4 33331 3 0.2942 94 3 0 0 10 YES (PAU RIGH PROC PLEA PLEA RELAX 26 33331 3 0.2954 3 1 0 0 10 YES WAKE_UP_NOW 44 33331 3 0.2997 0 0 0 0 10 ATTENTION 45 33331 3 0.2997 0 0 0 0 10 LAMP 22 33331 3 0.2997 2 1 0 0 10 YES RELAX 26 33331 3 0.2998 2 1 0 0 10 YES WAKE_UP_NOW 44 33331 3 0.3043 0 0 0 0 10 ATTENTION 45 33331 3 0.3044 0 0 0 0 10 NEW_STACK 8 33331 3 0.3045 83 12 0 0 10 YES (PAU CONT SLOW YES SLOW NO 43 33331 3 0.3058 83 1 0 0 10 RELA NO 43 33331 3 0.3059 83 1 0 0 10 BOWL RELAX 26 33331 3 0.3062 3 1 0 0 10 YES
Figure 3. Task-Level Representation from History List Sample
Fields in the cumulative task file ANALYZE.OUT:
Records in fields 1. NAME Task Name 2. NUM Code for task (1 - 53) 3. DATE Date (1=Jan 1, 1900) 4. DOW Day of Week (1=Monday) 5. TIME Time (proportion of 24 hour day: 0=midnight, 0.5=noon, ...) 6. T Duration (seconds) 7. N Number of commands in task 8. S Number of STOP commands in task 9. E Number of errors in task 10. SP Speed setting 11. CMDS List of in-task commands and messages in abbreviated form
Figure 4. Relationships between the Five Design Guidelines Guidelines (1) and (2) drive the specifications of (3) and (4), which are constrained by (5). Constraints can cause the modification of the definitions of the data user (client) as well as the data functions. The double-headed arrows accentuate the non-linear nature of design: any number of pathways may be traversed in parallel toward a final history list specification.
Figure 5. Usage Graphs for the CANDY Task These five graphs show task counts on the ordinate. The abscissae are: (1) day-of-week: dow: 1=Monday; (2) time of day in fraction of 24 hours: 0.3=7:12AM; 0.75 = 6:00PM; (3) task duration in seconds; (4) number of commands (cmds) per task; and (4) daily trend from 1/14/91 to 8/1/91 (date index=33252 to 33451). The time-based abscissae are given in ordinal rather than conventional form to facilitate data manipulation. The user typically works four 10-hour days per week, accounting for the low task counts on Fridays. The 70% increase from beginning to end of week in task frequency is not correlated with other variables or total number of commands. Mean task duration is 96 seconds; the three outliers between 200 and 475 seconds are from tasks paused for unknown reasons.
Figure 6. Top-Level Generalized Transition Networks of the DeVAR User Interface The top graph shows only the three super-states DOS, ASLEEP, and AWAKE, and the allowable transitions between them. Words above a line indicate user actions; words under a line indicate associated system responses. Node names in parentheses indicate they are expanded at a lower level (as a subroutine). The "POP" node indicates a return of the graph to the next higher level. The user operates in one of these three super-states. The nodes ASLEEP and AWAKE are expanded in the middle and lower graphs. The ASLEEP state exists principally to prevent speech that is not directed at the robot to be inadvertently recognized as commands. To transition to the AWAKE state, in which tasks are invoked, two commands, READY and ATTENTION, must be given. However, the ASLEEP state does have several operational aspects. Notably, an incoming phone call can be answered, and dial-out can be invoked. These features are only visible at the next lower level (not shown), since they are not specific to the ASLEEP state.
Figure 7. Top-Level GTN with Usage Data This graph shows robot usage data superimposed on the Generalized Transition Network for DeVAR: only the top-most level of interaction is depicted. Line widths scale as the log of the transition counts. These counts were derived from post-processing of the HL files. In this case, the transition counts uncovered a bug: the 8 ASLEEP to DOS transitions are anomalous, stemming from an error in the DeVAR interface that caused a program halt from spurious microphone input during start-up. The low frequency of this occurrence caused the bug to escape notice until the history list analysis.
Figure 8. Sources of Data Gathering Each box (except the observer) defines a data transmission node in the flow of information between a user and the application program of the computer. Each box defines a potential source of data for history list recording. The list on the right indicates the forms that the raw data can assume if captured at the corresponding node.
** DeVAR Automatic History List Collector 4/3/91 6:32:29 ** User: boby.tem
NO. TIME TIME[sec] COMMAND COMMENTS
06:47 Attendant takes spoons 06:48 User prints page on printer
27 06:48:35 24515 WAKE_UP_NOW 28 06:48:41 24521 ATTENTION 29 06:48:45 24525 PAGE_LEFT 30 06:48:48 24528 YES
06:48:50 Attendant returns clean spoons and gets in path of robot, so user aborts program:
31 06:48:58 24539 *Panic button pressed* 32 06:48:58 24539 *** VAL program stopped by user *** 33 06:48:58 24539 Stopped at slot.pick, step 7
06:49:30 Attendant finishes job, leaves the area. User wants to resume task: 06:49:30 "Proceed" 3 times (no response) "Ready" 8 times (no response)
34 06:49:58 24599 DISCRETE (misrec. to "Listen to me")
User now realizes that robot is awake and in MAIN mode
35 06:50:04 24604 WORLD (misrec. to "Microphone off") 36 06:50:10 24610 PAGE_LEFT (Says task name to restart) 37 06:50:12 24613 YES 38 06:50:27 24628 (PAUSED) 39 06:50:29 24629 RIGHT 40 06:50:31 24631 PROCEED 41 06:51:48 24709 Task Completed. 42 06:51:49 24709 Program completed
Figure 9. Annotated HL: Unclear User Actions during Task Execution This excerpt shows part of the morning DeVAR set-up. Annotations are from the videotaped record of the entire day. The user had just made a print-out, and had given the PAGE_LEFT command (line 29). The program started as the attendant, unaware of the user's actions, reached over the printer to place spoons with mints and medication in their holders. The attendant had placed himself in the motion path of the robot, so the user stopped it with the panic switch (line 31). The reason for this action was entirely inaccessible by the HL. After the attendant finished his task, the user tried to reactivate the job by saying, "PROCEED" at 6:49:30. There was no response, as if the voice recognition system was malfunctioning (and no HL event). The user then guessed that the robot was asleep (even though he had not said "RELAX"), and tried to wake it up by saying "READY". A third approach was to use his own computer to bypass the robot's voice recognition system and send a command via the serial line. To wake up his own computer's voice recognition system, he said, "Listen to me." The robot's voice recognition unit misrecognized this as the command DISCRETE (line 34). Serendipitously, the user realized that the robot was in its main mode, not the initially presumed in-task mode or asleep mode. The robot had never engaged the PAGE_LEFT task in the first place, even though it had started its motion. The user then restarts the task (line 36), which completes normally (step 42). The user did not understand how it was possible that a task that was confirmed and had already started could have been aborted by just saying STOP. The error in the user's model of task execution stems from an obscure `feature' in ROBOT9 that we had apparently neglected to teach this user, and which the display shows only discreetly in its mode box. There is a two to ten second grace period between task confirmation ("YES") and the start of the in-task mode while the robot is moving towards but has not yet engaged with the task's objects and equipment. As a result of this observation, the display was altered to provide a clear message during the initial task phase: "To cancel task, say STOP now."
Client Value Examples and References
User, Operator command reuse [Linxi & Habermann, 1986] [Lee,
scripting navigation, 1992] HyperCard [TimeLog, 1991]
re-register project [Brooks & Wallace, 1989]
management flight
simulation
Process user modeling see text see text USE
Controller predictive displays [Wasserman, 1985] see text
simulation and replay [Deming, 1982]
adaptive interfaces
fault diagnosis
maintenance scheduling
Designer interface design see text DeVAR DeVAR
application design
command set definition
feature use
Management inventory management [Deming, 1982] DeVAR [Deming,
user training employee 1982]
tracking quality
control
Regulation accident reporting [NSAC, 1980] [Deming, 1982]
maintenance upgrade [Linxi & Habermann, 1986]
schedule
Table 1 History List Clients, Potential Uses, and Applications
User Role of Reuse Data Presentation Timing Single User, scripts,
|
