14 August 1967

Quarterly Technical Letter Report 5

Covering the Period 9 May through 8 August 1967

Stanford Research Institute Project 5890


STUDY FOR THE DEVELOPMENT

OF

HUMAN INTELLECT AUGMENTATION TECHNIQUES

 

by


D. C. Engelbart

Contract NAS1-5904

Prepared for


National Aeronautics and Space Administration

Langley Research Center, Langley Station

Langley, Virginia 23365, Mail Stop 126


Copy No. 19
ARNAS, QPR5 May-August 1967

1 General
1a Through this quarter we continued a very heavy concentration on the implementation of our 940 multiconsole system, carrying it now on the ARPA/RADC project.
1a1 Appendix A includes the last monthly letter report on that project to indicate status of this implementation activity.
1b Associated with this has been the establishment of a modest activity under our SOG project.
1b1 The activity under this project will be to study the dynamics of the time-sharing system in pursuit of better support for the on-line service.
1b2 Appendix B includes a copy of the recent interim report to that project to indicate its activity.
1c The activity under the ARNAS project amounted to about 25 percent of its average, straight-line expenditure rate.
2 ARNAS Activity During The Quarter
2a Program Planning
2a1 A significant amount of Engelbart's time has been spent in exploring the candidate activities, and the possible organization and new-activity timing for the course of our research work after the facility becomes operational. Appendices C and D outline (as prepared for inclusion in NASA and RADC proposals) a reasonable summary of this thinking and planning.
2b Mr. Elmer Shapiro (of the Systems Engineering Laboratory, SRI), who has a strong hackground in both computer and communication systems problems, has been helping us with some analyses towards the communications system for the proposed ARPA computer network.
2b1 In this regard, he attended a meeting in Washington (May 18) of the interested ARPA contractors to discuss initial plans for the network.
2b2 Appendices F and F are recent memos by Shapiro relative to this problem, to represent the nature of his activities.
2c Mr. Elton Hay, a regular member of th AHI program, is lined up to do some initial information gathering relative to our forthcoming

1


ARNAS, QPR5 May-August 1967
role as a Network Information Center (NIC) for the ARPA computer network.
2c1 We are putting our first priority on the 940 system implementation, and as yet have taken no significant steps on the NIC work.
2d Dr. Charles Dawson, for the past year and a half a part-time participant in the AHI program, spent some time bringing our X-DOC system into a relatively usable state.
2d1 Over the seven-year period that Flexowriter, Teletype, and Dura paper tapes have been accumulated of the some 2800 citations, there have been several shifts in format and accumulation of a number of errors and ambiguities.
2d2 He wrote a FORTRAN program for the 3100 with which these paper tapes could be loaded onto the disk, which transformed the records into a newly established standard format and which searched out and corrected (or left special tags on) the majority of errors and ambiguities.
2d3 He also wrote a program which would allow us to produce a KWIC index, and an author index for these files.
2d4 It is expected that further cleaning up and indexing on the X-DOC system will be done at the CRT consoles of our 940 system after our Bryant disk becomes operational late next fall.
2e We have initiated a trial venture towards producing a sound track movie capable of communicating some of the principles and developments of our program to a lay audience, without the requirement of an experienced narrator.
2e1 We have budgeted the sum of $5,000 to produce this movie by the first of December.
2e2 We have set up a special team of three people to help us with this task.
2e2a Mr. Jeffrey Nipomnick is a graduate student in Communications at .Stanford University, specializing in movie making, with the school requirement of producing original movies during the forthcoming academic year. He will be retained by us for this period at a rate equivalent to a research assistantship.
2e2b Mr. William Bowman is currently an assistant professor of Art at Stanford University, and a part-time consultant in

2


ARNAS, QPR5 May-August 1967
industrial graphics, with many years of experience in the fields of industrial graphic illustration. His manuscript for a book (Graphic Communication) dealing with the language of graphic symbolism is being processed for publication by John Wiley and Sons. He will be remunerated via an hourly consultation arrangement.
2e2c Mr. David Casseres graduated in English Literature from Reed College and has been employed as a technical editor at SRI--having participated in the editing of most of the reports and proposals of our program during the last eight months.
2e3 Meetings are being held, mainly with English and Engelbart, for both discussion and development of the movie plans, and for the organized indoctrination of this trio in basic principles and techniques of computers, of our computer aids, and of our program developments and objectives.
2e4 The current plans for the movie are to include extensive use of simple animation techniques to communicate principles, techniques, etc. which do not lend themselves either to simple verbal statements or to the real-life filming of our actual CRT actions.
2f Visitors
2f1 Dr. Dan Peck and Francis Moakley, San Francisco State College
2f2 J. Broadus, D. Moore, and T. Kirkpatrick, IBM
2f3 Dr. Donald Lindberg and Larry Rowland, University of Missouri
2f4 Robert Stotz and Donald Haring, MIT
2f5 Robert Harshbarger, Capt. Jack Weatherford, Benjamin Erdman, and Lt. Col. Jack Panabaker, Defense Communications Agency
2f6 M. Maroney, NASA
2f7 Robert Russell and Wayne Wilner, Stanford IJniversity
2f8 William Kent, IBM
2f9 Ray de Saussure, Lawrence Radiation Laboratory
2flO William Mulroney, ARPA
2f11 W. Massena, D. Horning, and L. Sancher, Control Data Corporation

3


ARNAS, QPR5 May-August 1967
2f12 S. McIntosh and D. Griffel, MIT
2f13 Dr. Larry Roberts and Arthur Bushkin, ARPA
2f14 Ronald Resch, University of Illinois
2f15 Dr. David Goldberg, U.S. Office of Education
2f16 L. S. Coles, Carnegie Institute of Technology
2f17 C. Askanos, J. Bryson, D. Masters, and V. Grinich, Fairchild Camera and Instrument Corp.
2f18 C. S. Carr, S. Macdonald, and M. Smith, University of Utah
2f19 Paul Howerton, EBS Management Consultants, Inc.
2f20 Dr. F. Gilbert and Dr. R. Gilbert, University of Hawaii
2f21 J. Faber, G. Sell, N. Stahl, and C. Argast, Union Carbide
2f22  J. Eckels, W. Hillblom, A. Rizzi, and E. Klein, IBM
2f23 J. Pond and Major Stevens, National Military Command System Support Center
2f24 Dr. Rosen, IJniversity of Wisconsin
2f25 Group of 13 from University of California, Berkeley, School of Librarianship: K. Cartwright, D. Ferguson, A. De Fiuga, J. Pendleton, H. Zars, B. Rice, C. Kaspesek-Olest, G. Falk, D. Green, N. Moore, A. Humphrey, J. Meredith, and E. Elliott
3 Plans For Next Ouarter
3a We should have our Stage-I 940 multiconsole system running and we expect most of our energies to be spent in shaking it down, integrating the line printer, making ready for the Bryant disk, transferring our working files, developing and cleaning up documentation associated with the implementation, etc.
3b We expect to get started on initial data gathering for the Network Information Center.
3c The new movie should be designed and well into its production by the end of the quarter.
3d Depending upon the way the 940 facility shapes up, we will begin

4


ARNAS, QPR5 May-August 1967
by the end of the quarter to develop definite plans for the particular activities (See Appendix D) upon which we will concentrate our energies in the further stages of this project.
Submitted by:
[personal signature]
D. C. Engelbart, Project Leader
Approved by:
[personal signature]
Torben Meisling, Manager
Systems Engineering Laboratory

5


APPENDIX A: COPY OF RECENT RADC FACILITY-DEVELOPMENT REPORT
1 Management Research
1a We are continuing the modified Gantt-chart approach to task scheduling and monitoring.
1a1 While this has been an effective tool, it has severe limitations for the kind of work that we are involved with. We feel that computer driven displays will greatly increase this effectiveness.
1b We have looked into using the teletype console (with QED) for maintaining information on purchases and equipment status, but it appears that this approach may not be very efficient because of the use of tape files with our system and the long time required to access them.
2 New-Facility Hardware
2a 940 Computer
2a1 The 30-day acceptance period for the computer began on 5 July as planned.
2a1a Since then we have had a total of 13 hours of down-time (20 hours of down-time is cause to slip the acceptance period).
2a2 All three tape drives have been delivered and are operating.
2a3 The MIC is installed and operating as expected.
2a3a The MIC must still be modified for true "dynamic priority," but by agreement with SDS this will be done after the acceptance period.
2a4 Listings and symbolic files for the required software systems have been delivered but not all have been checked out as yet.
2a5 The one remaining problem with the system is that a temporary wiring modification is still in that makes all four memory banks look like a single bank. This prevents simultaneous access to core by the CPU and the Z buss (DACC and MIC). Removing this modification causes the machine to fail.
2a5a SDS is working on the problem and if it is not solved before the end of the acceptance period (4 August) the acceptance period will slip.

1


APPENDIX A: COPY OF RECENT RADC FACILITY-DEVF.LOPMENT REPORT
2b Work-Station I/O System
2b1 We have been informed by Tasker that they will slip three weeks (to 8 September) on delivery of the first display generator system.
2b2 The entire television system has been received and will be checked out with an installation engineer from General Electric in the next few weeks.
2b3 Interface and Controllers
2b3a The executive hardware portion of the system has been operated through the MIC and only a few hugs remain to be found.
2b3b Wiring is complete for the display controller except for a control panel. This should be finished in the next week and checkout will begin.
2b3c Wire lists are ready for the input devices controller and wiring will begin next week.
2b4 Remote Consoles
2b4a Three keyboards have been received from Friden and look very good.
2b4b Ten of the new mouse models have been received, and the remainder will be delivered in the next week.
2b4c A new five-finger keyset has heen designed and parts are on order for fifteen sets.
3 New-Facility Software
3a MOL
3a1 Programming
3a1a The MOL has heen completely rewritten in its own language. The listing of the compiler in MOL is approximately one-seventh the size of the same program written in ARPAS.
3a1b We are beginning to use the MOL for other programming efforts and are constantly finding new syntactic constructs that we would like to have. As a result, the compiler is in a constant state of expansion.

2


APPENDIX A: COPY OF RECENT RADC FACILITY-DEVELOPMENT REPORT
3a2 Documentation
3a2a The technical report on the MOL has not been updated recently because of the constant change in the syntax.
3b Meta
3b1 Programming: The first version of Tree-Meta is complete and thoroughly checked out.
3b2 Documentation: Progress on the technical report ahout Tree-Meta is slow. A great deal of time is being spent designing the report so that it can be used to teach people about the system as well as serving as a reference guide for experienced users.
3c Control Metalanguage
3c1 Programming: An initial version of the Control Metalanguage has been completed. This version is written in Meta-2. It needs considerable refinement and we are doing this as we translate it to Tree-Meta.
3c2 Documentation: The report on the Control Metalanguage is almost complete.
3d Control Metalanguage Library
3d1 Programming
3d1a General flow charts and specifications for the system have been laid out. Most of the data areas and data structures are well defined. The subroutines which do basic data manipulation, such as text editing and statement-structure manipulation, have been coded but are not checked out.
3d1b Specifications are currently being made for the programs to drive the display. The library programs that will handle user-console input have not heen started as yet.
3d1c A program to produce hard-copy output from files compatible with the Control Metalanguage library is being written. This program is being modeled very closely after the output package for the CDC 3100 On-Line Text System.
3d2 Documentation: We are attempting to produce thorough documentation of all the subprograms as they are designed and written. Thus far this documentation policy has heen successful.

3


APPENDIX A: COPY OF RECENT RADC FACILITY-DEVELOPMENT REPORT
3e Program-System Support Work
3e1 A considerable amount of time has been spent verifying that the software supplied by SDS meets our expectations as well as the contractual commitments of SDS. To the extent that we have been able to check out and verify parts of the system, we are pleased with the results.
3e2 We have added a feature to the system which allows us to get listings of files on an IBM Model 20 available within the Institute.
3e3 We are currently in the process of modifying the system to use the 8-level paper-tape punch.
3e4 We are just beginning work on diagnostic programs for the hardware changes being constructed by us.
4 Plans for Next Month
4a The major effort will continue to be on developing the hardware and software system for the 940.
4a1 We expect the 940 to be accepted and, from all indications, running very well.
4a2 The executive hardware and display controller should be checked out. We see no problem in completing the SRI- constructed portions of the system in time for the arrival of the displays from Tasker in September.
4a3 Minor debugging and modification of MOL and work on its documentation will continue.
4a4 Coding and debugging of CMLL will continue, and design and coding of hardware diagnostics should be essentially complete.
4b In the management-research activity we will continue to experiment with charting techniques and to explore the possible aid available in the teletype terminals.

4


APPENDIX B: COPY OF RECENT SOG RFPORT
0 (sogitpr) Interim Technical Progress Report, Project 6631, covering the period 16 May through 16 August 1967
Oa DCE 18 AUG 67
1 Introduction
1a The work reported follows the suggestions in our Document SOGPLAN (dce 6 JUN 67) rather than our proposal SRI No. ESU 67-31.
1a1 This is as per verbal discussions and agreement with the SOG technical-monitor personnel.
1a2 Item 1b below reviews the most relevant section (Section 3) from the June 6 document.
1b Our suggestion for applying SOG support: Use it to learn how to operate, maintain, and analyze a complex time-sharing system, with particular emphasis upon the problems associated with servicing display consoles.
1b1 Keep a detailed history of all of our problems and report upon those which would give a useful checklist for others who will follow a similar route (with any time-shared computer).
1b2 Develop solutions to these problems and report those that seem to work best for us--e.g., operator practice; trouble logs; crash-recovery techniques to minimize data loss or maximize diagnostic-information recovery; system-debugging while continuing time-sharing operation, etc.
1b3 If Stage 1 is not developed on schedule (so that there is no appreciable Stage-1 time left during the SOG contract period), the most likely reason will be slippage and unreliability in the time-sharing service.
1b3a This will call for maximum concentration on basic principles of operational reliability.
1b4 If Stage 1 is developed near schedule, there will be time subsequently, during this SOG contract period, to begin developing measurement and analysis techniques to help learn what actually goes on in the dynamics of the time-sharing system during our various types of usage.
1b4a This could be coordinated with the ARPA/NASA activity of

1


APPENDIX B: COPY OF RECENT SOG REPORT
user-action recording and analysis (which would get underway after Stage 1). We would be able to correlate system dynamics with concurrent user actions.
1b4b This type of analysis would be a big help to the design of better on-line CRT service by coordinated adjustment of the response-subsystem design and of the various functional parameters of a time-sharing system.
2 Progress During the Quarter
2a Our expenditures during the first half of the contract period have been relatively light--i.e., about 20 percent of the money has been used.
2a1 A very concentrated effort has been sustained during all of this time by the ARPA-supported "facility project," to develop the hardware and software which will give us the Stage 1 6-display time-shared system.
2a2 This system is maturing in a very promising way and we expect relatively soon to be able to free several more system programmers to concentrate upon this area of work.
2b One man (Mr. J. D. Hopper) has been full time on this project since mid-June. His activities have primarily consisted of:
2b1 Learning the system--its organization, its functional use of the hardware, the programming details of its critical parts, etc.
2b2 Making a fairly detailed daily log of our experience with the computer, particularly the time-sharing system.
2b2a This provides a good record of day-to-day reliability--serious crashes are discussed in detail regarding the causes and their cost to us in terms of work loss.
2b2b The occurrences of minor difficulties are also noted.
2b2c We expect to use this as a source of data for evaluating the performance of the system.
2b3 Taking care of the file-protection backup storage procedures--i.e., periodic copying of the users' files onto backup magnetic-tape reels.
2b4 Adding miscellaneous system features--both to develop a better understanding of the system, and to provide useful new services:

2


APPENDIX B: COPY OF RECENT SOG REPORT
2b4a Integrating a third magnetic-tape transport (the original time-sharing system accommodates two).
2b4b Changing the magnetic-tape handling programs to get around some of the demonstrated unreliability of these particular tape transports.
2b4c Adding time-sharing system drivers for paper-tape input/output.
2b4d Providing a generalized message service to distribute messages to all users.
2b4e Increasing the core space allotted to the user executive system to accommodate current and future additions.
2b5 Participating in the occasional discussions among the others of our program about the means of studying the time-sharing system, analyzing its operational characteristics, instrumenting and testing its dynamic behavior, and analyzing the efficiency and possible redesign of the functional overlap sequencing of the various component processes.
2c With respect to this latter type of discussion, we have evolved the following considerations:
2c1 That mysterious thing known as "response time" is the center of our study. Response time is affected by three almost distinct operations: user demands, program structure, and basic timings. Interaction among these three items is one area which certainly needs study.
2c1a To look at the interaction problem first, however, complicates the issue beyond comprehension.
2c1b It seems that the first step is to look at each item individually and try to find basic characteristics which most affect the item as well as its interaction with the other items.
2c1c The value, then, of knowing an operation characteristic is an intuitive "feeling" for the way that characteristic determines response time.
2c1d Later, it will be interesting to study the validity of these intuitive feelings.
2c2 By "basic timings" we mean the length of time the most

3


APPENDIX B: COPY OF RECENT SOG REPORT
primitive operations within the system take. Included in these primitive operations are such things as page swaps, certain BRS's, and basic file accessing. We do not mean here how the swapper works when there are different types of loads on the system; rather we are interested in how long it takes merely to get the page to the RAD and how well the RAD optimizer is working.
2c2a We feel that this is fundamental knowledge which we must have if any study of the total operating system is to be meaningful. There is a large yet-to-be-decided question on the operating system, and the extent of time lost in a system change.
2c2b This question could affect the choice of primitive operations included in the study.
2c3 "Program structure" refers to a global view of the way the program operates when viewed as a single program running on a stand-alone machine. Things which might be studied here are the size of programs, their loop structure, and their memory referencing.
2c3a Clearly, each program has its own individual set of statistics. However, when the programs being studied are the system programs, such as QED or ARPAS, then certain characteristics should have a telling effect on response time.
2c3b The most basic question to ask here is which statistics we really want to gather.
2c3c This area lends itself to hardware measurements more than the others.
2c4 The way in which a user makes demands on the system is such a complicated psychological problem and is affected so greatly by the speed of the system itself that we would like to stay away from this subject until the others are far more clear than they are now.
2c5 The emphasis of study through November will be on basic timings and program structure. As difficult as determining which questions should be asked is determining which can be answered.
2c5a While some statistics can only be gathered in the hardware or the software, others may be gathered in either place, and the cost tradeoffs themselves become an interesting question.
2c5b Finally, there is the problem of displaying the

4


APPENDIX B: COPY OF RECENT SOG REPORT
statistics and making sense from them.
2c6 As an initial two- or three-week start, we would like to spend some time just itemizing basic timing characteristics, trying to clear up this large question of value, and begin looking at the problems of measurement. After some headway has heen made in these most fundamental items, we could begin to look at program structure in greater detai1.
2d In support of the foregoing discussions, we have been exploring various ideas regarding hardware additions and modifications to our system that might facilitate various types of associated measurements.
2d1 One of these changes--simple enough for us to consider squeezing into the current, very crowded engineer-technician schedule--is to make available several special instructions which will set or reset some external flip-flops.
2d1a By patching in at various key points some of the set and reset commands, and by connecting an integrating meter and/or oscilloscope to the output of the flip-flop, we will be able to measure short-term time averages as well as actual time intervals between different internal-process events.
2d2 A number of other additions have been considered, whose value for obtaining a dynamic "view" of on-going internal activities seems quite high, but whose costs also seem relatively high (e.g., $10,000 to $20,000).
2d2a We had not expected to use SOG funds for this kind of hardware and the availability to us of these funds will depend to a considerable extent upon the degree of success which we experience in having our system check out satisfactorily within a "reasonable" time.
3 Plans for Remainder of Project Period
3a Continue the kind of general learning and experimenting activity described in Sec. 3b.
3b Steadily boost the activities described in Sec. 3b5 and discussed in Secs. 3c and 3d.
3c Implement relatively soon (under other sponsorship) the set/reset machine instructions for the external flip-flop as discussed in Sec. 3d1.
3c1 The more sophisticated hardware modifications, as discussed in

5


APPENDIX B: COPY OF RECENT SOG REPORT
Sec. 3d2, have a relatively small probability of being implemented during the remaining three months of this project, but their functional specifications, and the evaluation of their utility may well be included.
3d Within the last two months of the project we should have begun to measure operating timing characteristics, and perhaps to develop special experimental setups in which we can repetitively cycle the system through various processes, with various parameters and "contexts" adjustable--as a means of pursuing better understanding of the basic dynamic characteristics of the system.
3e We will publish at the end of the project a report descrihing our analyses, our instrumentation techniques, our experiments, and the resulting evaluations, as well as recommendations for future studies of this kind.

6


APPENDIX C: GENERAL AHI-PROGRAM BACKGROUND
1 Background of SRI-AHI Program
1a Recent Support History
1a1 Continuous ARPA support since February 1963, at varying levels--during 1965 about $80,000.
1a2 NASA support from midyear 1964 to midyear 1965 at a level of about $85,000.
1a3 Joint ARPA/NASA support since 8 February 1966 (contract NASI-5904) for two years at a level of $250,000 per year.
1a4 RADC support beginning 23 February 1966 (Contract AF 30(602)-4103) with a sum of $93,500, planned for one year, that has been dormant since Fall of 1966 to wait out our transistion to a new computer system.
1a5 Additional ARPA support (arriving in our hands 12 May 1967) of $565,500 to be used to acquire and support a time-sharing computer with multiple-CRT display stations until 8 February 1968 (channeled to us via RADC as an extension of the RADC contract mentioned above).
1b General Nature of Our Research
1b1 We pursue experience and know-how in the design and use of computer augmentation systems, in order to develop principles for their analysis, design, and implementation.
1b2 The basic goal is to accelerate as much as possible the inevitable development of the increased intellectual effectiveness that closely coupled computer aids make possible.
1b3 We are using a combination of "system evolution" and "bootstrapping" approaches.
1b3a By "evolution" we mean the establishment of realistic working systems and their continual study and improvement from a "whole system" viewpoint.
1b3b By "bootstrapping" we mean that the intellectual activity for which we pursue this evolutionary improvement is the AHI research activity itself--so that gains made in intellectual effectiveness turn into gains in the rate and/or quality of evolutionary progress.

1


APPENDIX C: GENERAL AHI-PROGRAM BACKGROUND
1b4 So far the technical developments have mostly involved "instrumentation"--i.e., the development of some basic conventions for the structure and style of-text writing, for a basic system of computer aids for the manipulation and study of such text, and some early changes to working methodology which coordinate with the foregoing.
1c In midyear 1966, these basic developments could be seen to represent a significant evolutionary stage.
1c1 But without a much increased availability of the computer aids, further evolution would suffer considerably from lack of realistic usage experience.
1c2 Moreover, the bootstrapping acceleration would not be realized if these tools were not available for actually doing the work within the AHI Program.
1d Now (midyear 1967) we have just acquired an SDS 940 time-sharing computer.
1d1 We are in the process of acquiring and building special I/O and display hardware, and programming a basic on-line service system to provide us, by fall, with six active display stations, and by later in the fall with a total of 12 stations.
1d2 This system is to be completely dedicated to the furtherance of the AHI research program.
1d3 By the end of the current contract period (7 Fehruary 1968), we expect this 12-station system to be giving smooth, reliable service that will actually allow each AHI researcher to spend most of his working time at a console.
1e The program has begun getting additional support ($24,000, running from 16 May 1967 to 15 November 1967) from the Special Operations Group of the U. S. Army.
1e1 The understanding is that this support will be used within the AHI Program for needed activites, on a mutual-agreement basis.
1e1a Care is being taken (as was done for RADC) to develop an activity area for this support which is explicit and different from any already being worked on within the Program under other support.
1e2 The understanding is also that further support will follow at about the same per-month level.

2


APPENDIX C: GENERAL AHI-PROGRAM BACKGROUND
1f There remains about $38,000 in RADC management-technique research funds, which we have just recently begun to use to establish scheduling techniques that hopefully will help coordinate the implementation of our time-sharing system.
1f1 The understanding is that this support will be extended at a yearly rate of about $85,000.
1g The planned establishment of an ARPA Contractors' network affects our future activity and funding picture.
1g1 We plan to couple our SDS 940 into the communication network, and to develop the software necessary to set up a "circuit" and to communicate with any of the other computers.
1g2 We plan to assume responsibility for establishing and operating a "reference-information" node for this network.
1g2a The nature of this role, and the level of effort required to fulfill it, have yet to evolve.
1g3 To do our part towards accelerating the establishment and operation of the network, we have begun investing manpower in these two efforts, taking the liberty of using the joint ARPA/NASA support funds, assuming a high probability of some form of ARPA "reimbursement" if the expense is significant.
2 General Remarks About the "Bootstrap Community"
2a The Basic Concept
2a1 On the basis of joint use of common data-file and computer-service resources, the various activities (regardless of source of support) within the AHI Program will interact in a coordinated fashion which can best be likened to the relationships within a more or less self-sufficient "Bootstrap Community."
2a2 As much as possible, all active participants in the AHI Program will be considered as members of this Community.
2a3 All Community members are expected to do as much as possible of their AHI-Program work by means of the conventions, procedures, and computer aids being developed and studied within the Community.
2a4 The underlying criterion for directing the activity of each Community member is to maximize the capability of the Community for pursuing the AHI research goals.

3


APPENDIX C: GENERAL AHI-PROGRAM BACKGROUND
2a4a This, of course, affects the process of selecting and directing the various research activities of the Community.
2a4b Also, of course, it affects negotiations and interactions with sponsors who contribute funds in support of various aspects of the Community's activity.
2a5 The policy with respect to sponsorship is to accept within the Bootstrap Community only those sponsors who wish to participate in this particular "experiment" in man-computer research--i.e., those who wish
2a5a to pursue techniques and problems particularly valuable to us in terms of the actual aids they provide to pursuit of AHI research, and
2a5a1 to pursue these by developing, using, and evaluating them in association with meaningful application within the Community.
2a6 Other types of useful man-computer research which do not happen to meet our "Bootstrap Community" requirements (i.e., which would pursue techniques and problems not of high enough value in terms of the actual aids they provide to pursuit of AHI research) would reduce the value derived from expending these resources within the AHI Program too far below what value we feel would be derived if these same resources were expended toward activities which coordinate in the specified manner.
2b In general, each of the activities described below involves a concern for certain services and techniques important to the Community.
2b1 It is assumed that the particular types of intellectual tasks involved in the pursuit of each of these activities will be done as much as possible by the people working at these activities doing their work at the CRT consoles.
2b2 It is further assumed that developing special conventions, procedures, and computer aids for these pursuits is a basic part of the work within each activity.
2b3 In other words, the specific augmentation techniques which we will be implementing and improving will be those to support our own research activities--and the aggregate of these techniques, insofar as they are specialized, will be oriented towards augmenting people who pursue the tasks of analyzing, designing, implementing, and operating computer augmentation systems.

4


APPENDIX C: GENERAL AHI-PROGRAM BACKGROUND
2c Generally, the proportion of total Community resources which go to a given activity will fluctuate with Community needs, progress, understanding of roles, etc.
2c1 A continual review and evaluation process will go on towards learning better how to isolate and evaluate the activities, and decide upon their levels of activity.
3 Full Utilization of the Display-Computer Facility
3a This facility will represent a unique and very valuable asset, and we feel it to be quite important to plan the activities of the Bootstrap Community so as to maximize the value derived from this facility.
3b If our facility is used primarily for first-shift operation, there will be idle time.
3b1 Full 24-hour-per-day utilization of 12 displays throughout the week could provide 50 people each with a full 40-hour work week at a CRT console.
3b2 If we assume that 6 hours per work day is suitably beneficial for our purposes, use of the full round-the-clock week could let some 67 people do full-shift work this way--which, if coordinated in a reasonable fashion, and if people are willing to work third-shift hours, could produce extremely significant research progress.
3c There are several ways in which we could get more useful utilization of the facility:
3c1 During off hours, when the Bootstrap Community is not using the facility, rent available CRT stations (and associated computer service) to any customer (including non-SRI employees) for a reasonable charge. Use the collected charges to reduce the Bootstrap Community computer-lease bill.
3c1a One hazard of this approach is that such users are often hard to cut back in case, for instance, the Bootstrap Community wanted more time.
3c2 We could begin as above, but give more favorable charge rates and scheduling priority according to the intrinsic value we placed upon the work to be done, using the same evaluation framework as we do for our own Bootstrap activities (i.e., how much benefit to Community capability would result).

5


APPENDIX C: GENERAL AHI-PROGRAM BACKGROUND
3c2a We could, for instance, give special encouragement to graduate students for relevant thesis work--and students might be more willing to work third-shift hours.
3c3 We can continue to preserve the entire facility exclusively for people working directly on the AHI Program, using funds directly administered within the Bootstrap Community.
3d The above utilization approaches all provide additional benefit to the AHI Program. We tend to feel that the latter approaches would produce more benefit than the former, but with an increased burden of promotion, planning, coordination, etc.
3d1 Of course, the first possibility would not be without headaches, since we would essentially be running a service, and could suffer considerable distraction thereby.
3e We feel that to derive the maximum value from any incremental utilization of the facility, requiring users to pursue activities which fit the bootstrapping principle remains the best alternative.
3f We would like to plan for a growth in directed Bootstrap Community activities, stemming from a growth of support.
3f1 Explicit plans in this direction are being left until the capacity of the facility to support on-line display consoles is determined. We assume that this will be reasonably well known by the beginning of the new contract period in February 1968.
3f2 We would like to plan on adding staff in various ways:
3f2a Hiring as regular SRI employees more people, and beginning to broaden the professional base, by including people with other than engineering or computer-science hackground.
3f2b Taking on, on a "yearly contract basis," a number of research assistantships for graduate students in various relevant disciplines (probably from Stanford, but possibly from UCB, Santa Clara University, San Jose State College, or San Francisco State College).
3f2c Taking in, under various financial arrangements, mature professionals from various organizations and with various backgrounds on a temporary "resident" basis (as described under "Resident Activity").
3f2c1 For some of these people, we might want to pay salaries; for others we could expect their salaries to be paid from their home organizations; for others we might

6


APPENDIX C: GENERAL AHI-PROGRAlI lA~GRO~D
expect to procure special grant money; and for some it might be appropriate to charge a certain fee.
3f3 We assume that this growth in Bootstrap Community personnel would take place with the approval of current sponsors (i.e., giving assurance that already committed funds will not be diluted--indeed, our confidence is that the return for each sponsor will be heightened).
3f4 On our own part, we will try to limit such expansion to the rate at which we can increase our capability to indoctrinate, coordinate, administer, etc.
3f5 We feel that without further expansion in the list of important activities upon which to invest our resources over the list given in Sec. 4, we could at least double our staff (i.e., to the equivalent of about 24 full-time people) and keep everyone very profitably employed in these pursuits.
3f5a We are fairly confident, however, that this list of activities will expand in scope and be refined in categorization, so that specific roles for even more people (or to be relegated for specific funding from a new sponsor) will rather naturally evolve.

7


APPENDIX D: REPRESENTATIVE RESEARCH ACTIVITIES
1 Overall Planning and Coordinating
1a Under ARPA/NASA support, to study the Community as a whole and to establish and monitor policies, decision criteria, etc., as necessary for its favorable development and evaluation.
1b An important task is activity categorization, evaluation, and planning.
1b1 This involves isolating the roles and responsibilities being assumed in the Community, as well as the needs and possibilities for new or modified roles and responsibilities,
1b1a studying the relationships among them,
1b1b categorizing them into natural groupings which can be treated as "activities,"
1b1c evaluating the dependence of Community effectiveness upon the various activities,
1b1d and deciding how our "activities" should be specified and our resources allocated among them.
1b2 Note: This bring out the fact that the activity categories described below are tentative (they are considered representative and valuable for "playing").
1c Although the actual "management" work associated with this activity is done under ARPA/NASA support, the study, development, and evaluation of more effective means for doing this work would be considered a proper subject for the RADC management-technique activity (below).
2 Study and Development of Management Techniques
2a This activity, as noted above, will properly be sponsored by RADC at least up to September or October 1968.
2a1 It is a very important activity in our scheme of things, and if the RADC support for any reason does not continue throughout the period of this proposed contract, we would want to reapportion the use of ARPA/NASA support to cover this activity at least to some degree.
2b The role of this activity within the Community will be to develop and implement operating management techniques, to study and evaluate

1


APPENDIX D: REPRESENTATIVE RF.SEARCH ACTIVITIES
their use, and to continue the cycle of improvement and evaluation.
2c The basic pursuit of this activity is to learn how to manage an organization composed primarily of on-line workers.
2c1 The "management need" which we thus seek to answer may be characterized as a need for, on the one hand, making more visible the past and present utilization of resources, and on the other hand making more effective the future planning and the modifications of plans to meet new contingencies and developments.
3 Techniques for Improving Time-Sharing Service to Community Users
3a This is the current (provisional) activity for support by the Army Special Operations Group--it will take perhaps six months for us to know how significant to the Community this will become.
3b Initially, it is characterized as a study of the special problems involved in serving CRT users under a time-sharing system and the development, application, and evaluation of special principles and techniques in programming, in time-sharing scheduling, in memory allocation and swapping conventions, in measures to increase service reliability and file safety, etc.
3c This does not involve directly the user features to be developed and experimented with under other Community activities--it merely seeks to improve the response and reliability for such features, so that in turn their study and evaluation can be more significant--and also so that we can perhaps increase the number of simultaneous users which the system can support.
4 Physical Improvement of the Time-Sharing Service System: Maintenance and Operation
4a This activity will be supported by ARPA through RADC.
4b It will include such possibilities as adding more or different types of display stations, increasing the disc-storage or swapping-drum capacities, special hardware to help monitor the dynamic behavior of the time-sharing system, etc., as well as the maintenance of the nonleased equipment.
4c It would also include the costs for miscellaneous supplies required to operate the computer facility.
4d Special equipment is a possibility.
5 Documentation Techniques

2


APPENDIX D: REPRESENTATIVE RESEARCH ACTIVITIES
5a This initially embraces many miscellaneous activities, and may later need to be more clearly broken up into smaller explicit activities.
5b The expository style, as used for documenting our planning, status, specifications, designs, user's conventions and procedures, etc., needs attention as to the practices and conventions which the group uses.
5b1 It is evident that the existing unique features for on-line study and manipulation of structured text not only influence the stylistic tendencies of users, but offer considerable advantages from special conventions and procedures associated with the style of writing.
5b2 For instance, there has appeared a noticeable tendency to chop the textual elements (i.e., statements) into smaller "parcels,"
5b2a and we should seek the reasons, perhaps as affected by subject matter and purpose;
5b2b we should also develop general principles for different kinds of conceptual material being represented.
5b3 Alternatively, detailed structuring within text statements might be established to aid in clarity of presentation and in the composing, studying, and analyzing processes on expository material.
5c Our needs apply equally to nontextual material and to specialized textual materials such as computer programs, mathematical statements, formal logic, etc.
5d There will be a need for coordinating and unifying the analyses, plans, and practices throughout the Community as used in all of its documentation, as well as for the evolution of the particular principles and techniques for studying these language problems.
5e Each of the Community's activities (including this one) will tend to develop special forms of representation and documentation to satisfy specialized needs, and this one centralized activity should have both the authority and the responsibility to lay down the principles and guidelines by which the actual conventions and representations are established.
5f This jurisdiction extends even into the special programming languages which we will be developing and using.

3


APPENDIX D: REPRESENTATIVE RESEARCH ACTIVITIES
5g In particular, this activity will concern itself with the metalinguistic conventions for describing and documenting our principles, rules, and practices of language development and use--and, of course, with the corresponding principles associated with the metalanguage.
6 Real-Time Computer-Control Language
6a To operate an on-line work station, a user has a repertoire of control directives which he may designate, and these often include parameters or operands which need associated designation.
6b The sequential steps in designating these desired service actions are frequently supported with various informative feedback messages from the computer, to enable the user to minimize the redundancy in his communication without risking too much trouble through human mistakes.
6c The continual dialog is carried on with what we call the "control language," which has message elements for both the human (i.e., mouse selections, keystrokes, foot-pedal actions, etc.) and the computer (brightening or blinking of display elements, posting on the display of current values of parameters, audio signals, etc.) which may combine in certain specified ways (i.e., by means of a grammar) to form meaningful dialogue messages.
6d The particular functions which a user needs to perform, relative to the various computer aids which will be developed by the various activities to augment their respective tasks, will of course be the concern of those respective activities.
6e However, study and coordination of the principles, conventions, and practices by which these functions are controlled by the user in his dynamic use of the system would be the responsibility of this control-language activity.
6f In other words, this activity is to develop techniques and principles for
6f1 studying and evaluating our control languages, and
6f2 for designing such languages to meet special criteria of efficiency, learnability, error avoidance, etc., in special realms of user activity such as text manipulation, program debugging, graphic manipulation, etc., as found to be important within the Community.
7 User Actions, Procedures, and Task Protocol

4


APPENDIX D: REPRESENTATIVE RESEARCH ACTIVITIES
7a The actual practices of users can be monitored and recorded with a new dimension of completeness, accuracy, and "sampling validity" when all users are doing the major part of their work at CRT consoles.
7b This particular activity will develop instrumentation (initially software, but possibly leading to special hardware) for deriving such records of user behavior as prove valuable to us.
7c Furthermore, it will develop special principles and techniques for isolating and portraying significant user characteristics.
7c1 This analysis will seek to categorize skills (or their corresponding task capabilities) in what we anticipate to be a hierarchical structure within the context of a given task.
7c2 Furthermore, the analysis will seek to assess the contributions of lower-level component skills to the measure of skill in a higher-level capability.
7d From this analysis, this activity should seek to develop the principles of "procedure design" with respect to criteria of efficiency, learnability, etc.
7e This activity will interact, then, with the other activities in their specification of procedures being specially developed by them.
7e1 In particular, it would coordinate with the control-language activity in the specification of human procedures for using the computer aids.
8 Training
8a Community members must acquire and keep up to date
8a1 an understanding of the principles and practices for using the system in the many different areas of utilization which the different activities will be developing, and
8a2 the operative skills necessary for them to use skillfully the whole range of tools applicable to their particular activities.
8b Since all of these concepts and tools will be continuously evolving, to keep this understanding and skill at a maximally efficient level will require continual testing and training.
8b1 This high level of understanding and skill is necessary from the point of view of maximizing the group's working capability, but it is also necessary to give validity to the Bootstrap

5


APPENDIX D: REPRESENTATIVE RESEARCH ACTIVITIES
Community's evolution of these conventions, procedures, and computer aids which it is our business to study.
8c This activity will be responsible for the following:
8c1 Setting up standards of understanding and skill for various types of Community members--which includes both the range of subject and capability and the level of understanding and skill
8c2 Testing and evaluating the understanding and skill of individual members
8c3 Training individual Community members to bring their understanding and skill to the specified levels
8c4 Developing special conventions, procedures, and computer aids to augment the above functions.
9 Service System Implementation
9a This activity includes hardware and software development to provide specified work-station response to the user's control actions.
9b Programming
9b1 This activity is responsible for the organization and implementation of software so as to give both effective service and flexible modification and growth.
9b1a It will develop conventions, procedures, and computer aids to improve our capability to study, evaluate, and design the overall software organization, and
9b1b to write, debug, and document specified programs.
9c Engineering
9c1 Directly parallel to the programming activity, this activity is responsible for the organization and implementation of hardware so as to give both effective service and flexible modification and growth.
9c1a It will develop conventions, procedures, and computer aids to improve our ability to study, evaluate, and design the overall hardware organization, and
9c1b for designing, constructing, debugging, and documenting specific pieces of hardware, as well as for locating,

6


APPENDIX D: REPRESENTATIVE RESEARCH ACTIVITIES
evaluating, and procuring "buy-out" hardwqre.
10 Intelligence
1Oa This activity involves acquiring, and making available for Community use, information from the outside world required for the effective pursuit of the Community's goals.
1Oa1 Such information includes the gamut of publications (from advertising flyers and newspaper clippings to journal articles and books) as well as notes taken on observations, second-hand reports, hardware "catalog" data, etc.
1Oa2 There will be considerable overlap with the Network Information Center activity.
1Ob This activity is also responsible for assessing the needs within the Community for this type of service, and
1Oc for developing conventions, procedures, and computer aids to improve its effectiveness in fulfilling the above responsibilities.
1Od Our X-DOC collection, containing some 2800 entries of assorted documents related toAHI research interests, represents a good starting base for this activity. It is already in machine form.
11 Dissemination
11a This activity involves the documents, presentations, movies, etc. given to the outside world to disseminate the results of our thought and work, to yield a maximum external-world benefit from it.
11a1 It is responsible for analyzing the products of our thought and activity for uniqueness and relative value;
11a2 for assessing the needs of various "audiences" and determining the best selection of material and means of dissemination to satisfy these needs;
11a3 and for developing conventions, procedures, and computer aids to improve our effectiveness in this activity.
11a3a This includes aids for making movies, for giving presentations, and for producing printed documents, as well as for the conceptual work of developing and organizing the given materials.
12 Residency Program

7


APPENDIX D: REPRESENTATIVE RESEARCH ACTIVITIES
12a This activity involves bringing in outside people to become full Community residents for limited periods of time in order to provide a special degree of transfer of skill, information, and perspective between these people and the Community.
12a1 This transfer is assumed to be a two-way matter, and thus this activity will be closely related to both the intelligence and dissemination activities.
12b It is responsible for assessing the needs and possibilities for this type of arrangement, for locating and evaluating candidates, and for arranging for transportation, housing, and financing.
12c It is also responsible for the methods and practices whereby the residents are given the necessary orientation and training and are integrated into useful roles in Community activities.
12c1 Their experience should be with participation in actual meaningful Community activities so that the attitudes, concepts, and skills to which they are exposed may be provided a meaningful framework, and so that interaction with Community members will be with regard to meaningful specifics.
13 Development and Operation of a Network Information Center for the ARPA Computer Network.
13a It is evident that a computer network will need a standardized technique for obtaining operational reference data. Where the operations in question concern the whole network and are subject to continual change (as with the experimental network), there will need to be a central responsibility for gathering and updating such information and for making it available to the network users.
13b The SRI AHI Program has agreed to take on this responsibility. It is not a clear-cut assignement, and the manpower and costs are rather unpredictable.

8


APPENDIX E: NETWORK MESSAGE-PROCESSOR SPECIFICATIONS
O (ebsmp) A Possible Specification for the Message Processor of the ARPA Computer Network
Oa EB SHAPIRO 18 Aug 67
Ob Distribution: 5890 File
1 Introduction
1a One of the present concepts of the ARPA computer network, as shown in Figure 1, involves the use of a message processor (MP). This small, digital computer would be directly attached to and collocated with the primary processor (PP). The MP would be interposed between the PP and the communication plant, the latter consisting of transmission links and switches.
1b The operation of the network necessitates the flow of data and control information between MP's and MP's, and also between MP's and PP's. For the moment, no interchange is assumed to occur directly between PP's. This memorandum discusses the qualitative nature of these flows.
1c From these flow considerations, a set of functions is derived for the MP's and the PP's. These functions describe the role of each type of processor in the operation of the network. The following are of concern:
1c1 Minimizing the "disturbance" to existing PP equipments and programs
1c2 Optimizing the use of PP and MP storage facilities and logic capabilities
1c3 Maintaining high uniformity among MP's for both equipment and programs.
2 Some Broad Concepts
2a Exchanges Between MP's
2a1 An MP may have a multiplicity of full duplex channels connected to the communication facility. We may think in terms of an upper limit of perhaps 25 such channels. For a given MP these channels would be independent, and for a given channel each direction of transmission would be independent. "Independence," as used here, has a restricted meaning referring only to the

1



Network Layout

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

2


APPENDIX E: NETWORK MESSAGE-PROCESSOR SPECIFICATIONS
nature of the messages handled--times of origination and sizes (measured in bits).
2a2 An MP has only one full-duplex channel connected to its associated PP. Each direction of transmission of this channel is independent. Typically, a PP will have only one MP serving it; however, if several MP's are connected to a PP (for purposes of traffic-handling capacity or reliability), it is assumed that a separate, independent, full-duplex channel will exist between the PP and each MP.
2a3 Two types of messages are assumed to exist within the network complex--immediate and deferred. For each channel (and direction of transmission), there may simultaneously exist a queue of immediate messages and a queue of deferred messages. Within each queue the service is on a first-in, first-out basis. However, for a given channel and direction of transmission, service cannot be provided to messages in the deferred queue until all immediate messages have been served. Furthermore, service to a deferred message can be interrupted to serve newly arrived immediate messages. Immediate messages are not interruptable. When service for an interrupted deferred message is resumed, the service begins at the point of interruption.
2a4 The deferred message is used because it provides a means of handling large volumes of data transfers as a background task (transfers of files, programs, core dumps), while the smaller volume demands of many on-line applications can be rapidly handled as a foreground activity.
2b Exchanges Between MP and PP
2b1 The manner in which the PP-MP channel should operate involves the problem of message storage in both processor types. If minimum storage is desired in the MP, then, as an extreme, only one bit per channel (MP to communication network) need be provided. Such an approach forces all message queues to reside in the PP; furthermore, extensive logic overhead is required to coordinate the actions of the MP and the PP.
2b2 At the other extreme, minimum storage is desired in the PP, in which case the MP may need to behave as if it had an infinite store. In this approach, all message queues are maintained in the MP; also, the logic overhead is minimized in coordinating the MP and PP.
2b3 Between these two extremes there exist intermediate forms of transfers--by character, by "block," by several blocks, by message, and by several messages. It is expected that a

3


APPENDIX E: NETWORK MESSAGE-PROCESSOR SPECIFICATIONS
combination of these intermediate forms will be used.
2b4 Consider that a "block" (used for error control purposes) consists of 256 characters (8-bit ASCII). Let eight blocks (2048 characters) be the maximum amount of data that can be transferred as an entity between an MP and PP. This volume could handle a display field on a large CRT. If a substantial fraction (90 percent) of all messages are accommodated in this manner, then only one MP-PP transfer is required to dispose of a message. The remaining messages, probably including all of the deferred type, will be larger and necessitate multiple transfers between MP and PP, with an increase in associated logic overhead.
2b5 An estimate of an upper bound of the MP message store can be made as follows: For a given channel the MP must allow, for transmission to the network, storage of 2048 characters for an immediate message, and another 2048 characters for a deferred message. An additional 4096 characters of storage are needed for the same channel to handle transmissions from the network, thus allowing full-duplex operation. Each network channel thus requires 8192 characters of storage.
2b6 If an MP is equipped for 25 channels, then 204,800 characters of message storage are needed. At seven bits per character, this is 1.43 megabits--a small disc or drum might be used.
3 An Operating Procedure
3a An operating procedure will be formulated encompassing PP-to-MP transfers, MP-to-MP transfers, and MP-to-PP transfers. From this procedure, a picture of data and control information flows will evolve.
3a1 The PP-to-MP Procedure
3a1a A PP should consider a message to be ready for handling by its MP only when all of the data are available in store (be it core, drum or disk). The MP is notified by the PP of the message's ID (perhaps it is the file ID), PP of destination, immediate or deferred designation, ASCII or binary coding, and the number of characters in the message. These details are retained in the queue directory maintained by the message processor.
3a1b The sending MP is responsible for establishing a communication path with the MP of the destination PP. If such a path already exists, then the message waits its turn in the queue. If a path does not exist, it is created by a "dial-up" activity.

4


APPENDIX E: NETWORK MESSAGE-PROCESSOR SPECIFICATIONS
3a1c When the message is ready for service, the sending MP requests the first 8 blocks from the sending PP. The sending PP proceeds to fetch the data and transfers 8 blocks or less. If less than all 8 blocks are transferred, the sending PP must notify the MP when the last character has been transferred.
3a1d The sending MP requests permission of the receiving MP to proceed with the transmission of the message. If permission is denied, the message is held at the sending MP, and all other messages to that receiving MP and PP are held in the sending PP. The sending MP, under this delayed condition, can still exchange traffic with other MP's; in fact, it can receive traffic from the MP to which it cannot send.
3a1e Once permission is granted, the 8 blocks are transmitted. Additional blocks are then requested if additional portions of the message remain in the PP. If this is the case, and it is an immediate message, a portion of a waiting deferred message, already in the MP, may be sent to the receiving MP, while the remainder of the immediate message is fetched.
3a2 The MP to MP Procedure
3a2a The sending MP precedes each 8-block transmission to the receivinR MP with a header block. This block is not one of the data blocks and never directly reaches the receiving PP. The header contains the ID of the sending PP, the message ID, indication of whether it is a new message or a continuation, immediate or deferred designation, ASCII or binary coding, and, if a new message, the number of characters in the message.
3a2b Following transmission of the header block, the receiving MP either grants or denies permission to send. The permission decision is made by the receiving MP, strictly on the basis of storage availability. Lack of storage space indicates a backup on the part of the receiving PP.
3a2c The receiving MP must notify the sending MP when a denial is rescinded. Upon receipt of the rescission, the sending MP must once again send the header. This process allows the sending MP to adjust its tactics to the network as of the moment of action.
3a2d Error checks are provided on each block, with repeat transmissions used to overcome errors. These checks and retransmissions are not known by the PP's.
3a2e The sending MP terminates each transmission of a group of

5


APPENDIX E: NETWORK MESSAGE-PROCESSOR SPECIFICATIONS
blocks with an "end of message" code or a "more to come" code. In the case of immediate messages, an "end of message" code should normally accompany all transmissions of less than 8 full blocks. An 8-full-block group may terminate with an "end of message" or a "more to come" code.
3a2f In the case of deferred messages, the "end of message" code can occur at any time, and the "more to come" code can occur at the end of any block.
3a2g Even if the entire final block group of a message is not used by a message, that entire block group remains dedicated to that message and cannot be shared with other messages. This applies at the sending MP and the receiving MP.
3a3 The MP-to-PP Procedure
3a3a Upon receipt of the header of a new message, the receiving MP forwards the information to the receiving PP. This information provides
3a3a1 partial identification of the user,
3a3a2 an ID for the file to be established, and
3a3a3 an estimate of the file size.
3a3b When the receiving MP has gathered a block group of data, it requests the receiving PP to accept it, indicating the message ID. If an "end of message" is involved, the receiving MP provides the receiving PP with an end of file, or equivalent, signals when the last data character has been transferred to the PP.
3a4 Some Observations
3a4a These procedures appear to lend themselves to
3a4a1 asynchronous transfers between processors,
3a4a2 great freedom in the operation of the communication system (highly flexible and dynamic writing is possible), and
3a4a3 the use of MP's in transit (store-and-forward type) operation.
4 A Description of Major Control-Information Exchanges

6


APPENDIX E: NETWORK MESSAGE-PROCESSOR SPECIFICATIONS
4a This section lists the major items of control information exchanged between processors--sending PP(SPP), sending MP(SMP), receiving MP(RMP), receiving PP(RPP). The specific forms (electrical and logical) of these signals are not covered here.
4a1 Signals
4a1a (S1): (SPP to SMP)--There is a new message with IDi, of type immediate or deferred, of coding ASCII or binary, of size a.
4a1b (S2): (SMP to SPP)--Send block group of message with IDi.
4a1c (S3): (SPP to SMP)--Here is block group of message with IDi.
4a1d (S4): (SPP to SMP)--There is more to come after this block group. If the last block is involved, signal S13 is used instead.
4a1e (S5): (SMP to RMP)--There is a new message with IDi, from PPc, of type immediate or deferred, of coding ASCII or binary, of size a. Signal S14 is used if a last block is involved.
4a1f (S6): (RMP to SMP)--Request to transmit granted (see S17 if request is denied).
4a1g (S7): (RMP to SMP)--Here is block group of message with IDi.
4a1h (S8): (SMP to RMP)--There is more to come after this block group. Signal S15 is used if a last block is involved.
4a1i (S9): (RMP to RPP)--There is a new message with IDi, from PPc, of type immediate or deferred, of coding ASCII or binary, of size a.
4a1j (S10): (RMP to RPP)--Here is a block group of message with IDi.
4a1k (S11): (RPP to RMP)--Send block group of message with IDi.
4a1l (S12): (RMP to RPP)--There is more to come after this block group. Signal S16 is used if a last block is involved.
4a1l1 The transfer of additional block groups of message i requires repeated execution of the sequence of S2 through S12 until the last block group is encountered.

7


APPENDIX E: NETWORK MESSAGE-PROCESSOR SPECIFICATIONS
4a1m (S13): (SPP to SMP)--There is no more to come after this block group.
4a1n (S14): (SMP to RMP)--Here is more of message with IDi.
4a1o (S15): (SMP to RMP)--There is no more to come after this block group.
4a1p (S16): (RMP to RPP)--There is no more to come after this block group.
4a1q (S17): (RMP to SMP)--Request to transmit is denied. When space is available at the RMP, the following occurs:
4a1r (S18): (RMP to SMP)--Transmit denial is rescinded. The SMP returns to S5, and does not interpret execution of S17 as permission to transmit. Some other special signals are useful:
4a1s (S19): (SPP to SMP, RPP to RMP)--PP out of service.
4a1t (S20): (SMP to SPP, RMP to RPP)--MP out of service.
4a1u (S21): (RMP to SMP)--RPP out of service.
4a1v (S22): (SMP to SPP)--RPP out of service.
4a1w (S23): (SMP to SPP)--RMP out of service.
4a2 Time Sequences
4a2a The previous description is not intended to imply that the operations need occur in strictly sequential order. Indeed, there exist many opportunities for overlapping operations; in particular, SPP and SMP actions can proceed in parallel with RPP and RMP operations. Furthermore, each processor is expected to be dealing simultaneously with a multiplicity of messages, and each MP with several other MP's.

8


APPENDIX F: NETWORK MF.SSAGE-HANDLING FUNCTIONS
0 (ebsmp1) Assignment of Message Handling Functions to Processors of the ARPA Computer Network
Oa EB SHAPIRO 18 AUG 67
Ob Distribution: 5890 File
1 Introduction
1a This memorandum describes a possible assignment of message handling tasks to the various processors--primary processor and message processor--at a node of the ARPA network. The philosophy described here is an extension of the proposed operating procedures previously described. These procedures are briefly reviewed.
1a1 At a typical node, there will be a single processor called the primary processor (PP). This machine could be an SDS 94D, an IBM 360/50, an IRM 360/67, a UNIVAC 1108, a PDP-6, etc. Connected to the primary processor is another machine called the message processor (MP). The MP also connects to the communication facilities, both transmission and switching, of the communication network; thus, the MP serves to isolate the PP from that network. MP'S can be interconnected via the communication network.
1a2 A single high-speed link permanently interconnects the PP and MP at a given node. For reasons of loading or reliability it may prove necessary at some nodes to connect several MP'S to a single PP; under these conditions, an independent link should permanently interconnect each MP to the common PP. In multiprocessor systems it is assumed that the assemblage of PP's will appear logically as a single PP to an MP, and one and only one PP-MP link will still exist for an MP.
1a3 The PP-to-MP links are assumed to operate in a full-duplex manner so that it will be possible for information transfer to occur from PP to MP simultaneously with an independent transfer from MP to PP. This full-duplex mode of operation is assumed to apply to all links interconnecting MP's. An MP will have several, perhaps as many as 25, ports connected to the communication network. Ports that are in use will be connected to the full duplex transmission channels, one channel per port. Some of these other ports will be connected to switched channels. These latter ports are "unused" when not connected to a switched channel. Other ports may be connected to nonswitched, i.e., permanent, channels.

1


APPENDIX F: NETWORK MESSAGE-HANDLING FUNCTIONS
1a4 The node initiating a communication is called the sending node; its PP is called the sending PP (SPP), and its MP is called the sending MP (SMP). The recipient of that communication is called the receiving node, and it consists of an RPP and an RMP. A given node may be in simultaneous communication with several other nodes. Furthermore, a given pair of nodes may simultaneously exchange several independent messages.
1a5 A result of this simultaneity of message handling is that a given node will at a given instant be playing several roles, some as a sending node, the others as a receiving node. Even with a given pair of nodes at a given instant of time, one node may play the roles of sending node and also receiving node with respect to the other node.
2 The Notion of A Message
2a A "message" will be considered to consist of two types of information--control and data. Control information supports the process of transporting the data information from source to destination. The data may be coded in terms of characters, it may represent binary data, it may be a program, etc.; in general, it need not be "textual" in nature. In the handling of a given message, there will be only one SPP, one SMP, one RMP, and one RPP. It is possible, however, that additional message processors may handle the message as intermediate, "transit" elements of the network. It is assumed, then, that all messages are single-address in nature; no multiple or broadcast messages are allowed. If the same data information is to be sent to several nodes, then it is assumed that the sending node will formulate a single message to each receiving node.
2b Consider now a given message and the pair of nodes involved. The control information of this message will flow in both directions between the pair. The data information is assumed to flow only from the sending node to the receiving node. If this message gives rise to a reply, then the reply is a separate message originated by the receiving node of the initial message.
3 Message Queues and Service Doctrines
3a It is to be expected that delays in message handling will be encountered in the operation of the entire network. A primary cause of these delays is likely to be congestion of the transmission links of the communication network. Other causes will be congestion in the message processors and in the primary processors. In these latter two areas, there will be problems with availability of processor time and storage space.

2


APPENDIX F: NETWORK MESSAGE-HANDLING FUNCTIONS
3b The message queues that result will, for the most part, be physically located in the PP's. It is assumed that the MP's will provide a limited amount of storage for their ports.
3c Movement of information through the computer network should occur by blocks on an asynchronous basis. This mode of operation can result in the cessation of transmission of a given message by the SPP when the RPP of that message lacks available storage space. When such space becomes available, transmission can resume.
3d For each node the MP has the responsibility of maintaining the directory of the queue for messages leaving the node. Using this directory, the MP requests data of the desired messages. For each active port, the MP maintains two queue directories--immediate and delayed. The MP attempts to keep each active port busy sending messages. To this end, the MP serves messages from the immediate queues on a first-in, first-out basis. When the immediate queue for a given port becomes depleted, then the MP serves messages from the delayed queue, also on a first-in, first-out basis. Transmission of delayed messages can be interrupted (to be resumed later) in order to serve newly arrived immediate messages. When both queues of a port are empty, then message transmission can cease.
3e This service doctrine provides for a reasonably orderly operation of the ports. At the single PP-MP link, this orderliness is lost because of the interleaving, in time, of portions of many different messages. This interleaving makes it necessary that messages, or portions thereof, be uniquely identified. Tags used for this purpose can be employed throughout the network.
4 Functions of a Primary Processor
4a This section describes a possible set of functions for a primary processor to support a computer network using an operating philosophy as described above.
4a1 Sending Functions
4a1a Function List
4a1a1 Provide storage space for data of messages to be sent.
4a1a2 Assign unique tag to message for identification purposes.
4a1a3 Determine message size.
4a1a4 Determine name of receiving node.

3


APPENDIX F: NETWORK MESSAGE-HANDLING FUNCTIONS
4a1a5 Assign priority to message--immediate or deferred.
4a1a6 Assign coding to message--ASCII or binary.
4a1a7 Send control data (Functions 4a1a2 through 4a1a6 above) to SMP when entire message text is in store.
4a1a8 Send a block of message data to SMP upon demand of SMP.
4a1a9 Relate the SMP-provided message tag to the stored data location.
4a1a1O Notify SMP if block ends message, if block is less than full size.
4a1a11 Release storage space when all of text has been transferred to SMP.
4a1a12 Notify its associated MP that it is OK.
4a1a13 Dispose of undeliverable messages.
4a1b Comments on Sending Functions
4a1b1 A major feature of these functions is that the SPP maintains complete control over the allocation, access, and use of its stores. Addressing is under the control of the SPP. Swapping can take place without undue restrictions. However, the text requested by the SMP may not be immediately available to the SPP. When this condition occurs, all subsequent SPP-to-SMP transfers are delayed until the text has been moved.
4a1b2 Function 4a1a13 above is concerned with the situation where a request for message delivery cannot be honored. There are three causes for such a condition: the receiving node is out of service, there are difficulties with the communication facilities, or the SMP is out of service. It should be appreciated that these conditions could arise at any time, even in the midst of a message transfer.
4a2 Receiving Functions
4a2a Function List
4a2a1 Provide storage space for the data of messages to be received.

4


APPENDIX F: NETWORK MESSAGE-HANDLING FUNCTIONS
4a2a2 Relate the message tag, as used by the RMP, to the file name associated with the allocated storage space.
4a2a3 Assemble the message data in accordance with the message coding--ASCII or binary.
4a2a4 Accept and process control data provided by the RMP to facilitate access validity checks and storage allocations.
4a2a5 Receive a block of message data from the RMP upon demand of the RMP.
4a2a6 Act upon receipt of an "end of message" signal from the RMP.
4a2a7 Notify its associated MP that it is OK.
4a2a8 Dispose of incomplete messages.
4a2b Comments on Receiving Functions
4a2b1 The RPP retains complete control over the allocation, access, and use of its stores. The RMP-provided information on the message sizes should aid the allocation task.
4a2b2 Lack of available PP storage space only results in a delay for all RMP to RPP transfers.
4a2b3 The last function above (4a2a8) is concerned with the situation where delivery of a message cannot be completed to the RPP because of an out-of-service condition of the RMP, the sending node, or the communication facilities.
5 Functions of a Message Processor
5a This section describes a possible set of functions for a message processor to support a computer network using an operating philosophy as described in Secs. 1 through 3 of this memorandum.
5a1 Sending Functions
5a1a Function List
5a1a1 Store the SPP-provided control information for all messages being sent or awaiting service.
5a1a2 Form a directory of the messages in the SPP queue.

5


APPENDIX F: NETWORK MESSAGE-HANDLING FUNCTIONS
5a1a3 Control the order in which messages are to be served.
5a1a4 Assign an SMP port to serve each message.
5a1a5 Determine message routing, and cause connections to be established as necessary on ports with switchable communication channels.
5a1a6 Relate the SPP-assigned message tag to the message.
5a1a7 Pass control information to the RMP for each message.
5a1a8 Assign storage space (buffer) to each sending direction for each active port.
5a1a9 Monitor the status of each sending buffer.
5a1a1O Cause nonempty buffers to send information to their associated ports.
5a1a11 Control the coding and disassembly of the data in the buffer in accordance with the message coding--ASCII or binary.
5a1a12 Request an additional message block from the SPP when the buffer becomes empty.
5a1a13 Control the filling of the buffer with the SPP-provided data.
5a1a14 Terminate buffer filling and emptying in response to "end of message" conditions.
5a1a15 Develop error-control signals.
5a1a16 Form blocks of information, for error-control purposes, for transmission to the RMP.
5a1a17 Respond to "acknowledge" signals from the RMP--proceed with transmission of next block, delay transmission, retransmit last block, resume transmissions.
5a1a18 Check for the presence of "invalid" control characters in the message data.
5a1a19 Interleave control characters and data in transmissions to the RMP.

6


APPENDIX F: NETWORK MESSAGE-HANDLING FUNCTIONS
5a1a20 Separate control characters in the information transmitted by the RMP.
5a1a21 Provide and maintain synchronization and fill on the outgoing communication channels; cooperate with the RMP on incoming channels.
5a1a22 Determine whether the receiving node or the communication facilities are in service.
5a1a23 Determine if the SPP is in service.
5a1a24 Tell the SPP of the status determined as part of the above function (5a1a22) and of the status of the SMP.
5a1a25 Gather traffic, error, and other statistics.
5a1b Comments on sending Functions
5a1b1 The tasks of error control, of maintaining the flow of messages through the network, and of exercizing control over the network and the communication facilities do not extend beyond the MP. Depending upon the structure of the MP, other more detailed tasks may appear. For instance, if a drum or disk is used as a mass store, then that flow of information must be controlled, too.
5a2 Receiving Functions
5a2a Function List
5a2a1 Receive message control information from the SMP, retaining that part necessary for the RMP, passing to the RPP that needed by the RPP.
5a2a2 Indicate to the SPP whether the RMP and the RPP are operable.
5a2a3 Assign storage space (buffer) to each receiving direction for each port.
5a2a4 Monitor the status of each receiving buffer.
5a2a5 Cause nonfull buffers to assemble and receive information from the SMP, associating the message and its tag with its assigned buffer.
5a2a6 Cause the SMP to suspend transmission to a port when a full buffer exists.

7


APPENDIX F: NETWORK MESSAGE-HANDLING FUNCTIONS
5a2a7 Indicate to the RPP the desire to transfer a specific message block when an error-free full buffer exists.
5a2a8 Transfer the proper data to the RPP from the buffer when the RPP gives permission.
5a2a9 Allow SMP transmission to resume when a previously full buffer is emptied.
5a2a1O Terminate buffer filling and emptying in response to "end of message" conditions.
5a2a11 Remove control signals interleaved in the information from the SMP, placing "clean" data in the RPP buffer.
5a2a12 Reassemble the received data according to the message coding--ASCII or binary.
5a2a13 Provide error detection on information received from the SMP.
5a2a14 Request retransmissions of SMP when errors are detected.
5a2a15 Cooperate with the SMP so as to provide and maintain synchronization and fill on the communication channel.
5a2a16 Handle the situation where a complete message cannot be received.
5a2a17 Gather traffic, error, and other statistics.
5a2b Comments on receiving Functions
5a2b1 The RMP tends to perform inverses of the operations of the SMP. Unlike the SMP, however, the RMP is not concerned with the problems of message queuing, except as they relate to the assignment of RMP storage space.
6 Other Functions
6a Transit Operations
6a1 It may develop that messages need to pass through one or more MP's before reaching the specified RMP. One simple way to accomplish this avoids the storage of more than one or two message blocks in any MP. The intermediate MP (IMP) determines from the

8


APPENDIX F: NETWORK MESSAGE-HANDLING FUNCTIONS
message-control information that its own PP is not involved. Instead the IMP originates a message to another MP. At the IMP, incoming data is handled as if the IMP were an RMP.
Incoming-to-outgoing buffer transfers can occur, and with outgoing data the IMP behaves as an SMP. To effect the buffer-to-buffer transfers, the IMP looks upon itself as a form of SPP and RPP.
6b Multiple MP's at a Node
6b1 When a node has several MP's, there is an additional burden to direct outgoing messages to the desired MP. The selection may be performed by the PP or the MP's. The choice is not clear at this time. If the selection criterion is based on the destination of the receiving node, then the task might best be performed by the PP. If the criterion is based on traffic and delay, then the selection might best be performed by an assemblage of cooperating MP's.
6b2 When directing traffic to such a node, the SMP may need to determine the one MP of the many to serve as the RMP.
6c User and Subsystem Control
6c1 A function not performed by the MP's is the identification of the user associated with a given message, nor the disposition of the message in the RPP. It has been assumed that the information associated with these matters is incorporated in the message data and outside the jurisdiction of the MP's, except that the MP's play the minor role of identifying the SPP, even if this information is also found in the data.

9