Challenges for Explainable AI

Robert Hoffman, Institute for Human and Machine Cognition


This presentation will discuss the state-of-the-art with regard to explainable AI, focusing on the conceptual and representational "traps" that seem to characterize work in the area. One trap has to do with terminology that is used to discuss explanation, and the concepts behind the terminology. Another trap has to do with the received model of the explanation process. A third has to do with history: The failure to recognize the historical precedents of ideas and past accomplishments. A fourth trap has to do with mentalistic terminology that is misleading to those outside of computer science. The presentation will then put forward some ideas about how the science might be improved: by recognizing and leveraging the process of "self-explanation," and by using analogies to create global explanations. Finally the presentation will concentrate on proposals for how to improve the research, especially the need for a collaboration of computer scientists and experimental psychologists.


Hoffman is a recognized world leader in cognitive systems engineering and Human-Centered Computing. He is a Senior Member of the Association for the Advancement of Artificial Intelligence, Senior Member of the Institute of Electrical and Electronics and Engineers, Fellow of the Association for Psychological Science, Fellow of the Human Factors and Ergonomics Society, and a Fulbright Scholar. His Ph.D. is in experimental psychology from the University of Cincinnati. His Postdoctoral Associateship was at the Center for Research on Human Learning at the University of Minnesota. He served on the faculty of the Institute for Advanced Psychological Studies at Adelphi University. He has been Principal Investigator, Co-Principal Investigator, Principal Scientist, Senior Research Scientist, or Principal Author on over 60 grants and contracts including alliances of university and private sector partners. He has been a consultant to numerous government organizations. He has been recognized internationally in the fields of psychology, remote sensing, human factors engineering, intelligence analysis, weather forecasting, and artificial intelligence, for his research on the psychology of expertise, the methodology of cognitive task analysis, human-centering issues for intelligent systems technology, and he study of cognitive work in cyber defense. His current research focuses on experimental design for Explainable AI, and the development of a unified theory of macrocognitive work systems.


Selected Books
  1. Hoffman, R.R., LaDue, D., and Mogil, H.M., Roebber, P., and Trafton, J.G. (2017). Minding the Weather: How Expert Forecasters Think. Cambridge, MA: MIT Press.
  2. Hoffman, R.R., Ward, P., DiBello, L., Feltovich, P.J., Fiore, S.M., and Andrews, D. (2014). Accelerated Expertise: Training for High Proficiency in a Complex World. Boca Raton, FL: Taylor and Francis/CRC Press.
  3. Hoffman, R.R. (Au., Ed.) (2012). Collected Essays on Human-Centered Computing, 2001-2011. New York: IEEE Computer Society Press.
  4. Crandall, B., Klein, G., and Hoffman R.R. (2006). Working Minds: A Practitioners Guide to Cognitive Task Analysis. Cambridge, MA: MIT Press.
  5. White, R.A., Coltekin, A., and Hoffman, R.R. (Eds.) (2018). Remote sensing and cognition: Human factors in image interpretation. Boca Raton, FL: CRC Press.
  6. Ericsson, K.A., Hoffman, R.R., Kozbelt, A., and Williams, M. (2018). Cambridge handbook of expertise and expert performance (2nd. ed.). Cambridge: Cambridge University Press.
Selected Recent Publications
  1. Hoffman, R.R., Mueller, T., Mueller, S.T., Klein, G., and Clancey, W.J. (2018, May/June). Explaining Explanation: A Deep Dive on Deep Nets. IEEE: Intelligent Systems, pp. 87-95.
  2. Klein, G., Shneiderman, B., Hoffman, R.R. and Ford, K.M. (2017, November/December). Why expertise matters: A response to the challenges. IEEE: Intelligent Systems, pp. 67-73.
  3. Hoffman, R.R., Mueller, S.T., and Klein, G. (2017, July/August). Explaining Explanation: Empirical Foundations. IEEE Intelligent Systems, pp. 78-86.
  4. Hoffman, R.R. (2019, Spring). The concept of a "Campaign of Experimentation for cyber operations. Cyber Defense Review, 4 (1), 75-84.
  5. Moore, R.T., & Hoffman, R.R. (2019). Cognition and expert-level proficiency in intelligence analysis. In Ward, P., Schraagen, J.M., Ormerod, T.C. & Roth, E. (Eds), The Oxford Handbook of Expertise. Oxford: Oxford University Press.
  6. Hoffman, R.R., Cullen, T. M., and Hawley, J.K. (2016). Getting a grip on the myths and costs of automation. Bulletin of the Atomic Scientists. [DOI: 10.1080/00963402.2016.1194619]