Holographic Reduced Representation
Distributed Representation for Cognitive Structures
Tony A. Plate
While neuroscientists garner success in identifying brain regions and in analyzing individual neurons, ground is still being broken at the intermediate scale of understanding how neurons combine to encode information. This book proposes a method of representing information in a computer that would be suited for modeling the brain's methods of processing information. Holographic Reduced Representations (HRRs) are introduced here to model how the brain distributes each piece of information among thousands of neurons. It had been previously thought that the grammatical structure of a language cannot be encoded practically in a distributed representation, but HRRs can overcome the problems of earlier proposals. Thus this work has implications for psychology, neuroscience, linguistics, and computer science, and engineering.
Tony A. Plate is a research scientist at Black Mesa Capital in Santa Fe, New Mexico.
 Preface
 1 Introduction
 1.1 Representations, descriptions, and implementations
 1.2 Connectionist issues
 1.3 Representational issues
 1.4 Connectionist representions
 1.5 Reduced Descriptions
 1.6 Book outline
 2 Review of connectionist and distributed memory models
 2.1 Localist connectionist models
 2.2 Distributed connectionist models for simple structure
 2.3 Connectionsist models that learn to process language
 2.4 Distributed connectionist models for explicit representation of structure
 2.5 Learning distributed representations
 3 Holographic Reduced Representation
 3.1 Circular convolution and correlation
 3.2 Superposition Memories
 3.3 The need for cleanup memories
 3.4 Representing complex structure
 3.5 Constraints on the vectors and the representation of features and tokens
 3.6 Mathematical Properties
 3.7 Faster comparison
 3.8 The capacity of convolution memories and HRRs
 3.9 Convolutionbased memories versus matrixbased memories
 3.10 An example of encoding and decoding HRRs
 3.11 Discussion
 3.12 Summary
 4 HRRs in the frequency domain
 4.1 Circular Vectors
 4.2 HRR operations with circular vectors
 4.3 Relationship to binary spatter codes
 4.4 Comparison with standard system
 5 Using convolutionbased storage in systems that learn
 5.1 Trajectoryassociation
 5.2 Encoding and decoding systems for trajectoyassociated sequences
 5.3 Trajectoryassociation decoding in a recurrent network
 5.4 Simple Recurrent Networks
 5.5 Training and productive capacity results
 5.6 Trajectories in continuous space
 5.7 Hierarchical HRNs
 5.8 Discussion
 5.9 Conclusion
 6 Estimating analogical similarity
 6.1 An experiment with shap configurations
 6.2 Models of analogy processing
 6.3 Analogies between hierarchical structures
 6.4 Why HRR similarity reflects structural similarity
 6.5 Contextualized HRRs
 6.6 Interpretations of an analogy
 6.7 Discussion
 6.8 Coclusion
 7 Discussion
 7.1 Holistic processing
 7.2 Chunks and the organization of longterm memory
 7.3 Convoltution, tensor products, associative operators, conjunctive coding, and structure
 7.4 Implementation in neural tissue
 7.5 Weaknesses of HRRs
 7.6 Conclusion
 Appendix A Means and variances of similarities between bindings
 Appendix B The capacity of a superposition memory
 B.1 Scaling properties of a simple superposition memory
 B.2 A superposition memory with similarity among the vectors
 B.3 Limitations of superposition
 Appendix C A lower bound for the capacity of superposition memories
 Appendix D THe capacity of convolutionbased associative memories
 D.1 A memory for pairedassociates with no similarity among vectors
 D.2 A memory for pairedassociates with some similarity among vectors
 D.3 Comments on the variable0binding memory with similarity
 Appendix E A lower bound for the capacity of convolution memories
 Appendix F Means and variances of a signal
 Appendix G The effect of normalization on dotproducts
 G.1 Means and variances of dotproducts of vectors with varying similarity
 G.2 Means and variances of dotproducts of convolution expressions
 Appendix H HRRs with circular vectors
 H.1 Means and variances of dotproducts of vectors with varying similarity
 H.2 Means and variances of dotproducts for similarity and decoding
 H.3 Results on analogical similarity estimation
 Appendix I Arithmetic tables: an example of HRRs with many items in memory
 I.1 THe objexts and relations
 I.2 Queries to the memory
 I.3 Using fast estimates of the dotproduct
 References
 Subject Index
 Author Index
4/1/2003
ISBN (Paperback): 1575864304 (9781575864303)
ISBN (Cloth): 1575864290 (9781575864297)
ISBN (Electronic): 157586939X (9781575869391)
Subject: Cognitive science

Distributed by the University of Chicago Press
