Danny Bankman

From Murmann Mixed-Signal Group

Jump to: navigation, search

Bankman.png

SBEE, Massachusetts Institute of Technology, 2012
MSEE, Stanford University, 2015
Admitted to Ph.D. Candidacy: 2013-2014
Email: dbankman AT stanford DOT edu

Research: Charge Domain Signal Processing for Machine Learning

GPU-accelerated machine learning has enabled neural networks to perform useful cognitive tasks, ranging from sorting cucumbers to diagnosing skin cancer [1, 2, 3]. There now exists a trend to push machine intelligence from cloud to edge due to concerns of latency, bandwidth, and privacy. This research focuses on energy-efficient hardware architectures and circuits needed to realize machine intelligence at the edge [4, 5, 6].
The BinaryNet algorithm for training neural networks with weights and activations constrained to +1 or -1 [7] has two key advantages for hardware implementation. First, multiplications are reduced to XNOR gates. Second, storage requirements are dramatically reduced (~1MB), eliminating the off-chip DRAM energy bottleneck. We recently validated a 28nm CMOS prototype of a mixed-signal binary CNN processor with all memory on chip [8]. We made several "CMOS-inspired" changes to the original BinaryNet topology, in order to simplify logic and interconnect at the interface between memory and compute. By customizing BinaryNet for a near-memory, mixed-signal compute fabric, we achieved 3.8μJ per classification at 86% accuracy on the CIFAR-10 dataset, demonstrating an always-on energy budget for an inference task representative of real-world applications. Our hardware architecture overcomes the memory energy bottleneck by exploiting the data parallelism and parameter reuse inherent to CNNs, and addresses the issue of arithmetic computation using an energy-efficient switched-capacitor neuron.

Architecture.jpg Msbnn_die.jpg

References

[1] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” in Advances In Neural Information Processing Systems, 2012.
[2] "How a Japanese cucumber farmer is using deep learning and TensorFlow" [Online]. Available: https://cloud.google.com/blog/big-data/2016/08/how-a-japanese-cucumber-farmer-is-using-deep-learning-and-tensorflow.
[3] A. Esteva*, B. Kuprel*, R. Novoa, J. Ko, S. Swetter, H. Blau, S. Thrun, "Dermatologist-Level Classification of Skin Cancer with Deep Neural Networks," Nature, vol. 542, no. 7639, pp. 115-118, February 2 2017.
[4] D. Bankman and B. Murmann, "Passive charge redistribution digital-to-analogue multiplier," Electronics Letters, vol. 51, no. 5, pp. 386-388, March 5 2015.
[5] B. Murmann, D. Bankman, E. Chai, D. Miyashita, and L. Yang, "Mixed-Signal Circuits for Embedded Machine-Learning Applications," Asilomar Conference on Signals, Systems and Computers, Asilomar, CA, Nov. 2015.
[6] D. Bankman and B. Murmann, "An 8-Bit, 16 Input, 3.2 pJ/op Switched-Capacitor Dot Product Circuit in 28-nm FDSOI CMOS," Proc. IEEE Asian Solid-State Circuits Conf., Toyama, Japan, Nov. 2016, pp. 21-24.
[7] M. Courbariaux*, I. Hubara*, D. Soudry, R. El-Yaniv, and Y. Bengio, “Binarized Neural Networks: Training Neural Networks with Weights and Activations Constrained to +1 or -1,” arXiv preprint: 1602.02830v3, 2016.
[8] D. Bankman, L. Yang, B. Moons, M. Verhelst and B. Murmann, "An Always-On 3.8μJ/86% CIFAR-10 Mixed-Signal Binary CNN Processor with All Memory on Chip in 28nm CMOS," ISSCC Dig. Tech. Papers, San Francisco, CA, Feb. 2017, pp. 222-223.

Personal tools