Lita Yang

From Murmann Mixed-Signal Group

(Difference between revisions)
Jump to: navigation, search
 
(8 intermediate revisions not shown)
Line 7: Line 7:
Admitted to Ph.D. Candidacy: 2013-2014  
Admitted to Ph.D. Candidacy: 2013-2014  
-
'''Email''': [mailto:yanglita@stanford.edu yanglita AT stanford DOT edu]<br> '''Research''': Energy-efficient, Approximate Memory for Error Tolerant Systems<br>  
+
'''Email''': [mailto:yanglita@stanford.edu yanglita AT stanford DOT edu]<br>  
-
<br>
+
'''Research''': Approximate Memory for Energy-Efficient Machine Learning Algorithms&nbsp;
-
As transistor scaling is coming to a halt, systems today are becoming more and more power limited. Given recent trends in increasing network sizes and the need to process more data (such as Deep Learning and Big Data applications), the cost to store and move data around in a system can far exceed the computation cost by energy overheads over 80%.
+
<br> As transistor scaling is coming to a halt, systems today are becoming more and more power limited. Given recent trends in increasing network sizes and the need to process more data (such as Deep Learning and Big Data applications), the cost to store and move data around in a system can far exceed computation costs, prohibiting hardware implementations of machine learning algorithms in embedded applications.  
-
<br>Recently, there has been an emergence of interest in the field of Approximate Computing, which explores the performance (accuracy) of an algorithm with reduced precision. Convolutional Neural Networks (ConvNets) are one example of a class of stochastic algorithms which can tolerate reduced precision for little degradation in algorithmic performance. Recent work in hardware accelerators for ConvNets and simulations in fixed-point representation indicate we can use much lower bit precisions than conventional GPU 64/32-bit floating point precision. Since power and area scale with precision (number of bits), this implies we can achieve significant energy/area savings by exploiting the algorithm’s tolerance to noise.
+
Recently, there has been an emergence of interest in the field of Approximate Computing, which explores the performance (accuracy) of an algorithm with reduced precision. Convolutional Neural Networks (ConvNets), the current top performing image classification networks, are an example of a class of stochastic algorithms which can tolerate reduced precision for little degradation in algorithmic performance.&nbsp;We propose to reduce the system energy by exploiting error tolerance of the algorithm using approximate memory. From a memory designer’s perspective, this is rarely considered a viable option since most general purpose systems require robust storage and communication.  
-
<br>We propose to reduce the system energy by exploiting error tolerance of the algorithm using approximate memory and interconnect communication design. From a memory designer’s perspective, this is rarely considered a viable option since most general purpose systems require robust storage and communication. By designing application-specific memory, however, we can achieve orders of magnitude improvement in energy, area, and performance. We propose to further improve the system’s classification performance by embedding known circuit nonidealities (i.e noise and coupling) into the algorithm’s training phase to better model translation from software to hardware. <br><br>  
+
We have shown that ConvNets are tolerant to bit flips and reduction in precision [1]. To accurately quantify the effectiveness of accepting bit errors under reduced memory supply voltages during ConvNet inference and training, we took measurements on an 8KB SRAM test chip in 28nm UTBB FD-SOI CMOS for emulating memory bit errors at low voltages [2]. The results demonstrate supply voltage reduction of 310mV on a MNIST ConvNet, resulting in 5.4x leakage power savings and 2.9x memory access power savings at 99% of floating-point classification accuracy, with no additional hardware cost.  
 +
 
 +
<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;[[Image:HILMemory.png]]<br>  
<br>  
<br>  
-
<br>
+
[1] B. Murmann, D. Bankman, E. Chai, D. Miyashita, and L. Yang, "Mixed-Signal Circuits for Embedded Machine-Learning Applications," Asilomar Conference on Signals, Systems and Computers, Asilomar, CA, Nov. 2015.
-
<br>
+
[2] L. Yang and B. Murmann, "SRAM Voltage Scaling for Energy-Efficient Convolutional Neural Networks," International Symposium on Quality Electronic Design (ISQED), Santa Clara, CA, Mar. 2017, pp. 7-12.

Latest revision as of 07:51, 19 July 2017

LitaYang.jpg

BSEE, California Institute of Technology, 2012

MSEE, Stanford University, 2015

Admitted to Ph.D. Candidacy: 2013-2014

Email: yanglita AT stanford DOT edu

Research: Approximate Memory for Energy-Efficient Machine Learning Algorithms 


As transistor scaling is coming to a halt, systems today are becoming more and more power limited. Given recent trends in increasing network sizes and the need to process more data (such as Deep Learning and Big Data applications), the cost to store and move data around in a system can far exceed computation costs, prohibiting hardware implementations of machine learning algorithms in embedded applications.

Recently, there has been an emergence of interest in the field of Approximate Computing, which explores the performance (accuracy) of an algorithm with reduced precision. Convolutional Neural Networks (ConvNets), the current top performing image classification networks, are an example of a class of stochastic algorithms which can tolerate reduced precision for little degradation in algorithmic performance. We propose to reduce the system energy by exploiting error tolerance of the algorithm using approximate memory. From a memory designer’s perspective, this is rarely considered a viable option since most general purpose systems require robust storage and communication.

We have shown that ConvNets are tolerant to bit flips and reduction in precision [1]. To accurately quantify the effectiveness of accepting bit errors under reduced memory supply voltages during ConvNet inference and training, we took measurements on an 8KB SRAM test chip in 28nm UTBB FD-SOI CMOS for emulating memory bit errors at low voltages [2]. The results demonstrate supply voltage reduction of 310mV on a MNIST ConvNet, resulting in 5.4x leakage power savings and 2.9x memory access power savings at 99% of floating-point classification accuracy, with no additional hardware cost.


                    HILMemory.png


[1] B. Murmann, D. Bankman, E. Chai, D. Miyashita, and L. Yang, "Mixed-Signal Circuits for Embedded Machine-Learning Applications," Asilomar Conference on Signals, Systems and Computers, Asilomar, CA, Nov. 2015.

[2] L. Yang and B. Murmann, "SRAM Voltage Scaling for Energy-Efficient Convolutional Neural Networks," International Symposium on Quality Electronic Design (ISQED), Santa Clara, CA, Mar. 2017, pp. 7-12.

Personal tools