EE 376A: Information Theory

Stanford University, Tsachy Weissman, Winter Quarter 2014-15


  • Homework 2 has been released (due on Jan 29th).

  • Solutions to Homework 1 have been posted.

  • Students are required to scribe ONE lecture each. Each student should enter their name in front of the lecture they wish to scribe here. The lecture scribe template can be downloaded here.

  • TA Office Hours have been updated

  • We will use Coursework for grade records. Please go to Coursework to look up your homework/midterm/final grades.

  • We have set up a forum in Piazza. You can sign up as a student here.

Course Overview

Information theory is the science of operations on data such as compression, storage, and communication. It is among the few disciplines fortunate to have a precise date of birth: 1948, with the publication of Claude E. Shannon's paper entitled “A Mathematical Theory of Communication”.

Our course will explore the basic concepts of Information theory. It is a prerequisite for research in this area, and highly recommended for students planning to delve into the fields of communications, data compression, and statistical signal processing. The intimate acquaintance that we will gain with measures of information and uncertainty - such as mutual information, entropy, and relative entropy - would be invaluable also for students, researchers, and practitioners in fields ranging from neuroscience to machine learning. Also encouraged to enroll are students of statistics and probability, who will gain an appreciation for the interplay between information theory, combinatorics, probability, and statistics.