The course outline and slides/notes/references (if any) will be provided on this page (see introductory lecture slides for tentative course outline).
Lecture 1, Jan 8: Introductory lecture [slides]
Lecture 2, Jan 10: Information Measures
Lecture 3, Jan 15: Asymptotic Equipartition Property (AEP) and nearlossless compression
Lecture 4, Jan 17: Variable length compression: Huffman code, KraftMcMillan inequality
Lecture 5, Jan 22: Dmitri Pavlichin  Entropy lower bound, genome + tabular compression, general tips [slides] [Bioinformatics] [IEEE Spectrum]
Lecture 6, Jan 24: Shubham Chandak  Stationary processes and entropy rate (Ref: Cover & Thomas 4.1, 4.2), Universal compressors  LZ77 (Ref: C&T 13.4, 13.5), Application to genomic data compression [Slides][Paper]. Additional resources on convergence of LZ [EE376C notes].
Lecture 7, Jan 29: Reliable communication I: channel capacity, examples
Lecture 8, Jan 31: Bianka Hofmann: From STEM to STEAM [slides]
Lecture 9, Feb 5: Mert Pilanci: Polar Codes [slides] [SC decoding example]
Lecture 10, Feb 7: Reliable communication II: Fano's inequality, channel coding converse
Lecture 11, Feb 12: Reliable communication III: Joint AEP, Achievability using random coding; Introduction to lossy compression
Lecture 12, Feb 14: Lossy compression I: ratedistortion function, examples
Lecture 13, Feb 19: Lossy compression II: direct and converse parts of main result
Lecture 14, Feb 21: Joint sourcechannel coding and the separation theorem
Lecture 15, Feb 26: Rosa Cao: The Allure of Informational Explanations [slides]
Lecture 16, Feb 28: Kedar Tatwawadi: Information Theory meets Machine Learning [slides] [SCW’19 panel]
Lecture 17, Mar 5: Irena FischerHwang: Image Compression: From theory to practice [slides], Additional resources: [GIF], [PNG 1, PNG 2], [JPEG], [Human compression]
Lecture 18, Mar 7: David Tse: Deconstructing the Blockchain to Approach Physical Limits [slides] [paper] [short Youtube video]
Lecture 19, Mar 12: Yihui Quek: Intro to quantum information theory. References: Nielsen and Chuang: Sections 1.1, 1.2, 1.3 (background), 1.3.7 (teleportation), 2.3 (superdense coding). Wilde: Section 6.1, 6.2.3 (superdense coding), 6.2.4 (teleportation).
Lecture 20, Mar 14: Jonathan Dotan: Science of information in entertainment and web 3.0 (Related talk on April 11) (Related course next quarter: GSBGEN 578)
Concluding remarks by instructor [slides]
Last year's (Winter 2018) course material
The lecture notes from last year are provided below and the lecture videos recorded by SCPD are available on Canvas. The timestamps connecting the topics to the lecture video are available here. The textbook used last year was Elements of Information Theory.
Jan 9: Introduction to Information Theory I
Jan 11: Introduction to Information Theory II
Jan 16: Information Measures
Jan 18: Asymptotic Equipartition Property (AEP)
Jan 23: Variablelength Lossless Compression
Jan 25: KraftMcMillan Inequality and Huffman Coding
Jan 30: Optimality of Huffman Codes, Communication and Channel Capacity
Feb 1: Channel Capacity, Information measures for Continuous RVs
Feb 6: AWGN channel, Joint AEP
Feb 8: Channel Coding Theorem: Direct Part
Feb 13: Channel Coding Theorem: Converse Part
Feb 15: Lossy Compression and Rate Distortion Theory
Feb 20: Method of Types
Feb 22: Sanov's Theorem
Feb 27: Strong, Conditional and Joint Typicality
Mar 1: Strongly Typical Sequences and Rate Distortion
Mar 6: Strongly Typical Sequences and Rate Distortion 2
Mar 8: Joint SourceChannel Coding
Mar 13: Joint SourceChannel Coding 2, Slides
Mar 15: Information Theory in Machine Learning
Midterm
Final
