← List all talks ...

Learned Data Compression

Ashish Khisti – Professor, University of Toronto

Fri, 25-Feb-2022 / 2:00pm / https://stanford.zoom.us/j/92716427348

Abstract

In recent years, deep learning methods are being increasingly used in both lossless and lossy data compression. Deep generative models when applied to (lossy) image compression tasks can reconstruct realistic looking outputs even at very low bit-rates, when traditional compression methods suffer from significant artifacts. Deep learning has also been applied in lossless compression tasks involving high dimensional data with intractable distributions, in techniques such as bits-back coding.

In this talk, we will revisit the rate-distortion-perception (RDP) tradeoff that underlies the performance of learned image compression and study the quadratic-Gaussian source model in detail. Motivated by the Gaussian case, we will introduce a notion of universal encoded representations, where the same compressed representation can be used to simultaneously achieve different operating points on the distortion-perception tradeoff. We will demonstrate through both theoretical analysis and experimental results involving deep learning models that near-optimal encoding representations can be constructed for a broad class of source models.

We will next consider a lossy compression setup where the reconstruction is additionally constrained to a given distribution that could be different from the source distribution. Our proposed setup generalizes the RDP framework and is also related to the problem of optimal transport. We will demonstrate how our information theoretic analysis can be validated using deep learning models applied to image restoration tasks such as de-nosing and super-resolution over compressed data representations.

Finally, we will consider the setting of lossless data compression and present a variation of the bits-back coding algorithm that can be applied for efficiently compressing multisets, where the order of the data sequence does not have relevance.

The talk is based on the following papers: 1. G. Zhang, J. Qian, J. Chen, and A. Khisti. “Universal Rate-Distortion-Perception Representations for Lossy Compression” NeurIPS 2021 2. H. Liu, G. Zhang, J. Chen, A. Khisti, “Lossy Compression with Distribution Shifts as Entropy Constrained Optimal Transport”, To Appear ICLR 2022

1. D. Severo, J. Townsend, A. Khisti, A. Makhzani, K. Ullrich, “Your Dataset is a Multiset and You Should Compress it Like One” NeurIPS Workshop on Deep Generative Models and Downstream Applications, 2021 (Also accepted at DCC 2022)

Bio

Ashish Khisti is a Professor at the University of Toronto, Canada. He received his undergraduate degree from the division of Engineering Science at the University of Toronto and his SM and PhD degrees from the EECS department at the Massachusetts Institute of Technology. His research interests include Information Theory, Error Control Coding, Physical Layer Security and Machine Learning. He presently serves as an Associate Editor for the IEEE Transactions on Information Forensics and Security. He was previously an Associate Editor for the IEEE Transactions on Information Theory (2015-2018) and IEEE Transactions on Communications (2011-2015).