The Information Bottleneck: A Unified Information Theoretic View

Shlomo Shamai (Shitz)
Professor, Viterbi EE Department, Technion
Date: Jun. 19, 2020 / Time: 1:15pm / Room: Packard 202

Abstract

This talk focuses on connections between relatively recent notions and variants of the Information Bottleneck and classical information theoretic frameworks, such as: Remote Source-Coding; Information Combining; Common Reconstruction; The Wyner-Ahlswede-Korner Problem; The Efficiency of Investment Information; CEO Source Coding under Log-Loss, Hypothesis Testing Error Exponent and others.

We overview the upink Cloud Radio Access Networks (CRAN) with oblivious processing, which is an attractive model for future wireless systems and highlight the basic connections to distributed Gaussian information bottleneck framework. For this setting, the optimal trade-offs between rates (i.e. complexity) and information (i.e. accuracy) in the discrete and vector Gaussian schemes is determined, taking an information-estimation viewpoint. Further, the performance cost of the simple 'oblivious' universal processing in CRAN systems is exemplified via novel bounding techniques.

The concluding overview and outlook addresses in a unified way the dual problem of the privacy funnel and recent observations on the additive noise channels with a helper. Connections to the finite block length bottleneck features (related to the Courtade-Kumar conjecture) and entropy complexity measures (rather than mutual-information) are shortly discussed. Some challenging problems are mentioned such as the characterization of the optimal power limited inputs ('features') maximizing the 'relevance' for the Gaussian information bottleneck, under 'complexity' constraints.

The talk is based mainly on joint works with A. Zaidi, I.E. Auguerri, G. Caire, O. Simeone and S-H. Park. The research of S. Shamai is supported by the European Union's Horizon 2020 Research And Innovation Programme: no. 694630.

Bio

Shlomo Shamai (Shitz) is with the Viterbi Department of Electrical Engineering, Technion---Israel Institute of Technology, where he is now a Technion Distinguished Professor, and holds the William Fondiller Chair of Telecommunications.

He is an IEEE Life Fellow, an URSI Fellow, a member of the Israeli Academy of Sciences and Humanities and a foreign member of the US National Academy of Engineering. He is the recipient of the 2011 Claude E. Shannon Award, the 2014 Rothschild Prize in Mathematics/Computer Sciences and Engineering and the 2017 IEEE Richard W. Hamming Medal. He is a co-recipient of the 2018 Third Bell Labs Prize for Shaping the Future of Information and Communications Technology.