Research

Neural Networks for Joint Source-Channel Coding

Consider the problem of transmitting a structured data source such as text, audio, or video over a noisy communication channel using as few bits as possible. Seminal results in Information Theory suggest that a compression scheme followed by an error control coding scheme is optimal for certain channels for asymptotically large blocks of data.

We propose a neural network based coder that performs this source and channel coding jointly. The coding is done over the learned representation of a neural autoencoder. In the rate limited regime or when the channel is very noisy, this approach can significantly outperform information theoretic baselines. In the transmission of text, the semantic content of the sentence is preserved even when errors do occur. Also, in the severely rate limited regime, this encoder can perform abstractive summarization.

Distributed Convex Optimization with Limited Communications

We study the problem of nodes on a network collectively solving an optimization problem. This framework arises in decentralized control, tracking, or estimation problems. It also appears in the big data paradigm where a large dataset is stored at multiple locations due to concerns of scalability, memory, and privacy. Existing distributed optimization algorithms such as distributed subgradient methods assume that nodes can exchange large messages at each time instant. We propose algorithms that converge to the optimal solution even when the network communication links are band limited by using a random compression strategy for the messages exchanged.

Time-Series Parameter Estimation from Partial Observation

In the paradigm of the Internet of Things, a large number of wireless sensors can be deployed. Communication and energy bottlenecks preclude the collection of all such measurements at every instant in time. We propose random sampling and compressive sampling strategies to reduce the amount of data being transmitted and also develop algorithms that use this partial information to estimate the parameters of linear time-series that these measurements could arise from or detect the subspace that observations lie in. We show how additional priors such as sparsity or low-rankness on the system can be factored in to improve the accuracies of the estimates.